Science experiments via telepresence at a synchrotron radiation source facility

Station 9.8 is one of the most oversubscribed and high-throughput stations at the Synchrotron Radiation Source, Daresbury, whereby awarded experimental time is limited, data collections last normally no longer than an hour, user changeover is normally every 24 h, and familiarity with the station systems can be low. Therefore time lost owing to technical failures on the station has a dramatic impact on productivity. To provide 24 h support, the application of a turnkey communication system has been implemented, and is described along with additional applications including its use for inter-continental classroom instruction, user training and remote participation.

1. Introduction

The need to study difficult samples which are normally too small or too weak to be analysed efficiently on laboratory systems was the principal driving force behind the success of the small-molecule crystallography (SMX) facilities at the SRS (Cernik et al., 1997). However, with hardware advancements in the home laboratory such as the application of CCD detectors and X-ray optics, the necessity to use synchrotron light to undertake many experiments decreased, but the number of applications for beam time did not. More difficult problems were envisaged and, with the change in emphasis from conventional samples to more bespoke experiments, the focus of the station altered appropriately. When station 9.8 underwent a hardware refit in 2003, it was brought back on a par with laboratory detector technology and saw a drop in data collection times from 6 to 1.25 h. This had an immediate impact on the way user beam time was awarded and scheduled. Typical four-day blocks changed to four one-day blocks and interaction with the hardware by the user increased. The result was a significant increase in out-of-hours support and, with that, the ability to remotely direct the user whilst being able to see the state of the hardware became paramount. To fulfil this requirement the application of a turnkey system in the form of the AXIS 214 PTZ (https://www.axis.com/products/cam_214/ ) camera server system was implemented.

Clearly, remote access or telepresence technology can be applied in numerous ways and, when combined with the limited-access nature of central facilities time, core applications become apparent: out-of-hours and remote technical support; remote/guided participation; user training and teaching.

1.1. Out-of-hours and remote technical support

The technical and experience levels of users of the SMX facility differ greatly. They range from groups consisting of only two PhD students to six-member experimental teams with PDRAs, academic staff and senior professors. It is therefore rare that any single team member will be totally familiar with the equipment, hardware and the station protocols. Whilst every effort is made to train the users to allow them to function without the aid of the station scientist, it is impractical and unsafe to provide limited training on the more complex and administrative attributes of the software and hardware, e.g. crash recovery.

Traditionally the station scientist had to return to the laboratory or attempt to diagnose and rectify faults via phone conversations, often with little additional information. The exact cause of a fault could often be masked by the user self-troubleshooting and the complexity of the station hardware leading to confusion in terms of origins and directions. Once alerted to a problem, the station scientist can remotely log in to the camera system, observe the state of the hardware and direct the users as if within the room. They can also determine if the fault is too great to continue and assess the likely length of time to recovery.

1.2. Remote/guided participation

Firstly, there are those experiments where wide consultation amongst team members is required during an experimental run, and those team members can be distributed between continents. Secondly, the gulf between those countries that have synchrotron and neutron facilities and those that do not can be lessened if much wider participation could be organized. Thirdly, more complex experiments might be attempted if direct participation is established.

1.2.1. Short review of remote experiment systems

Pre-delivery of samples and robotic sample loading has been pioneered especially at Stanford Synchrotron Radiation Laboratory (SSRL) (González et al., 2005). This has thus far not included internet audio link-up owing to firewall restrictions on the software that can be permitted at Stanford Linear Accelerator (SLAC) (S. M. Soltis, personal communication). In Australia a web-based system, including access specifically to a Bruker diffractometer, as described here, has been initiated (Atkinson et al., 2006). At centralized facilities with conventional X-ray source diffractometers both in the UK and in the USA, a seamless distributed computing approach has been used involving the GRID to provide remote secure visualization, monitoring and interaction with the laboratory and the diffraction experiment, supervision and input to the data workup and analysis processes, and to enable dissemination and further use of the resulting structural data (Coles et al., 2005; Bramley et al., 2006). In the microscopy community there is the Argonne National Laboratory `AAEM TPM Collaboratory TelePresence Server' (https://tpm.amc.anl.gov/ ), which is one of the remote collaboration tools enabled by means of a server for `TelePresence Observation Modes in Scientific Research Environments'. At this microscopy facility each of their TelePresence server windows is organized as a module, which is linked into a thin client web-enabled page.

Possibly the earliest project development is that at the University of California San Diego where their Telescience project (https://telescience.ucsd.edu/ ) emerged from the early efforts of researchers at the USA National Center for Microscopy and Imaging Research (NCMIR) to remotely control bio-imaging instruments. In particular, in 1992, NCMIR demonstrated the first system to control an electron microscope over the Internet from the SIGGraph Conference in Chicago. This proof-of-concept system enabled conference attendees to interactively acquire and view images via remote control of one of the intermediate voltage electron microscopes at the NCMIR. In 1999 their web-based telemicroscopy system was released and researchers were able to use the remote interface to acquire data. It became clear, however, for a complete remote research scenario, that the ability to remotely acquire data had, in their view, to be closely coupled to data computation and computer storage resources. Their current Telescience project was specifically developed to address that issue.

1.3. Impact of Telepresence on user training and teaching

The training of new researchers to synchrotron radiation and neutron facilities will be expedited offering flexibility in training times and number of people trained. A specific situation of interest was of the instrument scientist being at station 9.8 linked into a classroom on a different continent, which was demonstrated within the NATO Advanced Training Workshop held immediately prior to ECM24 in Marrakech (August 2007).

2. Station 9.8 experimental details

In accordance with SRS safety protocols for SMX beamlines, there is no direct control of the station hardware (shutter, diffractometer etc.) from outside of the user work area. Therefore a physical presence of one or more members of the research collaboration is required on the beamline. The turnkey system provides embedded video and audio as an encoded video stream and returns audio to participants via proprietary web-broadcast plug-ins. Camera systems (with audio) are located in both the instrument `hutch' and the data acquisition room. Access is controlled via IP address level security at the camera and facility firewalls. We now detail the various aspects at SRS station 9.8 (Fig. 1 shows a block diagram scheme).

Figure 1 Block diagram depicting security, audio and control layout for AXIS camera systems.

2.1. Telepresence experimental approach used

Two AXIS-214 PTZ media servers (Figs. 2a and 2b) provide both video and audio/verbal support. One is located in the station 9.8 experimental/`hutch' area and the other in the data acquisition room. Control of the camera systems is achieved by the use of Microsoft Internet Explorer and the `axis media control' ActiveX plug-in. A simple command queue is built into the media server allowing direction, zoom and other functions to be queued on `a first-come first-served' basis when more than one remote participant wishes to operate the system. The internet bandwidth requirements are quite straightforward by today's standards, and are as follows: outgoing communication, 1 Mbit s⁻¹; incoming communication required, 5 Mbit s⁻¹. The costs of providing the two cameras plus microphone and speakers is very modest at approximately GBP 2000 (USD 4000).

Figure 2 Telepresence camera (a) and close-up view (b). (The same type of camera is employed in the instrument hutch and in the data acquisition hutch.)

The camera systems are self-contained media servers which `publish' and allow direct control of the camera and audio content. Access to this content can be achieved via a number of routes at a number of user levels. Users are given accounts to the website which govern their access rights from `simple viewer' (i.e. someone who can see and hear but not move the camera) to `operators', who can move the camera's pan/tilt/zoom. The camera movement is simple `point and click', making operating the camera and following the experiment extremely simple. A menu of pre-programmed positions of each camera is an especially effective approach for quickly panning between key locations and views within the hutch or work area that are required.

Audio/verbal contact has two modes: full-duplex, providing real-time two-way audio and streaming video to a single remote viewer (the standard technical support mode), and half-duplex, providing real-time streaming of video and audio to all viewing participants but with push-to-talk capability for remote viewers to broadcast so that multi-site communications are possible and therefore not just experimental access to a single remote participant. For large-scale `conferencing' events the camera output can be viewed using the teleconferencing facilities within an organization, even directly into Microsoft Windows Mediaplayer.

2.1.1. Layout of station 9.8 AXIS cameras

As the principal use of the `hutch' camera was for technical assistance (predominantly out of hours) to users, AXIS camera 1 was sited where a clear view of the diffractometer and jacking table, door and X-ray optics could be achieved when utilizing the 240° rotation and 120° tilt of the camera (Figs. 3a and 3b). The data acquisition system (AXIS camera 2) was located so that it could see the optical microscope display screen (Fig. 3c) and the data analysis and control computer screens and where, with the 18× optical zoom, text from those screens could easily be read (Fig. 4).

Figure 3 (a) Telepresence camera view of the in-hutch diffractometer at SRS 9.8; (b) zoom-in confirming choice of collimator beam size of 0.2 mm; (c) Telepresence camera view of the light microscope thus allowing joint decision-making regarding the crystal sample selection between remote research teams. The carotenoid needle crystal in the centre of the LCD display is of approximate dimensions 50 × 5 × 5 µm.

Figure 4 Text viewed for data processing area AXIS camera 2 at 18× optical and 2× digital magnification.

This then allows the remote participant the ability, respectively, to view and join with discussion of choosing of a crystal sample, see the diffractometer and any ancillary equipment located in the interlocked hutch area and to read text on the computer screens in Daresbury during data analysis. Thus, the remote participant can join in with setting of parameters such as data exposure/collection times.

Recently the system was used in experiments involving two research centres (UK and Morocco), both pursuing research interests in colouration chemistry (Bartalucci et al., 2007; Thalal et al., 2005). An additional fruit of the experiment is the chance to plan further experiments involving similarly weakly scattering crystals from our laboratories.

2.1.2. Experimental recording

Additionally the camera systems allow for electronic records to be made for the entire experiment. These are either in the simple form of time- and camera-stamped snapshots or via the recording of MPEG2/4 encoded movies, which could be used for subsequent user or student training/classroom-teaching. [See supplementary deposition example (1), `crystal mounting and centring on the diffractometer', and example (2), `on screen, i.e. in situ, diffraction data analysis, crystal space-group determination and structure solution' (Reference: WL5143 ). Services for accessing these data are described at the back of the journal].

3. Discussion

3.1. Global collaborative sharing of facilities and data initiatives

`Telepresence' access for collaborative research studies between the remote centres and the central synchrotron radiation and/or neutron facility will allow more optimal direct collaborative participation. The procedures will be of wide interest as a future user model within the Global Information Commons for Science Initiative (https://www.codata.org/wsis/GICSI-prospectus.html ). This is sponsored by CODATA, ICSTI (of which IUCr is a member), INASP, the World Data Centers (WDC) and ICSU, in collaboration with the InterAcademy Panel on International Issues (IAP), the Academies of Science in Developing Countries (TWAS), the OECD, UNESCO, and Science Commons. The GICSI prospectus includes among its goals `cooperative sharing of research materials and tools among researchers'. Therefore the development of Telepresence also details an important step forward in that context.

3.2. Remote technical support and directed experiments

By providing a two-way audio and direct video stream of the station and area, the station scientist (or external collaborator) is able to remotely direct the local user freely. Remote control of the camera position allows the viewer to follow individuals, experiment displays and even to read hardware parameters. Thus the user is not tied to the telephone and its limited range, and can carry, use both hands and move freely throughout the environment as directed. Indirectly this system reduces the need to travel and therefore has the ability to reduce the `carbon footprint', whilst improving and speeding up the interaction of the station scientist.

4. Conclusion

The Telepresence technology and the set-up at SRS station 9.8 have been described, for which the costs of the camera and the installation are minimal. The use of propriety plug-ins and turnkey solutions removes the reliance on limited internal network and programming support whilst providing a solution to two-way audio not dependent on peer-to-peer programs, which are often not allowed on central facilities machines. The relevance to user training and connecting the facility directly into the student classroom has been demonstrated within a NATO Advanced Training Workshop (Marrakech, August 2007) along with the application to remotely directed beam time, Manchester, UK, and Marrakech, Morocco, in various tests and beam time undertaken in May and June 2007.

Footnotes

‡Present address: Institut Laue-Langevin, BP 156, 38042 Grenoble CEDEX, France.