Figure 1. Components of the ALS Collaboratory
The Spectro-Microscopy Collaboratory at the Advanced Light Source is one of four projects funded by the U. S. Department of Energy to build a Distributed, Collaboratory Experiment Environments (under Contract No. DE-AC03-76SF00098).
The project is a collaboration with the University of Wisconsin-Milwaukee, with the goal of applying computer and communications technologies to the problem of remotely operating a sophisticated synchrotron-radiation beamline in the Advanced Light Source (ALS) of the Berkeley Laboratory. The original design document describes the plan for the collaboratory project, along with the various components.
The work described here is the research and development of the underlying distributed mechanisms and infrastructure required to make collaboratories a reality. Some of these mechanisms have already been developed. Several other mechanisms, such as data dissemination, resource arbitration for the sharing of experiment control, safety and security, electronic notebooks, tele-presence, and integrated user interfaces need further research and development. In addition, high-speed network support must be established, and network architectures and protocols adapted or improved for scientific collaborative environments.
We split the project into two phases. The first phase providing remote experiment monitoring capabilities and the second phase allowing remote control of the experiment. This division made it possible to implement and deploy the monitoring system quickly and get feedback from the remote researchers regarding their needs. The end product of the project will be not only a functional Spectro-Microscopy Collaboratory but also a set of tools and guidelines for building future collaboratories.
The ALS Collaboratory was organized into four layers of functionality, illustrated in Figure 1.
Figure 1. Components of the ALS Collaboratory
The bottom layer consists of the physical network with associated data link and network controls supporting the ALS Collaboratory facility. This layer provides the mechanisms for the unicast and multicast transmission of messages between sites. The next layer consists of data dissemination mechanisms, which provides various levels of reliable and ordered transmission of multicast messages. In order to offer secure access to the ALS Collaboratory, a layer containing security, authentication and authorization interacts with the the data dissemination component and higher-layer components. Contention for access to the ALS Collaboratory resources is resolved by the resource manager. The top layer is composed of the video conferencing and tele-presence, remote experiment monitoring and control, safety mechanisms, and the electronic notebook.
The access to a remote collaborative environment, whether it is for monitoring or for control of experiments, requires that security and access limitation mechanisms be in place, so that safeguards against unauthorized access and privacy of proprietary data exist.
The advent of collaboratories brings a new class of user to the ALS. These users are likely to be much more occasional and less experienced with the equipment than has been the case in the past. Collaboratories will provide network based access to very expensive equipment and must be designed to avoid several potential security and safety problems. They must also be designed to have automated equipment failure modes with sanity checks on all incoming data and be resistant to network-based tampering. With respect to remote users, one significant issue is to ensure that use-conditions of the remote resource have been met.
Multicast communication is another important element of remote monitoring, because multiple remote sites may participate in each experiment. Several different types of communication may be used, including video conferencing, dissemination of video images of the sample chamber, experiment parameters and results, collaboratory control information, and security information, as well as on-line notebook updates and transactions. Various classes of multicast communication services can exist, depending on whether reliability and order of delivery is required. The various classes of multicast services encompass the data dissemination component of this collaboratory.
Experiment parameters, for example, need to use unreliable ordered multicast mechanism, because the last received parameters are the most important and supersede the old settings. Data points to update `real-time' display of an experiment output need to use reliable ordered multicast, because all data points must be reliably received.
The safe remote operation of laboratory facilities is paramount for the remote control of experiments. The remote researchers are unlikely to be as well versed in the equipment and procedures of the ALS as researchers currently operating the facilities. Therefore, adequate safeguards must be in place to prevent unsafe operation. In addition, faults introduced in the system by loss of communications, or by unknown behavior of distributed control algorithms may pose safety risks, and additional fail-safe recovery mechanisms must also be considered.
When multiple researchers collaborate on a experiment, it is necessary to have in place arbitration mechanisms that determine which site is currently in control of the experiment. Control of experiments is distributed among sites, and thus it is possible that more than one researcher may attempt to control a given experiment parameter or device.
Successful remote operation and collaboration requires rich multimedia conferencing and tele-presence capabilities. Current network and video conferencing technology can be used to provide remote access to the Advanced Light Source thus creating a laboratory environment at the remote sites. This capability provides increased access to the facility by users and increased opportunity for collaborations between experimenters.
A problem we encountered early in the project was the need to have someone at the Advanced Light Source "babysit" the videoconferencing feeds. Since the video feed takes up a large amount of bandwidth during actual use, we need to be able to scale back and ramp up the transmission rates depending on whether there are remote users watching the video. With the current videoconferencing tools this requires a user at the source of the transmission actively changing the transmission rate. Since there are often times when there is noone there or the personnel are busy or untrained this is infeasible and leads to frustration of the remote users. A conference control tool which provides ability to change the parameters of a videoconference transmission originating at a remote site is required.
Collaborating researchers also need a whiteboard to draw on, display postscript files, display screen images cut from elsewhere on the screen and to annotate information. In addition there are many analysis tools which are not installed at all of the sites that need to be shared by the researchers. A televiewer application that allows viewing and operation of software interfaces not installed locally is required.
Video conferencing should not be a burden to the researchers conducting experiments in a collaborative fashion. The interfaces to the tools should be natural to use. This includes audio and video hardware that allows the remote users to feel like full participants in the experiment.
Video conferencing alone does not offer adequate tele-presence to the participants of the collaboratory. Beam line 7 covers a large area consisting of the beamline itself and several computers and readouts. To provide the remote researchers with the ability to walk around the floor and view the various instrument settings, a variety of cameras should be available---remotely controlled robotic cameras for area monitoring, fixed cameras at the workstations, and data cameras for monitoring of the sample chamber. The various video sources can be switched using a remotely controlled video switch, and distributed via multicast mechanisms to the participating researchers. A remote researcher should also have a video camera mounted by the remote workstation to transmit video of himself to the ALS and to the other collaborators.
Two-way radio communication can be used at the ALS to provide communication between the researchers at the beamline. The remote researchers can participate in the radio discussion by patching one of the radios into a workstation connected to the MBone.
The components of the collaboratory relate directly or indirectly to network capability and infrastructure. There are several separate streams of data within the ALS Collaboratory environment. Some data streams carry experiment parameters, experiment results, and control and coordination in formation. Others carry multiple videoconferencing sessions, the video from the sample chamber camera, so that the remote researchers are provided with feedback on experiments.
The data transmission rate requirements at BL7 are between 8 - 280 Kbits/second. The video and image transmission rate requirements are 2-10Mbits/second. An additional video signal provided to monitor the sample chamber requires 2 Mbits/second. The tele-presence videoconferencing transmission rate requirements are 128Kbits/ second for each compressed video stream. In order to fulfill some of these requirements, a high-speed network support is needed. If only Internet connections are available, the quality of video and image transmission is compromised.
The network interconnections for the support of this collaboratory are shown in Figure 2.
Figure 2 
In studying the ALS BL7 experiment note taking methods we have been able to determine many of the characteristics needed in the on-line notebook.
An electronic notebook is a medium in which researchers can record aspects of experiments that must be shared in real-time by all collaborators and that provides at least the capabilities available using a paper notebook. It is also a medium for collaborative scientific inquiry or engineering design discussions. A notebook for collaboratories may include these capabilities:
A fundamental characteristic required of electronic notebooks is ease of use. Electronic notebooks for collaboratories are inherently shared among collaborating researchers or engineers. However, it is desirable that participants may keep some of their annotations privately, thus it is paramount that it includes privacy mechanisms. Associated aspects are non-tamperability (write-once) of notebooks, and their use as patent records for inventions or as proof of conformance to procedures. Multiple notebooks may be necessary for each collaboration effort, and each researcher or engineer may need to access notebooks of different collaboratories. Tools to coordinate access to multiple notebooks are needed.
One aspect of the notebook is the ability to collect and analyze large volumes of data as well as high speed data streams such as live video sources. There are several aspects to this problem: the basic ability to handle the bandwidths and volume of data involved; the mechanisms to bring computational capability to bear on the high speed data streams, and; the practical issues of system architectures that can be easily assembled and afforded by the scientific community. Higher level organization of blocks of data in order to allow analysis, query, and visualization of scientific data associated with the ALS experiments is required.
Notebooks for remote collaboration require the interoperability and integration of a multitude of technologies, and the use of heterogeneous computer and communication systems. We envision that the following technologies will need to be integrated in order to build the electronic notebook:
 Lawrence Berkeley National Laboratory Home Page. |
 Advanced Computing for Science Department Home Page. |
This document was last updated on October 04, 1996, and is located at http://www.dsd.lbl.gov/~deba/ALS.DCEE/summary.html.
To report Web page problems, e-mail webmaster-george@george.lbl.gov. To request further information about ACS activities, e-mail DAAgarwal@lbl.gov. For suggestions about content, e-mail DAAgarwal@lbl.gov.
Support Credits identify the funding sources and the organizational context of the work described in this document.
Disclaimers and Legal Notices apply to this document and its uses, and access to and uses of the system on which it resides at LBL. Like all Web pages, it may be linked with pages and systems beyond LBL jurisdiction. Their contents and uses are the responsibility of their authors and administrators. .
Sonia Sachs (SRSachs@lbl.gov)and Deb Agarwal (DAAgarwal@lbl.gov are responsible for this WWW document.