![Computing Sciences at Berkeley Lab Navigation Banner [Load to Use]](quarterly-reports_files/CShomebanner.gif)
Pervasive Collaborative Computing Environment Project (PCCE)
|
|
| Distributed
Systems Department Pervasive Collaborative Computing Environment Project (PCCE) |
Personnel
We have also worked closely with the Condor implementation team to install and begin debugging of a SOAP-enabled Condor scheduler. This effort has focused on implementation of a client web service interface capable of direct job submission from a BPEL workflow to the Condor scheduler. The complexity of the present (alpha) SOAP job submission API has slowed integration of these two packages. Further discussions with the Condor team has led us to create an intermediate job submission manager that is capable of presenting a fairly simple job submission and control view to the BPEL workflow engine while managing the complexity required for Condor job submission (class ad descriptions, etc.). Initial implementations of this "job manager" web service are functioning and further integration and elaboration (user job submission profiles, OGSA-based job submission capabilities, etc.) are being added. Planning and implementation of a second generation SOAP interface for Condor job submission are in progress.
Work this quarter on the secure messaging and presence capability concentrated on switching to a Jabber-based solution for these capabilities and implementing an archiving server that can run as a client. We are now fully switched to a Jabber-based server and clients which means we are now using the XMPP standard protocols. Our concentration now will be on providing certificate and password based security in the Jabber software. A Jabber server and an archiving server are both now running and are available for use by external groups.
Personnel
Our development effort this quarter has focused on our collaborative workflow capabilities. Toward this end, we have configured a BPEL testbed at LBNL consisting of a BPEL engine. We have also verified the basic BPEL engine functionality (sequences, parallel flow, external web service invocation, etc.). We are also implementing monitor and status web services to instrument BPEL workflow execution. This will allow graphical display of both BPEL workflow topology and progress of a given workflow instance. Note: monitoring and status functionality are not addressed in IBM's "research" BPEL implementation.
We also defined a BPEL model for stopping and restarting (under user control) user workflows. The goal is to provide a framework within which users can examine results or exceptions and steer subsequent operations within the workflow. The ANL graph editor is being adapted to parse and display BPEL xml workflow descriptions. Graphics produced by the adapted graph editor will be used to describe state and progress of individual BPEL workflows.
In preparation for delivery of the web services interface to Condor, we have installed Condor on one of our test machines and with much help from UWisc gotten the installation working. UWisc personnel are busy developing the web Services interface to Condor and plan to deliver a first prototype at SC'03.
Personnel
In addition, this quarter we began work on an ability to archive the secure messaging sessions. To avoid the issues with maintenance of authorization information for a central repository, we have designed an archive server that users can run and control individually or in groups. This allows the authorization model to remain much more intuitive for the user.
This quarter our papers describing the incremental trust model and the use of group communication capabilities to support dynamic and ad hoc formation of collaborating groups were accepted and presented at the Workshop on Advanced Collaborative Environments. Also, a paper describing how PCCE can be used to provide a high energy physics collaboration environment was presented at the 2003 Conference for Computing in High Energy and Nuclear Physics.
Personnel
During this quarter we have also expanded our involvement with the International Committee for Future Accelerators - Beam Dynamics Panel, Working Group on Remote Experiments in Accelerator Physics by publishing a paper describing our ideas about how remote operational access to accelerators might be accomplished. This paper was presented at the 2003 Particle Accelerator Conference in May.
Our work on integrating InterGroup group communication capabilities into the PCCE infrastructure led to a paper which we submitted to the workshop on Advanced Collaborative Environments. This paper describes how integration of the group communication capabilities will allow the dynamic and ad hoc formation of collaborations without requiring servers to be running or reachable.
We have also expanded the capabilities of the secure messaging software to provide support for asynchronous messaging. Users can leave a message for a user that is currently off-line and can also turn messages in a chat session into an asynchronous note if the user never joined the chat. This feature is an essential part of supporting users in different time zones. Alos this quarter we have redesigned the login process. We were having trouble with the logins taking too long when the user was far away and this being frustrating. In response we parallelized many of the startup operations and begin to bring up elements of the user interface immediately so users have an improved login experience.
Personnel
We have also started the migration to a peer-to-peer model by integrating InterGroup reliable multicast capabilities into the messaging tool. We are using the InterGroup protocol for joining and exchanging messages in nonsecure venues, while continuing to use the SSL connections to the IRC server when security is required.
The messaging client interface has been deployed to a few LBNL users in the physics community and to some users at Argonne National Lab. We plan to also release our servers shortly. We are designing mechanisms that will support categorization of the users by organization and to allow access to multiple collaboration groups/servers from a single client interface. We are working with several science projects in high energy and nuclear physics interested in using the PCCE tools including: the LHC Atlas and CMS collaborations, the D0 collaboration, the CDF collaboration, and the Global Accelerator Network. We are also working to deploy the tools to NERSC support personnel.
Another activity we have been involved with this quarter is development of an operational model for our LBNL Access Grid. We have been in the process of moving to have our LBNL support personnel take over support and operation of the LBNL Access Grid. This has brought up several development and operation issues. We are working with this group to help define a business model and operational procedures for the node. We have also been active participants in the Access Grid development communittee including attendance at Town Hall meetings and other related meetings such as SC2002.
Development of the collaborative workflow tool has progressed on several fronts. A new standard for expression of workflow is emerging called BPEL4WS. We are currently evaluating this standard as a replacement for WSFL. We are just beginning the process of developing a workflow prototype with the ECCE development group at PNNL. The plan is to have ECCE provide the workflow description and for our workflow tools to then take care of submitting and monitoring the jobs.
We continued to work with the GAN and LHC communities this quarter to define collaboration requirements. Progress on the GAN efforts included a field trip to the SLAC PEP II control room to observe the activity in the control room for a day and better understand the dynamics involved in running the accelerator. The PCCE project is identified as a target technology for integration into the US LHC collaboration environment. We are participating in the writing of the LHC ITR proposal describing the development efforts needed to develop the LHC collaboration environment. Deb also was served on the US LHC review committee this quarter.
Personnel
We have also spent extensive efforts this quarter interfacing with the community to better understand collaboratory requirements and to educate users. We have attended several meetings and hosted one meeting to discuss the Global Accelerator Network. This effort is planning for a next generation accelerator that they hope to operate collaboratively. We have also been interfacing with a group from LBNL that will be using the LHC soon and would like to have remote access to their instruments and collaborative capabilities. The Science of Collaboratories workshop provided an opportunity to learn about other collaboratory environments and get honest assessments of their successes and failures. The High Performance Networking workshop provided an opportunity to feed the requirements of collaboratories into the network planning for the future.
Personnel
The workflow job submission and monitoring capability is currently a commandline tool. The DAG which represents the workflow and acts as a visual aid to the present status of the workflow is generated from the internal representation of the workflow in the workflow manager. The internal representations are based on the semantics of WSFL. The visualization tool for the workflow uses the Graphviz package from ATT. We convert our WSFL representation of a particular instance of a workflow to a Graphviz dot representation, which is the input to their webdot package. Webdot can transform the workflow graph to a visual image in any standard image format. A cgi script displays the current status of the workflow through a web-enabled interface to webdot. Features such as clickable nodes for detailed information about a particular job are in the process of being implemented. Our goal is that all the services in our collaborative environment will be spawned using servlets.
We are following the increased interest in workflow systems shown in grid development forums such as Global Grid Forum. At the most recent GGF meeting held in Edinburgh, Scotland, we presented an overview and status report on our current workflow manager at the Grid Computing Environments BOF session on Workflow.
The PCCE messaging component has also been enhanced beyond its core functionality. Using an "open/closed door" metaphor, it now includes support for private as well as public discussions with invitation capabilities. The security has also been extended to work with DOE Science Grid certificates issued by ESnet. This extended version has been deployed for use and further testing within our group. We plan on incorporating asynchronous messaging in the next release.
In addition, we have been participating in discussions with the accelerator community about the possibility for collaborative operation of a particle accelerator. We are participating in the discussion workshops and are organizing a workshop which will bring together accelerator operators and collaboratory researchers to investigate collaborative tools for this endeavor.
Personnel
External user descriptions of the workflow model are also loosely based on the WSFL syntax. We have created a dagman-like syntax to aid users in constructing the scripts containing the XML workflow model. A workflow based on a pre-existing model description is initiated through a web service interface. Monitoring and control of a workflow's execution is performed in a similar fashion. We have evaluated a number of graph visualization and editing tools for use in displaying current workflow execution state and creating new workflow models. We have decided to use the Graphviz package developed by ATT labs. Work has begun on integrating graphical rendering of workflow state and status, through the Graphviz package, with a web-based workflow control system.
We have also made significant improvements to our messaging application. We have re-organized the code using a more object-oriented approach and have begun work on presence and asynchronous communication mechanisms. The revamped implementation includes a more intuitive and easier to use interface that supports presence information and locating people and channels. Users are now able to set and propagate status information (e.g. available, busy, or away). Channels can be made public or private using the metaphor of an open or closed door and other users can be invited to join the channel. The implementation of the secure communication code is also being reorganized and reworked to improve its capabilities and better integrate with the grid security infrastructure. We have completed most of this work and will be completing the rest next quarter. The new alpha version of the messaging application, which includes the core capabilities, has been tested on multiple platforms (Solaris, Linux, and Windows 2000) and has been deployed for use and testing within our group.
We are also helping to provide the infrastructure for developing and deploying web services. We have installed and are using the Apache SOAP-RPC distribution under the existing Apache web server. We have installed and are using the Java Web Services Developer Pack (JWSDP) under Tomcat. We wrote some test web services to evaluate development and deployment under each of these frameworks. There is now a stable Apache-Tomcat installation on a group workstation for developing and deploying web services under either Apache SOAP-RPC or JWSDP.
For document-sharing we have been using the BSCW. However, it is very slow on the client side in displaying its web pages. We installed and worked with Zope to evaluate its feasibility as an alternative document-sharing tool. Although Zope is faster, it is aimed at web development and web site maintenance and is not a good fit for the PCCE. We will continue to investigate existing document-sharing tools while we continue to use BSCW.
"Securing Collaborative Environments," D. Agarwal, K. Jackson, M. Thompson, Proceedings of the Workshop on Advanced Collaborative Environments, Edinburgh, Scotland, July 26, 2002.
"Security Requirements for Dynamic and Asynchronous Collaboration Environments," A working draft of the Advanced Collaborative Environments Research Group of the Global Grid Forum.
Personnel
Our successful implementation of Grid-based Web Services has prompted a re- examination of the proposed Web Services Flow Language (WSFL) as a basis for both description and management of scientific computing workflow on Grid systems. Current efforts in workflow management have been hampered by the lack of a standardized language for expressing both work activity sequencing and branching conditions. While WSFL does not represent a marked improvement over existing alternatives, it is a public specification with published XML schemas and bindings. Furthermore, the use of Web Services to invoke and control both applications, as well as the workflow engine itself, is integral to its design. We have therefore begun implementation of a WSFL-based workflow system. To date, we have implemented the required parsing and syntax utilities needed to construct a machine interpretable workflow model. Subsequent efforts will center on the construction of a workflow engine to execute and track applications based on this model. We will also be producing web-based tools to graphically show present status for both individual flow elements as well as the state of the overall process itself. In these efforts we are working closely with the Condor team at Wisconsin to integrate elements of Condor-G into the workflow infrastructure.
File sharing is an important component of collaboration. To address this need, we have obtained the license and source code for the BSCW server. This server has been installed and configured on a workstation for testing within our project. The BSCW server uses CGI scripts accessed from the browser. We have augmented the BSCW server to allow it to leverage off Grid-based certificates for authentication and authorization decisions. To limit access to workspaces, users first register with the BSCW server and subsequent access is via a client certificate or login. In order to enforce that users present either a valid certificate or a valid userid and password, we modified the BSCW source code and reconfigured the Apache web server. We also configured BSCW to use SSL connections for privacy of connection traffic.
We have made several improvements to the design of our secure chat program and are currently working on integrating the use of Grid certificates for authentication and authorization into the chat. Our secure chat has also been integrated into the BSCW by adding the launch option to the BSCW source code and pages. We have also confirmed that a single BSCW server can be used by multiple collaborations. Each collaboration's shared workspace is accessible only to authorized users. In order to integrate the BSCW with WebDAV, we are also investigating how to use the BSCW API to allow WebDAV to add and interact with documents in the BSCW workspace.
Personnel
In an effort to stay abreast of new collaboratory tool developments, we have spent effort researching, installing, and using tools such as Groove, the Collaborative Virtual Workspace (CVW), and the Basic Support for Collaborative Work (BSCW). The Groove and CVW software were initially attractive for integration into PCCE but upon investigation both were found to be poor fits with the PCCE goals. The CVW use of the MOO for the underlying server makes adding security to CVW very difficult. The Groove architecture is limited in scalability and has an inadequate security model that is not likely to be easily replaced or integrated with the Grid security infrastructure. BSCW appears to be suitable for incorporation into the PCCE. BSCW supports asynchronous and synchronous collaboration (e.g., document sharing, discussion groups, notes, instant messaging, and videoconferencing). Since the BSCW server uses its own mechanisms for document sharing, we are investigating methods of integrating it with WebDAV. We are working to determine if the Grid security model can be incorporated into the BSCW architecture and code.
We have also continued our work on the design, implementation, testing and debugging of our PCCE Secure Chat application. The chat application client runs in Java and uses an Internet Relay Chat server. Our effort on the chat this quarter has concentrated on implementing the security for the Chat application (using SSL) and improving robustness.
In our work with the Supernova Factory project, we are addressing collaboration tools that support the integration of their daily computing activities into the collaborative workspace. A grid-based persistent file transfer service has been identified as a critical component of this environment. This service will allow grid users to collaboratively manage and delegate data file transfer requests to a persistent agent that will efficiently manage these transfers. It will manage these transfers even in the presence of network outages and unavailability of required hosts, and it will notify users of successful completion. We have completed a preliminary requirements model for this service and are implementing a first prototype. The first design of this system is composed of two separable layers. The top layer is optimized to the requirements of the user domain. The underlying layer provides file transfer services via GSI FTP and will utilize existing grid security services. Both layers will have embedded database support to insure persistent state information and facilitate file transfer service restart. The supernova search images will be moved using this service from remote observatories to LBNL's NERSC facility. This task contains a good mix of data volume (approx. 30-50 Gbytes/day), numbers of files (approx. 800/day) and time constraints for a first instance demonstration.
The file transfer service will be prototyped as SOAP-based web services. SOAP is a recently formalized, XML-based remote object access protocol. By providing a SOAP- based interface to these services, we anticipate a number of benefits associated with commercially supported software technologies. These include server-side integration with servlet engines (e.g. Tomcat), simplified interactions across Internet firewalls, and early availability of quality development tools (e.g. IBM Web Services Tool Kit). Based on the success of this interface model, we anticipate using the SOAP web services model for interaction with the workflow management services we are developing.
We have also begun a test installation and deployment of the Condor system on several of our local Linux workstations. Once the Condor installation is operational, we will install Condor-G and begin the work toward making it an essential component of our workflow management system. We are working closely with University of Wisconsin personnel to complete the Condor and Condor-G installations.
In addition, we have worked this quarter to upgrade our Access Grid node to the most recent operating systems, software versions, and hardware. Our Access Grid phone booth (small 3-4 person space) is available for use by local collaborators. We have also participated in and helped to found the Grid Forum Advanced Collabortion Environments Working Group. We attended and presented at the HPDC Workshop on Advanced Collaborative Environments - August 6, 2001. In addition, we participated in the invitation only University of Michigan Workshop on the Science of Collaboratories - July 18-19, 2001.
 Lawrence Berkeley National Laboratory Home Page. |
 Distributed Systems Department Home Page. |
Collaboration Technologies Group Home Page |
You are visitor 1827 to this page since February 1999.
This document was last updated on March 3, 2004, and is located at http://www-itg.lbl.gov/Collaboratories/quarterly-reports.html.
To report Web page problems, e-mail webmaster@www-itg.lbl.gov. To request further information about Distributed System Department activities, e-mail DAAgarwal@lbl.gov.
Credits: Distributed Collaboratories research and development is funded by the U.S. Dept. of Energy, Office of Science, Office of Advanced Scientific Computing Research, Mathematical, Information, and Computational Sciences Division. Support Credits identify the funding sources and the organizational context of the work described in this document.
Privacy and site security notice to Users .
Deb Agarwal ( DAAgarwal@lbl.gov) is responsible for this WWW document.