Daniel LaLiberte National Center for Supercomputing Applications 152 Computing Applications Building 605 East Springfield Avenue Champaign, IL 61820 Email: liberte@ncsa.uiuc.edu Phone: (217) 244-0013 FAX: (217) 333-5973
Mr. J. J. Kamerer, Director, Grant & Contract Administration 109 Coble Hall MC-325 801 S. Wright Street Champaign, IL 61820 FAX: (217) 333-2186
SMALL DISADVANTAGED BUSINESS Crystaliz, Inc. 696 Virginia Road Concord, MA 01742
ARPA/ITO, BAA 96-06,
Attn: BAA 96-06, 3
701 N. Fairfax Drive,
Arlington, VA 22203-1714
Electronic Mail: baa9606@arpa.mil,
The Electronic File Retrieval address:
http://www.ito.arpa.mil/Solicitations.html
Fax: (703)-522-2668
We will design and implement this framework and demonstrate it with a "Briefing Room" application featuring capture, synopsizing, annotation, editing, and replay of collaborative sessions. Tools will allow users to condense long collaborative sessions into multimedia 'minutes' for rapid human absorption, as well as augment them with high-level semantic markup to relate with background material, all of which can then be used in any future synchronous or asynchronous forum. Collaborators may be within the same room or distributed across a network, and may work in real-time or use asynchronous workflow; either way, the environment should add significant value to collaborations.
The component that supports collaborative viewing and synopsizing of captured sessions as objects will investigate advanced techniques in speech/language and image/video analysis and understanding. These will be explored and developed to aid the automated archiving and synopsizing of multimedia collaborative sessions.
While the integrated framework is innovative, so are the components. On the client side, we will design and develop a Java-based framework, called Montage, supporting generic, web-integrated, real-time collaboration capabilities between small or very large numbers of users. Transparent interoperation with external, legacy applications will be through a Java, CORBA, or OLE API, but a "portal" or transparent frame around a portion of a screen will provide immediate integration of legacy applications for collaborative viewing and remote control.
On the server side, a Repository will provide capture and replay services for timed-event media streams, including video streams. The Repository will be an object-oriented, extensible HTTP server CORBA/ORB functionality. Support for asynchronous document deposit, annotation, registration, and notification will reside in the Repository.
We will incorporate support in Montage and the Repository for a video datagram protocol [VDP] that adapts to network and server load and client needs. We will generalize VDP to the collaborative environment and to other media types including session objects.
The framework will interoperate with Knowledge Query Manipulation Language (KQML) servers to support the higher-level semantics of wide area, scalable workflow and workspace organizations. Additionally, we will use KQML's capability of exchanging metadata and objects to support transport of objects across applications and the use of applications pre-loaded with data.
We will design and develop the Montage and Repository frameworks simultaneously, insuring that they interoperate, and we will progressively integrate them into one framework by merging the many common infrastructure elements. This single framework will expedite construction of user environments such as the "Briefing Room" that supports multiple modes of collaboration.
Crystaliz, Inc. claims intellectual property rights to the KQML server software developed previous to this project and during the course of this project.
Second, integration reuses common infrastructure elements of both. For example, synchronous events routed to collaborators constrained by a floor control policy are analogous to asynchronous transactions routed through a work flow network. Solutions for a synchronous framework can, therefore, also be solutions for an asynchronous framework.
Two important differences are barriers to integration. Timeliness and persistent storage tend to be correlated with synchronous and asynchronous modes, respectively, but each is relevant to both modes. While a modest number of collaborators who require timeliness or high bandwidth can impose significant performance, scalability, and adaptability constraints on a framework, note that sufficiently large asynchronous collaborations, which must be reasonably timely, impose the same constraints. Although persistence is clearly required for asynchrony, synchronous collaboration is greatly enhanced with support for short term persistence, e.g., when using store-and-forward event routing, and to resynchronize after failures.
Another important difference concerns the kinds of collaborative applications people prefer to use given either immediate feedback or an indefinitely long delay. But a smooth integration will allow a more graceful degredation between these extremes.
Our general solution to the integration problem is to use object-oriented and distributed object technology where possible, and use existing standard protocols and systems where appropriate. We will approach the design and development of the integrated framework from both the synchronous and asynchronous sides simultaneously and, initially, largely independently. This independence is to ensure that we make progress on the more well-known and managable tasks while we gain the experience required to generalize.
In parallel with the HTTP extensions, we will define ILU or CORBA interfaces for each of these services with the intention of migrating HTTP toward the full capabilities of distributed object technology. The HTTP-NG working group is moving in a similar direction.
To deploy these new capabilities on the WWW, existing clients and servers would need to be extended. Instead of making ad hoc changes specific to only these capabilities, we have been redesigning the client and server with greater extensibility in mind by using distributed object technology for communication between core modules and extension modules.
Downloadable, platform independent, computational objects, such as Java applets, also play an important role here by allowing servers to extend clients, or vice versa, with locally executed code. When collaborators need to work with a new type of object or wish to use a new collaboration tool, the necessary code can be quickly and automatically distributed to avoid interruption of the work in progress. Certainly another advantage is the local, instead of remote execution of code, and the ability to more efficiently transfer compact high-level representations of objects that are rendered locally rather than transferring larger, low-level presentation data computed remotely.
The framework will be platform independent by building on open infrastructure technologies such as HTTP, Java, CORBA, and KQML. A major challenge will be navigating the interactions between these and other competing technologies to position our framework for rapid adoption into the national information infrastructure.
We will also investigate support for compound documents (e.g., SGML, MIME, OpenDoc, OLE) to allow, for example, an embedded applet within an annotation of a part of a document.
For the forseeable future, Montage-specific router architecture will be used to facilitate experimentation with scalable, dynamically morphable virtual networks for interconnection of collaborators. A wide variety of routing topologies remain to be examined, with the examination of the associated complexities of automatic initiation of routing topology and/or mode change. This dynamic rerouting capability is essential to session survivability and usage-optimization of potentially scarce network bandwidth.
Transmission of CM requires efficient, adaptive protocols to sustain near real-time throughput. The Video Datagram Protocol (VDP) (developed at UIUC) delivers video over heterogeneous networks, including the Internet, by adapting to network conditions and the capabilities and requirements of the client side. We will generalize this protocol for other timed-event media, and support multiple input and output streams which need to be synchronized.
In the "Briefing Room" application, session streams will be captured as short segments, each of which will be an object stored along with appropriate metadata and relationship information. Replay will require very fast location of the segments required, efficient caching, and anticipatory scheduling of retrievals from storage.
For performance, the timed-event streams will typically bypass much of the infrastructure of the Repository and Montage frameworks, although the establishment and control of the streams may remain in the framework.
Cross organizational workflow has to deal with the fact that organizations would like to keep their autonomy. A requirement of administrative control within the organization is addressed by using KQML's meta level query protocol elements to implement a context exchange mechanism between collaborating organizations. The KQML server will respond to queries from outside organizations by providing the context information. Outside organizations can then use this information to properly translate the information into the semantic context that is appropriate for interpreting the messages.
A requirement of flexibility to change the way work flows within the organization is met by using KQML's networking protocols to establish an intelligent directory that is a logical information space on top of physical directories used in the organization. This supports separation between "roles", "routes", and "workflow rules" and the actual persons, places, and policies that implement the workflow. Since this higher level knowledge base can respond to deductive queries (using KQML's query protocol elements), intelligent agents can be deployed on top of this directory. These intelligent agents can dynamically alter the work flow, by extracting different actual physical addresses in response to the context of the KQML queries.
An important issue to be addressed in workspace organization is the support for cooperative transactions. Cooperative transactions in this context are defined to be transactions originating from members of different groups modifying the same data item. At the core, this problem is addressed through notifications. Using KQML's publish/subscribe protocols, members of different groups can express interest in data items. When these data items are changed (say by an update), then the interested parties are notified. The receiving parties can implement several different policies to deal with these notifications. A policy used often is to merge changes into the receivers current working copy. Additionally, we will also explore implementing different lock modes on data items.
Additionally, the same content language can also be used to send mobile code with state using higher level content models, further reducing event and data traffic across the network. For example, the mobile code could be used to represent an annotation that consists of several user performed operations on the higher level objects. This annotation packet bundles a set of user interactions, information about the tool used to create the user interactions, and the modified state of the objects. The annotation packet is interpreted by the KQML server and actions described in the annotation packet are performed using the appropriate tool.
In view of all these, NCSA can commit resources from outside the scope of this BAA toward the implementation of the basic Montage transport mechanisms and tools. Although essential for the construction of the testbeds and experiments addressed herein, a need for funding of this portion of the Montage work within this BAA is not foreseen. This BAA focuses on the outlying generations of collaboration infrastructure that can be built with JAVA, CORBA, and Montage-based technologies.
A large variety of software tools available from NCSA, other activities at the University of Illinois, and collaborating Colleges, Universities, and Agencies/Organizations will be examined for potential usefulness as distributed collaborative tools. Additions of the Montage/Java wrappers to legacy tools will be done to create distributed user communities as testbeds for the Montage/Java/CORBA distributed object technologies. A series of iterative improvements and subsequent experimentation is planned.
KQML has been used extensively in engineering cross functional, multi-site, collaborative projects ["Agents, Agencies, Artifacts, and Appliances", Sankar Virdhagriswaran, et. al., World Wide Web Conference, Nov. 1995, Poster Session]. In this BAA, we address the limited context of engineering cooperative problem solving ["Collaborative Engineering Based on Knowledge Sharing Agreements", G. Olsen, et. al., http://www-ksl-stanford.edu/knowledge-sharing/papers/index.html]. We plan to extend this work to supporting collaboration in a general, platform independent way. Related work in this area include the CARNOT project performed at MCC [D. Woelk, W. M. Shen, M. Huhns, and P. Cannata, "Model Driven Enterprise Information Management in Carnot," Enterprise Integration Modeling: Proceedings of the First International Conference, The MIT Press.].
The KQML server used in this project will re-use technologies developed as part of ARPA SBIR projects and will also attempt to use technologies to be developed as part of DLA BAA (contract to be negotiated).
Related proposals for this BAA is "(A Testbed for Continuous Media)", and one submitted by NIIIP.
Other collaborations: IETF, W3C, EIT, Microsoft, DLI (UIUC, CMU), SUN, SGI, and Doug Englebart at Bootstrap Institute.