BAA number: 96-06

Electronic Submission ID: AB9606UnIL02

Technical topic area: Collaboration Infrastructure

Framework for Integrated Synchronous and Asynchronous Collaboration

Last modified: Wed May 14 22:04:25 CDT 1997

Technical Point of Contact:

Daniel LaLiberte
National Center for Supercomputing Applications
152 Computing Applications Building
605 East Springfield Avenue
Champaign, IL 61820

Email: liberte@ncsa.uiuc.edu
Phone: (217) 244-0013
FAX: (217) 333-5973

Administrative Point of Contact:

Mr. J. J. Kamerer,
Director, Grant & Contract Administration
109 Coble Hall   MC-325
801 S. Wright Street
Champaign, IL   61820
FAX:    (217) 333-2186

Contractor's Type of Business: OTHER EDUCATIONAL

Subcontractors:

SMALL DISADVANTAGED BUSINESS
Crystaliz, Inc.
696 Virginia Road
Concord, MA 01742

Agency Address:

ARPA/ITO, BAA 96-06,
Attn: BAA 96-06, 3
701 N. Fairfax Drive,
Arlington, VA 22203-1714

Electronic Mail: baa9606@arpa.mil,
The Electronic File Retrieval address:
     http://www.ito.arpa.mil/Solicitations.html
Fax: (703)-522-2668

Section II. Summary of Proposal

A. Key Innovations

The primary innovation of this project is the integration of synchronous and asynchronous collaboration technologies into one framework. The generality of the framework will allow applications to be deployed for various military related areas, e.g., group meetings, command and control, crisis management, and training.

We will design and implement this framework and demonstrate it with a "Briefing Room" application featuring capture, synopsizing, annotation, editing, and replay of collaborative sessions. Tools will allow users to condense long collaborative sessions into multimedia 'minutes' for rapid human absorption, as well as augment them with high-level semantic markup to enrich them with background material, all of which can then be used in any future synchronous or asynchronous. Collaborators may be within the same room or distributed across a network, and may work in real-time or use asynchronous workflow; either way, the environment will significantly enhance collaborations.

While the integrated framework is innovative, so are the components. On the client side, we will design and develop a Java-based framework, called Habanero, supporting generic, web-integrated, real-time collaboration capabilities between small or very large numbers of users. Transparent interoperation with external, legacy applications will be through a Java, CORBA, or OLE API, tightly integrating fine-grained functionality of third-party applications. In addition, a "portal" or transparent frame around a portion of a screen will provide immediate integration of legacy applications for collaborative viewing and remote control.

On the server side, a Repository will provide capture and replay services for timed-event media streams, including video streams. The Repository will be an object-oriented, extensible HTTP server with CORBA/ORB functionality. Support for asynchronous document deposit, annotation, registration, and notification will reside in the Repository, as well as support for capture, annotation, editing and replay of any session activities, including real-time video and the the full range of multimedia session interactions.

We will incorporate support in Habanero and the Repository for a video datagram protocol [VDP] that adapts to network and server load and client needs. We will generalize VDP to the collaborative environment and to other media types including session objects.

The component that supports collaborative searching, viewing and synopsizing of captured sessions as objects, and sequences of objects, will investigate advanced techniques in speech/language and image/video analysis and understanding. These will be explored and developed to aid the automated archiving and synopsizing of multimedia collaborative sessions.

We will investigate the security implications of a collaborative environment and implement solutions using mostly extant protocols.

The framework will also interoperate with Knowledge Query Manipulation Language (KQML) servers to support the higher-level semantics of wide area, scalable workflow and workspace organizations. Additionally, we will use KQML's capability of exchanging metadata and objects to support transport of objects across applications and the use of applications pre-loaded with data.

We will design and develop the Habanero and Repository frameworks simultaneously, insuring that they interoperate, and we will progressively integrate them into one framework by merging the many common infrastructure elements. This single framework will expedite construction of user environments such as the "Briefing Room" that supports multiple modes of collaboration.

B. Deliverables

The proposed project will use the application deliverables as the driving force for developing infrastructure deliverables. Following a cycle of design, prototyping, and development we construct the infrastructure and demonstrate its utility in the application. The project will deliver the following in approximately this order.

Habanero Framework

An extensible framework for sharing distributed Java objects. This framework will be demonstrated by constructing a representative set of basic synchronous-collaboration tools (e.g., 2D/3D whiteboards, shared text editor, and remote controllable Web browser).
As Java bindings for CORBA-compliant ORBs become available, many of the underlying services of the Habanero framework will be replaced with CORBA-based equivalents. As gateways for OLE/COM and OpenDoc become freely available, these technologies too will be incorporated into demonstration software.
Modified Web client, using HotJava as a base, to additionally support: Server Push, VDP, PEP, generalized proxy extensions (for URNs, at a minimum), annotations, neighborhoods.
Rehostable-code integration into Habanero components.
Dynamic & Adaptive Router providing a basic routing capability, scalabile to Multicast-based routing, virtual (superimposed) LAN topology experimentation, demo of a "very large" collaborative session.
Floor control in the collaborative session, and adaptive control in a session that grows in size far beyond the initial size.
Easily encapsulate the rapidly increasing number of web-based Java applets and Java applications that support domain-specific data, analysis, visualization, and interactions; e.g., Visible Human, Weather Applet from Horizon Project.
VRML editor and 2D graphics editor integrated with Habanero framework.

Repository Framework

New session object and session stream as a continuous media. Develop conceptual framework for building, adapting, subclassing, session templates and constraints on sessions.
Session replay, including optional annotations, from the object store.
Develop a timed-event protocol using HTTP for establishment and control (using keepalive) and a separate typed channel for the real-time stream and for synchronization. Support collaborative, synchronized multi-output and multi-input channels and random access to stream content.
Specializations of this protocol to support several standard (or extended) media types: video, audio, VRML, animations, simulation output, control streams. Interfaces and tools for markup, editing, and annotating of continuous media objects.
Algorithms, software, and demonstrations of speech and video analysis techniques for automating the process of archiving, and synopsizing of a multimedia collaborative working session. The techniques include topic-spotting using speech and video, video segmentation, key-frame detection, and video compression for different types of indexing/retrieval.
Annotation and Notification protocol specifications, demonstration server

KQML and Workflow

KQML server integrated with the repository and security service of the repository.
KQML server augmented with a directory service to support definition of cross organizational workflow.
KQML server integrated with Habanero applications.
KQML Query protocol elements to support meta information exchange between various collaborating work groups.
KQML Networking protocol elements to support formulation of a hierarchy of shared workspaces for interaction between large groups of collaborators whose work can be divided into a project/sub-project hierarchy, and to support modeling of roles and routes that are inter-organizational.
KQML Notification protocol elements to manage interaction of adhocracies (on the fly association) and to control and execute workflow schedules.
KQML Facilitation protocol elements to support brokering of interactions between autonomous workspaces.
KQML algorithms to avoid manual optimization of routing networks by automatically and dynamically reconfiguring the network based on factors such as server availability and end-to-end latency.

Briefing Room Demo

Demonstration of prototype "Briefing Room".
Final demo.

Integrated Framework

Design of an integrated security framework addressing the sophisticated authorization, authentication and privacy needs of collaborative environments.

Support for common security technologies, such as Kerberos authentication capabilities.

Autonomous agents as collaboration participants.

Distributed timed-event adaptive multi-client - multi-server infrastructure. Support for caches and recovery of missed info.

Intellectual Property

It is intended that all software will be made available on the Internet, under licensing terms similar to those used with NCSA Mosaic (i.e., free of charge for individual, government, or educational use, with a license required for commercial use). Associated with this release will be documentation and reference code to facilitate the construction of Java/Habanero. Negotiations are in progress with the applicable commercial vendors in the interest of also making partial-capability versions of their works freely available.

Crystaliz, Inc. claims intellectual property rights to the KQML server software developed previous to this project and during the course of this project.

C. Cost, Schedule, and Milestones

Distribution of costs throughout the life of this project is shown in the table entitled "Proposed Budget". Research, development, and demonstration schedules are shown in the table below. Note that the dates shown are generally the estimated dates for first working demonstrations of the listed technologies. Refinement into publically releasable versions will follow, as will integration and improvement of the continuously expanding feature sets, and as will the exploratory research into balancing network tradeoffs in these new environments. The Funding Years are assumed to begin with the awarding of the grant.

The tasks are coded for reference by the detailed cost section into the following categories:

Habanero - Synchronous Collaboration, 2 FTEs
Repository - Asynchronous Collaboration, 2 FTEs
KQML - Workflow, Integration, 2 FTEs
VDP - Adaptive Timed-Event Protocol, 4 students
Analyze - Video/Audio Analysis, Compression, 1 student

Funding Year 1:

Public (Beta) Habanero release with minimalist toolset, Collaborative Web browser and CORBA for Habanero transport. (Habanero)
Timed-Event Protocol specialized to sessions, audio, simulation and Session object and session stream as a continuous media. (VDP)
Repository as Object Store, CORBA in Repository, Bulk encryption facility and Annotations server. (Repository)
Image and video representation and segmentation and Begin wavelet/morphological algorithms for still images. (Analyze)
KQML directory services and KQML in Repository. (KQML)

Funding Year 2:

Habanero floor control, Authentication of Habanero session participants, Reconfigurable virtual-LAN topology and Multicast-capable router. (Habanero)
Timed-Event Protocol specialized to sessions, audio, simulation and Editing of recorded sessions. (VDP)
Habanero-to-Repository proxy and Repository-to-Habanero proxy (replay). (Habanero, Repository)
Begin motion estimation for video coding, Complete wavelet/morphological algorithms for still images, Develop neuronet architectures for fusing auio and video cues, Develop algorithms for synopsizing speech and Automatic indexing of Text and Speech-to-Text materials. (Analyze)
KQML workflow and Hierarchical workgroups. (KQML)

Funding Year 3:

Agents participate in sessions as proxies, Migration of agent programs between hosts and Integrated Habanero session control capabilities. (Habanero)
Distributed timed-event adaptive multi-client/multi-server infrastructure and Digitally signing session histories. (VDP)
Interchangeable authentication and bulk encryption. (Repository)
Complete motion estimation for video coding and Automatic indexing of video materials (Analyze)
VRML editor and 2D graphics editor integrated with Habanero framework. (Crystaliz)

D. Technical Rationale

Integration of Synchronous and Asynchronous Collaboration

We desire an integrated synchronous and asynchronous collaboration framework for two reasons. First, integration facilitates interoperation. Real work environments typically cycle between many synchronous and asynchronous collaborations, frequently making use of the same media for both. E.g. a live, synchronous session regarding a document may be captured to allow asynchronous annotation, synopsizing of the session, revision of the document, and subsequent replay of the session within another synchronous session. Our goal is that all collaboration tools, whether synchronous or asynchronous, should support manipulation and annotation of the same set of media types. Furthermore, we should be able to capture the synchronous sessions and asynchronous transactions themselves as objects to be manipulated collaboratively.

Second, integration reuses common infrastructure elements of both. For example, synchronous events routed to collaborators constrained by a floor control policy are analogous to asynchronous transactions routed through a workflow network. Solutions for a synchronous framework can, therefore, also be solutions for an asynchronous framework.

Two important differences between the synchronous and asynchronous collaboration modes are potential barriers to integration. Timeliness and persistent storage tend to be correlated with synchronous and asynchronous modes, respectively, but each is relevant to both modes. While a modest number of collaborators who require timeliness or high bandwidth can impose significant performance, scalability, and adaptability constraints on a framework, note that sufficiently large asynchronous collaborations, which must be reasonably timely, impose the same constraints. Although persistence is clearly required for asynchrony, synchronous collaboration is greatly enhanced with support for short term persistence, e.g., when using store-and-forward event routing, and to resynchronize after interruptions.

Another important difference concerns the kinds of collaborative applications people prefer to use given either immediate feedback or an indefinitely long delay. We don't deny this issue, but an integrated system will at least allow a more graceful degredation between these extremes, and at best allow us to support the best of both within one environment.

The Briefing Room

An experimental demonstration of many aspects of the entire framework will be a "Briefing Room" consisting of a set of tools and modules in Habanero and the Repository to support a common collaborative activity, the production of a brief. The brief synopsizes and annotates relevant material from a number of sources, some from static documents, such as a map, and some from previously captured synchronous collaboration sessions, such as a video conferenced planning meeting. The production of the brief itself may involve a cycle of sychronous and asynchronous collaborations. The brief becomes a stored document that may be referenced, retrieved, and annotated like any other object.

Integrating WWW and Distributed Objects

The World Wide Web was designed to support collaboration, but it currently provides modest support only for transmission of documents from servers to clients via the HyperText Transfer Protocol (HTTP). We propose to extend HTTP, using the Protocol Extension Protocol (PEP) under development by W3C, to add new protocols for the collaborative capabilities described in this proposal. Specific protocols we will create or adopt from others include: document deposit (e.g. PUT), annotations, multi-user sessions, real-time input and output streaming, registration and notification.

In parallel with the HTTP extensions, we will define ILU or CORBA interfaces for each of these services with the intention of migrating HTTP toward the full capabilities of distributed object technology.

To deploy these new capabilities on the WWW, existing clients and servers would need to be extended. Instead of making ad hoc changes specific to only these capabilities, we have been redesigning the client and server with greater extensibility in mind by using distributed object technology for communication between core modules and extension modules.

Downloadable, platform independent, computational objects, such as Java applets, also play an important role here by allowing servers to extend clients, or vice versa, with locally executed code. When collaborators need to work with a new type of object or wish to use a new collaboration tool, the necessary code can be quickly and automatically distributed to avoid interruption of the work in progress. Certainly another advantage is the local, instead of remote execution of code, and the ability to more efficiently transfer compact high-level representations of objects that are rendered locally rather than transferring larger, low-level presentation data computed remotely.

The framework will be platform independent by building on open infrastructure technologies such as HTTP, Java, CORBA, and KQML. A major challenge will be navigating the interactions between these and other competing technologies to position our framework for rapid adoption into the national information infrastructure.

We will also investigate support for compound documents (e.g., SGML, MIME, OpenDoc, OLE) to allow, for example, an embedded applet within an annotation of a part of a document.

Video and Timed-Event Streams

Many applications, the "Briefing Room" in particular, require the capture, storage, manipulation, and delivery of continuous media (CM) or timed-event streams, which are data streams that have temporal constraints. These may include video, 3-D visualizations, instrumentation traces, and the output of simulations.

Transmission of CM requires efficient, adaptive protocols to sustain near real-time throughput. The Video Datagram Protocol [VDP] (developed at UIUC) delivers video over heterogeneous networks, including the Internet, by adapting to network conditions and the capabilities and requirements of the client side. We will generalize this protocol for other timed-event media, and support multiple input and output streams which need to be synchronized.

In the "Briefing Room" application, session streams will be captured as short segments, each of which will be an object stored along with appropriate metadata and relationship information. Replay will require very fast location of the segments required, efficient caching, and anticipatory scheduling of retrievals from storage.

For performance, the timed-event streams will typically bypass much of the infrastructure of the Repository and Habanero frameworks, although the establishment and control of the streams may remain in the framework.

Security Issues

For a viable, effective collaborative environment, there clearly must be strong and flexible capabilities for authentication, access control and data confidentiality. However, security models applicable to such dynamic, large-scale, collaborative contexts are relatively undeveloped. How is mutual trust established between members of a decentralized, spontaneously-created work group? What are efficient and sufficiently flexible methods of managing group session keys for handling encrypted data, and role-based credentials for performing authorization? This situation is complicated by the fact that modern communications security technologies are typically applicable to either synchronous or asynchronous exchanges but rarely to both.

We intend to design an integrated security framework suitable for this demanding environment. This framework will augment the authentication and access control systems employed by the repository server as well as the security services built into the Habanero and Web clients. The system architecture must have security functions and interfaces specified in its earliest stages of development, as adding them later in an ad-hoc fashion has been repeatedly shown to be unworkable.

Concretizing and implementing this security framework will involve making use of many recent security-related standards designed for the Web and distributed systems in general. Combining the security services provided by CORBA and Java will enable safe, reliable handling of distributed objects in the collaborative application. Additional schemes for group credentials and session data encryption will be designed using standards for security-enhancement of connections, messages and documents.

Workflow

A collaborating work group often consists of individuals who need to solve problems synchronously (e.g., field level resource planning) and individuals who need to review and/or approve (e.g., battalion commander at base) the results of the synchronous problem solving session. The review/approval process can be integrated into the synchronous conferencing using workflow. Support for workflow allows crossing of organizational boundaries and organization of the workspaces into a set that makes sense for large scale collaboration. In the proposed project we address these problems using KQML.

Cross organizational workflow has to deal with the fact that organizations would like to keep their autonomy. A requirement of administrative control within the organization is addressed by using KQML's meta level query protocol elements to implement a context exchange mechanism between collaborating organizations. The KQML server will respond to queries from outside organizations by providing the context information. Outside organizations can then use this information to properly translate the information into the semantic context that is appropriate for interpreting the messages.

A requirement of flexibility to change the way work flows within the organization is met by using KQML's networking protocols to establish an intelligent directory that is a logical information space on top of physical directories used in the organization. This supports separation between "roles", "routes", and "workflow rules" and the actual persons, places, and policies that implement the workflow. Since this higher level knowledge base can respond to deductive queries (using KQML's query protocol elements), intelligent agents can be deployed on top of this directory. These intelligent agents can dynamically alter the work flow, by extracting different actual physical addresses in response to the context of the KQML queries.

An important issue to be addressed in workspace organization is the support for cooperative transactions. Cooperative transactions in this context are defined to be transactions originating from members of different groups modifying the same data item. At the core, this problem is addressed through notifications. Using KQML's publish/subscribe protocols, members of different groups can express interest in data items. When these data items are changed (say by an update), then the interested parties are notified. The receiving parties can implement several different policies to deal with these notifications. A policy used often is to merge changes into the receivers current working copy. Additionally, we will also explore implementing different lock modes on data items.

Asynchronous Workspace Information Exchange

A problem that tends to be exacerbated in asynchronous workspaces is the expected wide scale distribution of participating sites and the variety of applications and operating system platforms used in those sites. In order to support import and export of objects and interoperability of applications used in asynchronous workspaces, the KQML server will also use a new platform independent content language at the application level. Modeled after the AppleEvent suite, but modified for inter-operability across WANs, this high-level content language will support object-oriented content models in the virtual collaboration environment for a wide range of applications including word processing, spread sheet, 2D and 3D graphics, project planning, and relational databases.

Additionally, the same content language can also be used to send mobile code with state using higher level content models, further reducing event and data traffic across the network. For example, the mobile code could be used to represent an annotation that consists of several user performed operations on the higher level objects. This annotation packet bundles a set of user interactions, information about the tool used to create the user interactions, and the modified state of the objects. The annotation packet is interpreted by the KQML server and actions described in the annotation packet are performed using the appropriate tool.

E. Related Research

NCSA has already started its Habanero project, building on past experience with Collage, to investigate and capitalize on potential Java-enabled functionality extensions for Internet collaborative technologies. Commonly used collaborative tools which are candidates to be built on top of the Habanero object sharing and dissemination system (e.g., shared Whiteboard capabilities, shared Chat sessions, shared text editing, audio and video conferencing) are of interest to a number of NCSA-collaborating groups, and are potentially applicable in a number of projects involving NCSA (e.g., distance learning systems, digital libraries, the National Performance Review, the Virtual Agency project). Further, a number of commercial firms are contributing-in-kind as part of their concern for developments in this area.

In view of all these, NCSA can commit resources from outside the scope of this proposal toward the implementation of the basic Habanero transport mechanisms and tools. Although essential for the construction of the testbeds and experiments addressed herein, a need for funding of this portion of the Habanero work within this proposal is not foreseen. This proposal focuses on the outlying generations of collaboration infrastructure that can be built with Java, CORBA, and Habanero-based technologies.

A large variety of software tools available from NCSA, other activities at the University of Illinois, and collaborating Colleges, Universities, and Agencies/Organizations will be examined for potential usefulness as distributed collaborative tools. Additions of the Habanero/Java wrappers to legacy tools will be done to create distributed user communities as testbeds for the Habanero/Java/CORBA distributed object technologies. A series of iterative improvements and subsequent experimentation is planned.

KQML has been used extensively in engineering cross functional, multi-site, collaborative projects [AAA]. In this proposal, we address the limited context of engineering cooperative problem solving [Collab-Eng]. We plan to extend this work to supporting collaboration in a general, platform independent way. Related work in this area include the CARNOT project at MCC [Carnot].

The KQML server used in this project will re-use technologies developed as part of ARPA SBIR projects and will also attempt to use technologies to be developed as part of DLA proposal (contract to be negotiated).

Other collaborations: IETF, W3C, EIT, Microsoft, DLI (UIUC, CMU), SUN, SGI, OSF, and Doug Englebart at Bootstrap Institute. See references in the bibliography.

F. Key Personnel

The following people will be responsible for the administration of this proposal and ultimately responsible for the completion of deliverables. Full time staff and research assistants responsible for completion of the technical details of this proposal will include both current staff and new personnel.

Joseph Hardin: Associate Director for the Software Development Group at NCSA. Joseph will serve as a PI and have high level responsibility for personnel allocations and administration of this proposal.
Daniel LaLiberte: Research Programmer, NCSA. Dan will serve as a PI for this proposal and be responsible for the integrated architecture.
Elizabeth Frank: Technical Manager of the SDG Server Development Team. She will manage the repository development for this project.
Larry Jackson: Senior Research Programmer. Larry will serve as a PI supervise technical staff responsible for the Java-related collaboration efforts of this proposal.
Thomas S. Huang: Currently a professor of Electrical and Computer Engineering at the University of Illinois in Urbana/Champaign. He will supervise one RA and be responsible for the Bell Lab consultant. (1 month per year)
W. H. (Fred) Juang: Is Group Leader for Speech Understanding and Senior Technical Member, AT&T Bell Laboratories and Adjunct Professor of Electrical and Computer Engineering, UIUC. (50 days per year)
Sankar Virdhagriswaran: Engineering manager and product manager at Crystaliz, Inc. Sankar will be project manager for Crystaliz, Inc. at 25% for 2 years.
Mike Webb: Lead Engineer, Crystalize. 75% for 2 years.
Dr. Gordon Dakin: Support Engineer, Crystalize. 100% for 2 years.
Roy H. Campbell: Professor in the Department of Computer Science at the University of Illinois. He will supervise 4 graduate students working on the adaptive timed-event protocol.

Section III. Detailed Proposal Information

A. Statement of Work

The project work can be roughly divided into three categories: tasks that are most closely related to synchronous collaboration, asynchronous collaboration, or the integration of both.

Synchronous Collaboration Tasks

Sessions: Define and implement support for multi-participant sessions, session IDs, session security, session streams, and persistent storage of session streams as objects.
Timed-Event Streams: Define and implement more general timed-event stream objects, including real-time audio and video streams, capture and replay. Session streams become a specialization of timed-event streams.
Screen Capture Provisions: Platform-dependent routines will be created to allow the periodic capture of screen images from unmodified legacy software. This will make the results from a variety of tools and viewers available to all participants in the session, even if the specific tool is not installed on all machines.
Extensible Framework for Shared Java Objects: NCSA has designed and is developing the Habanero sharing framework to capitalize on the apparent single-platform environment made possible by the Java programming language. Under the Habanero framework, Java objects are shared between platforms, and transformed back to compliance with legacy programs where necessary.
Dynamic & Adaptive Routing: Presently unicast, the Habanero router will be expanded to include multicasting and variable routing topologies. Automated transitioning between routing topologies will generally be possible during a session.
2D Graphics Editing and Annotation: Functionality will be built into the base classes of the Habanero framework which allows annotation of shared on-screen objects. As such, these mechanisms will be available, if applicable, to all tools that are built under Habanero.
3D Graphics Editing: 3D VRML-based graphics display and editing tools will be constructed which conform to the Habanero sharing framework.
Rehostable-code Integration: To the extent practicable, and meaningful, the Habanero object suite will be outfitted with mechanisms to incorporate dynamically installed functionality additions, comparable in spirit to the Applet extensions of the HotJava browser.
CORBA Interface: Session control and routing mechanisms will be retrofitted with CORBA shared objects, pending arrival of the necessary Java bindings.
OLE Interface: Pending arrival of freely available Java/OLE/COM bindings, the all-Java browser will be modified to support OLE controls. This will enable communication between this browser and the suite of OLE plug-ins under development by NCSA's Windows Mosaic development team, and by Microsoft and other commercial firms.

Asynchronous Collaboration Tasks

Adaptive VDP

The all-Java browser will be modified to support the Video Datagram Protocol exemplified in Vosaic, but generalized to support dynamic control via HTTP of timed event replay at desired rate and quality.

Novel image/video compression schemes

Need to meet requirements of high compression ratio and suitability for content-based retrieval.

Algorithms for indexing and synopsizing video/audio

The implementation may take advantage of the correlation between the two.

KQML Server to support Workflow

The following KQML protocol elements will be developed for use in supporting various asynchronous workspace functionality.

Network services (register, unregister, forward, transport-address, broadcast, pipe, break, fork, kill) for setting up a virtual network between workspaces.
Information and query services for meta information exchange (evaluate, reply, tell, error, sorry, ask).
Capability definition services for dynamic modification of the workspace network (advertise, brag).
Notification services for reacting to changes in workspace configuration (subscribe - monitor is useless).
Facilitation services for knowledge based routing of workflow (broker, recommend, recruit).

Integration Tasks

Session Capture Proxy: Derived classes will be developed from all Habanero sharing components, for the purpose of adding communications objects to the Repository generalized object store facility. These new classes will constitute a logical gateway from Habanero transmission protocols (sockets, CORBA, and Multicasting) to the HTTP of the Repository software.
Session Replay Proxy: Derived classes will be developed from all Habanero sharing components, for the purpose of retrieving communications objects from the Repository generalized object store facility. These new classes will constitute the inverse logical gateway, for session retrieval and replay purposes.
Annotations and Neighborhoods: The all-Java browser will be modified to support document annotations and information discovery via neighborhoods. We will define annotation and neighborhood protocols and generic markup mechanisms for Web objects and specialize these for timed-event streams. A reference implementation of annotation and neighborhood servers will be built.
Notifications: Define a notification protocol for Web events and specialize the protocol for different routing schemes.
Habanero Launch from Neighborhoods: Habanero session control and initialization will be made automatically launchable upon discovery of a group of colleagues via the Neighborhoods mechanism.
Remotely-Controllable Browser: The all-Java browser will be modified to support operation under a colleague's control. This will allow sessions to traverse Web-legacy materials as a group, whenever such a mode of viewing would facilitate the communications process.
Server Push: The all-Java browser will be modified to support Server-Push extensions to HTTP.
Protocol-Extensions Protocol: The all-Java browser will be modified to support the Protocol-Extensions Protocol of HTTP.
Generalized Proxy Extensions: The all-Java browser will be modified to support generalized proxy extensions comparable to those currently deployed in the X-Window version of Mosaic.

B. Results

Expected Project Results

The Habanero project will provide significant advances in the understanding of shifting the nature of information work from islands of individual workers toward simultaneous team sharing of designs and ideas under formulation. By facilitating collaboration-enabling retrofits to legacy materials, as well as newly developed Java materials, a very large number of professional working groups and distributed work teams are expected to benefit. As differing communities will have their own specialized tools, part of this project is providing the necessary documentation and support to advise in the tool conversion process. With the limited trials done to date, significant issues are being uncovered, such as identifying optimal single-user design guidelines to best position a software product for the eventual incorporation of multiple collaborating users.

Information discovery mechanisms, leading to the launch of synchronous collaborations with colleagues, forms one Habanero startup scenario. The Habanero software is also launchable from a variety of toolbox-like windows, and the like. Due to this reconfigurability, the Habanero environment is expected to be able to provide synchronous, cross-platform collaboration functionality in a variety of application domains.

Improvements and adaptations of router technology and session management will provide user-configurable virtual-network topology, critical in locations or times of limited bandwidth. Effective transitioning between unicast message delivery, multicasting, and reliable multicasting without placing the burden on the human operator is essential to establish the viable use of this adaptive routing technology.

The notion of capture, edit, annotation, and replay of a recorded multi-media collaborative session is a powerful one, opening possibilities in the production of derivative works specific to the information needs of sub-communities, and in the ability to replay historically critical decision making processes largely as they occurred. Further, additional software can monitor the captured multi-media fragments as they occur, and can be utilized in a variety of supporting roles such as automated indexing and the retrieval of related references. Trials begun utilizing the Habanero environment will address not only the beginnings of session retracability/reusability, but will also explore the problems specific to capture and replay of high-bandwidth media types.

The proposed project will advance the state of the art in wide area workflow dramatically. It will support definition of intersecting workspaces and managing the consistancy of transactions in those workspaces. Furthermore, it will support flow of work within and between workspaces across the Internet.

Project Products

Collaboration-enabling Habanero framework

This framework would be usable by:

all-Java programs, exploiting object inheritance from Habanero base classes to acquire functionality without redundant coding,
legacy programs with Habanero/Java interfaces, reusing Habanero routing and control objects, and
unmodified legacy software, via screen capture, paste buffer, drag-and-drop provisions, and the like.

A representative suite of applications will accompany this framework, forming both a very usable toolset and an extensive body of reference code to guide users from other communities.

Collaboration-enabled Web browser

This browser will be consistent across multiple platforms, to provide access in collaborative work sessions to all information reachable by the Web, including server gateways to legacy databases. This browser will additionally complement other active research areas and education applications, embracing new HTTP extensions, including Server Push, Keep Alive, the Protocol Extensions Protocol, and Generalized Proxy Extensions. This browser will not be developed from scratch, but will (pending finalizing a suitable software license) utilize the HotJava browser of Sun Microsystems as a base.

Session capture, edit, annotation, and replay software

Software gateways between the synchronous Habanero session and object store facilities will be constructed utilizing subclasses of the base Habanero classes. These subclasses will function as gateways, communicating with the object store facility utilizing HTTP (as in the case of the Repository software), or other mechanisms in connection with commercial databases.

Screen capture mechanism for multiple platforms

This mechanism will feed the Habanero/Java object-sharing system, providing sharing features to legacy software, without modification to that software.

Video Datagram Protocol-enabled Web browser on multiple platforms

This browser will support research into adaptive protocols for streamed data types. Graceful degradation of the data fidelity in a session is an essential technology for applications where high-quality communications bandwidth is overtaxed, as may be anticipated in forward deployment areas. Optimal exploitation of streamed data is of great interest in applications including video, audio, and visualization imagery. As such, experiences gained in this browser setting are directly applicable to the real-time multi-media information sharing in synchronous collaboration sessions such as Habanero.

Image/video compression

Algorithms will be developed which achieve lower bit rates than existing methods at the same image quality. The image representations will be hierarchical so that browsing of the image data is made easy. Furthermore, the image/video representations will be in forms suitable for image content-based retrieval based on color, texture, shape, layout, and even high level concepts such as "human face" and "moving automobiles". These algorithms will be useful for all multimedia database applications.

Video/audio indexing and synopsizing

Various software tools will be developed to aid the automatic or interactive indexing and synopsizing of video/audio sessions. These algorithms will include: Video segmentation and keyframing, automatic indexing and synopsizing of speech using a detection-based approach, correlating audio with video. These basic tools will have a wide range of applications including topic spotting and tracking of audio/video, and gisting of audio/video.

KQML Server

A key deliverable of the project is the KQML server that supports asynchronous workspace organization and workflow management between them.

Transferable Technologies

The following is a partial list of transferable technologies that will result from this project.

Habanero/Java object sharing: As described above, the ability to bring collaboration to tools that were not originally designed for collaboration is very broadly applicable, and holds considerable promise in the rethinking of the whole problem of distributed work teams.
Java/CORBA/OLE interoperability: Pending the arrival of suitable free versions of the necessary bindings, the Habanero software, including the Web browser, will incorporate CORBA and OLE technologies in the interest of providing gateways to the broadest possible range of legacy software. This allows leveraging off past investments wherever possible, instead of redeveloping.
Collaborative Session manipulation: The ability to capture, annotate, edit, and replay synchronous sessions forms a bridge between the real-time collaboration tools and a wide variety of asynchronous tools (e.g., e-mail, Web CGI/Forms gateways, database management systems). The derivative materials fill communications voids in training, briefing absent colleagues, custom briefings divulging only that information required by distinct groups.
Neighborhoods information discovery system: A community-server-based system, originally focused on identifying collections of Web documents of interest, is extensible as a means of identifying groups of colleagues, and as a mechanism for launching synchronous collaboration sessions with those colleagues. [Neighborhoods]
Annotation Server: An extended HTTP server, using CGI or other extension mechanisms, will support annotations of any web documents. This service is an asynchronous partner to the neighborhood server for synchronous collaborations.
Adaptive Timed-Event Protocol: A generic protocol for dynamically adaptive transmission of all manner of timed-events. HTTP will be used for initialization and feedback control of the media stream.

Expected Technology Transfer Path

A representative suite of applications will accompany the Habenaro framework, forming both a very usable toolset and an extensive body of reference code to guide users from other communities. Further, documentation and source code examples of how to adapt single-user software to support distributed synchronous use within work teams will be provided. This suite of technology components will suffice to enable collaboration-enabling experimentation in a very large number of communities.

Sample modules for the Repository will be delivered along with the NCSA http server redesigned to support this architecture. In particular, modules that support the collaboration framework of this proposal will be included.

Government and education user communities will be able to utilize Habanero software in both binary/byte-code form and source form without royalty payment. However, copyright of the NCSA-developed materials will remain with the University of Illinois. At the University's option, signature custody may be required for source code release. Contractors working for the government, creating products for the sole use of government agencies/activities, will have the same access to Habanero materials as would a government agency.

Technologies incorporated into this project from other collaborators may require license for use of their source code by the government. The binary/byte-code form of these technologies will be available to the government without royalty.

Technologies incorporated into this project from other collaborators, and those technologies developed by NCSA, may require license for use within commercial products of both their binary/byte-code and source code.

Crystaliz, Inc. will distribute the server developed during the course of this project for free to research institutions and universities. Furthermore, the KQML server will be distributed for free for application developers. Commercial licences will be negotiated on a per seat basis.

C. Detailed Technical Rationale

Integrating Synchronous and Asynchronous Collaboration

Asynchronous collaboration involves several interrelated capabilities. For example, when an author submits a document to a review/edit workflow process involving many collaborators, this relatively simple collaboration requires secure, reliable submission and version control, transmission of documents, annotations, and management of the workflow process. Transaction recording, replay, and rollback are also essential for accountability and reliability. Supporting technologies, such as indexing and searching, should also be available, but they are not central to the collaboration problem, which is how people collaborate with each other or with intelligent agents. In comparison, synchronous collaboration involves several of the same kinds of capabilities as for asynchronous collaboration.

In addition to integrating synchronous and asynchronous capabilities, the architecture will generalize across several other dimensions, including workgroup size, organization and distribution, media types, processor and network capabilities, centralized vs distributed services, and remote procedure calls vs mobile code. This generality of the architecture will allow the project to study the performance, scalability, and adaptability tradeoffs of alternative configurations.

The suite of representative applications developed to demonstrate the underlying Habanero/Java object-sharing will itself be sufficient for a variety of distributed collaborations in communities where customized software tools are not ubiquitous. The basic Habanero demonstration suite will, by design, support applications such as distance learning and on-line "help desk" or "reference librarian" conferencing.

CORBA vs Java

Two alternative but complimentary mechanisms for supporting distributed object environments, such as collaboration environments, are CORBA and Java. These mechanisms may be characterized as either moving the data to the code (for CORBA) or moving the code to the data (for Java).

CORBA assumes a heterogeneity of host platform types throughout the distributed system. Using Java, this assumption is unnecessarily restrictive, and can be relaxed. In the Java-based Habanero object-sharing framework, all the disparate platforms running Java appear to be a single platform type. As such, objects can be marshalled, routed, reinstantiated, and executed on the receiving platform. The network dynamism facilitated by this mobility of potentially executable objects will be examined through the construction of inter-operating Habanero clients and Repository components. Dynamic load-balancing capabilities using selectable server-based execution hosting and Applet-like downloads for client-based hosting will be examined.

Session Security

Many of the security needs of the collaborative framework are session-oriented and must be provided by Habanero. When initiating a collaborative session, access control policies must be specified. This includes listing the authentication requirements for individuals joining the session, and choosing mechanisms for protecting the privacy of the session communications, if desired. Joining a session can then involve authentication and key exchange, with the subsequent session data encrypted using the shared symmetric key.

Workflow

Organizations want to maintain administrative control over work flow within their organizations while at the same time providing the best service to the work flow request from outside their organization. Additionally, in long term collaboration, they would want the flexibility to change the way the work flowed within their corporation without affecting the semantic content of the work flow.

Context exchange is supported using KQML's meta level query protocol elements. Organizations that would like to keep their autonomy in determining how the work flows will, in order to support collaboration with other organizations, export information that outside organizations would like to get at. Two types of information will be exported: 1) The semantic context within which work will be generated - this consists of the vocabulary used in the generating organization, and 2) The semantic context within which incoming work will be interpreted. The KQML server will respond to queries from outside organizations by providing the context information. Outside organizations can then use this information to properly translate the information into the semantic context that is appropriate for interpreting the messages.

A requirement raised earlier is for the flexibility to change the way work flows within an organization. This requirement is addressed in the proposed project by using KQML's networking protocols to establish an intelligent directory that is a logical information space on top of existing directories used in the organization. The physical directories include the system directory which keeps information about objects that can be accessed and access control rights on those objects and user address directories (e.g., X.400). KQML's networking protocols will be used to establish a higher level knowledge base on top of these physical directories. Since this higher level knowledge base can respond to deductive queries (using KQML's query protocol elements), intelligent agents can be deployed on top of this directory. These intelligent agents can dynamically alter the work flow, by extracting different actual physical addresses in response to the context of the KQML queries.

The content models to support asynchronous workflow information exchange will be developed at two levels. At the first level, object-oriented descriptions of objects and operations that can be performed on those objects are defined. These objects are typically at a higher level than the ones exposed by the applications. This higher level abstraction reduces event and data traffic across the Internet. This higher level description is built on top of application independent interfaces that operate with specific applications. For example, in order to integrate 3D drawing programs, a lower level description of 3D objects such as solid, sphere, etc., and operations such as intersection, union, etc. will be specified. The higher level description however will consist of parts, features (e.g., holes, slots, etc.), and constraints on those features. The higher level content model is preferred to reduce the data transmitted across WANs.

Annotations

The annotation protocol defined on top of HTTP will be used between servers and clients to communicate about the availability of annotations, the metadata about each annotation, and the text of the annotations themselves. The protocol must also support the creation, editing, and destruction of annotations. Initially, only HTTP accessible annotations of HTTP documents will be supported. The protocol is very similar for both group and public annotations; an implementation of a group annotation server should be usable as a public annotation server.

The largest deployment obstacle for annotations is client-side support, either in browsers or in proxies. Browsers could use the protocol directly or proxies could be used by browsers to augment all documents as they are fetched. With a large number of different kinds of browsers already in use, it is difficult to get new versions of browsers installed. Java-applets might be used to augment document content handling with annotation markup, but security constraints must prevail. We will explore both the Java and proxy routes for deployment.

Notification

Both synchronous and asynchronous collaboration may involve notifications. In synchronous systems, session participants are typically always connected to each other and events are always propagated immediatly to other participants. In asynchronous systems, participants are typically not connected until a message needs to be sent, and then the connection is dropped. To achieve our integrated framework, we will explore how these two extremes can be generalized into one message routing scheme.

Finding out about events of interest to collaborators can be done in two complementary ways: polling and pushing. For scalability to a large number of collaborators and to a large number of events, a system in which clients poll passive information providers for new events should often be replaced by a system in which providers actively notify clients. A prerequisite is that collaborators register to be notified for classes of events, although they may wish to register with intermediate brokers instead of specific servers.

The notification process itself should be distributed: instead of a single server handling all the notifications analogous to an email server, for scalability and reliability we need a web of cooperating, partially redundant servers analogous to the network of NNTP servers. For increased performance, each server in a web may multicast messages with neighboring servers. Unlike NNTP, there would be many independent, relatively sparse webs and each collaborator may subscribe to a different set of webs. To avoid manually configuring each of the many webs, we will develop technology to automatically and dynamically reconfigure them based on server availability and end-to-end latency.

We will explore use of KQML to support notification. KQML assumes a transport infrastructure, be it asynchronous HTTP, NNTP, SMTP, or UDP. KQML is used to provide support for the semantics of publish/subscribe (meta data and semantic protocol support), multicast, and dynamic reconfiguration. Also, when you have several different notification servers, then KQML can be used to synchronize data (i.e., make replicas) across different servers. Also, if you want to group several servers into one logical entity, KQML's networking protocols may be used to group several notification servers into one logical entity. We would then use the KQML server as a way to route the requests to the appropriate notification server in that logical group.

Adaptive Timed-Event Protocol

We want to explore an adaptive continuous and timed-event protocol for several reasons. Although there are many different compression techniques out there already, and we propose to explore some new techniques in that area, the adaptive protocol is aimed at a different problem, that of dynamic adjustment of quality of service based on available bandwidth, and the ability of clients to receive at the available rate. Generally, not much research has been done on adaptive continuous media transmission. We are interested in variable bandwidth real-time transmissions as opposed to constant bit rate transmissions.

Variables in network bandwidth obviously play a large role is such a protocol. First of all, current TCP/IP applications use available bandwidth for maintaining connections thus reducing the effective bandwidth for the continuous media. More importantly, in other systems with quality of service guarantees, there is an assumption of no bandwidth lost to errors or reduced transmission media. Radio and current network hardware have variable transmission rates. Furthermore, quality of service guarantees require bandwidth allocation which is expensive on current low bandwidth networks < OC48. Error recovery for continuous media only makes sense if the latency for the recovery is less than the actual latency for retransmit. We would also like to explore applicability of new ATM and IP protocols to handle quality of service, to guarantee particular real-time bandwidth.

The client is the other major factor in the adaptive feedback loop. The previously developed VDP protocol measures client capacity to process the continuous media traffic stream. If the client is handling multiple video streams, its capacity to handle one or more of the streams may be reduced. VDP handles this dynamically using feedback from screen display to file system. We propose to generalize the VDP to other media as we come to understand their requirements. For example, audio transmission must be more reliable than video transmission since loss of audio continuity is more annoying than loss of a few video frames.

Another important factor to consider for this proposal is the interaction of multiple streams within synchronous collaboration sessions. A session may have different numbers of channels open between servers and multiple clients at different times. Each channel might carry a different type of stream, say audio narration of a simulation, that may need to be synchronized with each other. We wish to dynamically prioritize channels and provide better throughput for high priority channels without interrupting lower priority channel delivery. To do this, we need graceful degradation of lower priority channels, which is not something addressed by today's QoS mechanisms. We will also explore applying similar adaptive techniques to the MBONE [MBONE].

Also related to the timed-event streams, but not specifically to the adaptive protocol, we need to support annotations of these streams, attached to anchors within or paralleling the stream, and delivery of the annotations in their own streams. Each type of timed-event stream may permit general anchors based on the appropriate measure of time (e.g. continuous, or discrete units), or may be specialized further (e.g. events such as "scene change 39"). Interesting user interface problems await, such as how to interact with annotations of streams.

The security concerns raised elsewhere in this proposal apply to all streams and the adaptive timed-event protocol just as much. We will apply and generalize what we have learned from our work in encrypted video streams [EncryptedVideo].

Video/Audio Analysis

The aim of the work on audio/video analysis is to develop advanced software tools to aid the automatic or interactive archiving and synopsizing of multimedia collaborative sessions.

(1) Image/video compression

We shall develop multiresolution representations of image/video using wavelet decomposition. We have recently developed very efficient wavelet-based approaches to coding images and video. These methods exploit the ability of the wavelet transform to compact signal energy simultaneously in space and in frequency. An image can be compressed efficiently because most of the signal energy can be associated with only a few clusters of space-frequency coefficients.

We shall investigate a data structure designed to directly capture the high-density clusters of significant-valued wavelet coefficents. This framework departs from existing methods in that it uses a radically different set of primitive operations (nonlinear morphological operations) to produce a more efficient bitstream, by exploiting dependencies in the wavelet data unaccounted for in existing methods.

(2) Video/audio indexing and synopsizing

(i) Video segmentation and keyframing- A natural way of indexing and synopsizing a video sequence is to segment it into shots and represent each shot by a small number of keyframes. The first step in segmenting is to estimate the camera parameters (zooming, panning., etc.). Then segmentation of the video is done based on changes in image characteristics and motion information. Finally, for each shot keyframes are selected according to saliency. In some applications, it may be useful to establish the relationships between the shots.

(ii) Automatic indexing and synopsizing speech- We propose a detection-based approach which is more suitable than a transcription- based approach because of the anticipated, high degree of un-rehearsed interaction among the participants of a collaborative session. A transcription-based approach which bears the assumption of a well formed linguistic structure often find severe limitation in a spontaneous task environment.

(iii) Correlating speech and video- In a collaborative session, speech and video are highly correlated. In addition to the obvious use of the time line, we propose to develop tools which will be useful in indexing and synopsizing combined video/audio. These will include the combined use of video and audio in: Speaker identification, speech recognition (especially in a noise environment), and facial expression (and to some extent emotion) recognition. A powerful new neuronet architecture will be developed to fuse the audio and the video cues.

D. Detailed Technical Approach

Our general approach to the design and development of the integrated framework is to use object-oriented and distributed object technology where possible, and use existing standard protocols and systems where appropriate. We will approach the tasks from both the synchronous and asynchronous sides simultaneously and, initially, largely independently. This independence is to ensure that we make progress on the more well-known and managable tasks while we gain the experience required to generalize.

Habanero

The Habanero framework will consist of components for session management, event serializing and routing, and several collaborative user interface components or browsers for different media such as streamed data, graphics, or text. Instantiations of these components will provide the combination of features desired by end users. The following figure shows the major components of the framework when configured to use a centralized router:

centralized router

For the forseeable future, Habanero-specific router architecture will be used to facilitate experimentation with scalable, dynamically morphable virtual networks for interconnection of collaborators. A wide variety of routing topologies remain to be examined, with the examination of the associated complexities of automatic initiation of routing topology and/or mode change. This dynamic rerouting capability is essential to session survivability and usage-optimization of potentially scarce network bandwidth.

Repository

The Repository framework will support asynchronous collaboration capabilities including persistent storage and retrieval of the entire session transaction record, annotation and metadata services that may be applied to the session, and registration for subsequent notification about interesting events. The following figure shows the broader range of functionalities in the Repository architecture, including those in support of collaboration:

Continuous Media

CM tend to have very large storage requirements: many hundreds of megabytes. Conventional mass storage systems are incapable of meeting the time constraints for CM; access is not only too slow, but does not have predictable, guaranteed temporal behavior. The requirements for CM can be met by a hierarchical storage management system such as XFS.

Security Extensions Architecture

To allow for extensible authentication, encryption and key exchange within HTTP messages involved in collaborative sessions, security extensions for PEP [PEP] will be implemented in the Web browser and repository server. This will provide a flexible framework for incorporating a range of message security enhancements, easing both the development and deployment of software in compliance with export regulations. We have been working with the technical staff of W3C to refine the Security Extensions Architecture of PEP [SEA]. This architecture lends itself well to implementation in an extensible, object-oriented application environment such as Habanero.

KQML and Security

To integrate with the Habanero environment, the KQML components may need complimentary security services. Ways of including features for authentication, integrity, non-repudiation and encryption into KQML have been proposed [KQMLSEC], and these may be implemented and made to interoperate with the other security systems in Habanero and the Repository.

KQML can be used as the meta protocol to support federation of security domains. Much as we use KQML to extract meta information about roles, routes, and rules, KQML can also be used to exchange context information about security policies and security mechanisms. The context information can be used by the sending server to apply the right kind of security mechanism (e.g., authentication or encryption) on their side. Additionally, security can be used to provide a "smart proxy" server that acts as a front end to an organization's collaboration infrastructure hiding the specifics of the names, addresses, etc.

LogicWare

LogicWare is a KQML-based system that consists of the following parts:

SIMPLE language subsystem: Based on MIT Scheme with extensions to support object oriented programming
A distributed infrastructure based on KQML that supports development of workspaces
A set of gateways that allow LogicWare to interface to databases, tools, etc., using the system specific application inter-communication architecture (e.g. Microsoft DDE, OLE, etc.) and a network layer that can be used to reach distributed implementations of LogicWare using HTTP and TCP/IP.

Asynchronous Workspace

The LogicWare server can be programmed to set up asynchronous workspaces. However, in order to ease the programming burden on the end user or their administrators, we will supply several modules.

First, an ontological dictionary that provides the semantic namespace for supporting collaboration will be developed. This will be done by populating the LogicWare object system with ontological definition of artifacts that users and their agents can create, roles of users and their agents, and semantic description of appliances used in the workspace.

Second, a workspace template class will be provided that can be customized by collaborating users to create workspaces that suite their needs.

Third, a composite template will be provided that can be used to set up a network of workspaces in a particular organization (e.g., hierarchical, recursive).

Finally, we will provide a way to generate 3D views of the collaborating workspaces such that individual workspace users can organize their private view to be different from the public, shared view.

Integration of Synchronous and Asynchronous Collaboration

The architecture of the system we would develop will initially reflect the artificial subdivision between client and server processes, each with its own partial framework. Distributed object and mobile code technology will dissolve this distinction as we integrate into a single framework.

One degree of integration will make use of KQML. The KQML server and the applications described earlier in this proposal would be used to link and arrange a set of intersecting synchronous collaboration workspaces. While the synchronous collaboration would be supported for a workgroup on a networked environment with high bandwidth connections, the KQML server would provide the back bone work flow services between the workgroups. Furthermore, the KQML server would be used to set up workspaces where in members from different synchronous workgroups can participate.

Adaptive Timed-Event Protocol

Here is a diagram of how the server and client interact with the continuous media and the feedback control.

	                Continuous Media
            Server ----------------------------> Client------->|
               ^                                               |
               |                                               |
            Control<-------------------------------------------V
	                Bandwidth Estimation

We will build a model of the communication involved in a synchronous collaboration activity or video browser activity. The model is used to construct control algorithms for the feedback loop used in the activity. Note: we dont really have any way to predict how collaborators will react given the facilities we are providing. Will they open up many channels or a few? How big will the channels be? How reliable do they need to be to make the collaboration effective? Here are some examples of things we have measured so far:

Variability of latency during start up caused by router holding packets while looking up IP addresses.
Rate at which backoff algorithm should drop video frame rate when congestion occurs.
Rate at which algorithm should increase frame rate so as not to endanger stability of control algorithm.
Impact of timed sending of messages and the timed sending of individual packets within a message.
Measurement of round trip latency and the use of round trip latency and bandwidth measurement to retransmit lost continuous media data.
Buffering sizes and smoothing.
Multicast schemes using bandwidth management of traffic through MBONE routers.
Impact of XFS file system and other quality-of-service provisions on adaptive algorithms.

We need to extend our work to accommodate greater variability in adaptability. At present we can manage about 20:1 variations in bandwidth but we need to accommodate even large bandwidth variations if our results are to apply easily to an all-Web continuous media network.

Another necessary generalization (for this proposal) is to provide adaptive bandwidth management from servers to clients in a collaboration setting. We will estimate the bandwidth of the multiple connections (may be many to many) and we will manage the bandwidth transmitted from each server to each host so that TCP/IP/VDP bandwidths gracefully degrade according to prioritization provided by the collaboration requirements.

Regarding deployment of this technology to web browsers, one very viable technique we have just recently explored is the use of downloadable Java applets. This approach agrees with and augments that of the rest of the proposed project.

Video/Audio Analysis

(1) Image/video compression

Though current research has concentrated on the usefulness of the wavelet's compaction properties for achieving high coding efficiency, it is clear that these are also attractive properties for representing image databases. for example, the tree structure of wavelet coefficients provides an ideal framework for developing scalable representations for 3D terrain, which may change with time. Both low and high resolution representations can be accessed from the same database by controlling the depth at which the tree of coefficients is accessed. We shall also be developing embedded versions of our wavelet-based coders. An embedded coding algorithm permits a user to continuously adjust the resulotion of the received representation simply by adjusting the number of bits it accessed, thus addressing efficiently the interoperability requirements that are critical to future architectures. The spatial compaction properties of wavelets give a natural data structure for convenient object-based scene manipulation, indexing, and accessing.

Year 1:: Develop hierarchical image representation using wavelet packets. Develop morphological methods to locate clusters of significant wavelet coefficients.
Year 2:: Complete wavelet/morphology compression algorithm for still images. Develop motion estimation and compensation schemes (based on regions) for video coding.
Year 3:: Complete algorithm for video compression based on wavelets and region-based motion estimation and compensation.

(2) Video/audio indexing and synopsizing. We start with the generized McCulloch-Pitts model that we have recently proposed. This model extends the McCulloch-Pitts model by including an additional set of inputs which regulate the excitability of the neuron, i.e., they control the nonlinear function in the neuron. One potential advantage of this new model lies in its ability to implement nonlinear, dynamic multisensory interactions as observed in the superior colliculus. With this neuron model, we hope to achieve the same quality of approximation as a conventional neuron with a network of reduced size or faster convergence. Research will be carried out to develop suitable network architectures based on this neuron and associated learning algorithms. In particular, architectures involving wavelet decomposition will be investigated.

Archival of a session begins with a specification of the session domain which can either be user-designated a priori or automatically deduced from the session content after completion. Techniques and tools to be developed in the proposal would enable the latter in a systematic manner, eventually accomplishing the goal of domain spotting.

Automatic indexing of text and speech encompasses two main tasks, selection of salient contexts (words, phrases, utterances) to be associated with a particular session and detection and verification of prescribed contexts in a "document." While the selected salient context set can serve as a synopsis of the session, an extended synopsis would also include succinct, descriptive paragraphs which depict linguistically the relationship among the salient contexts as accurately reflected in the session as possible. This would be for future research.

Our approach to automatic selection and detection of salient contexts, words and phrases, from a text is based on an information-theoretic measure of divergence between the probability measure (language model) for a local context which forms a support for the keyword or key phrases and the probability measure pertaining to a general language environment. This is a generalization of and a new departure from several prior arts. A number of key issues need to be considered, such as the effects of function words and their quantifiable roles in the support context, the conditional dependency between hypothesized salient contexts and its effects on the measure of divergence, and so on.

When applied to indexing of speech, a critical component beyond the selection and detection of salient texts is acoustic modeling of speech units. In particular, modeling of acoustic units for speech decoding needs to be formulated for the purpose of maximizing the probability of detection of the intended salient contexts. This is the key characteristic of a detection based approach in that modeling is done to maximize separation of a targeted event from other extraneous inputs which are impossible to model for implementing a maximum a posteriori (MAP) decoder. The methodology of minimum classification error and minimum verification error criteria developed in the past few years has proven effective for this purpose. The framework will be extended and integrated with the text processing components to enable optimal synopsizing and retrieving of a documented work session.

Years 1 and 2:: Develop algorithms for video segmentation and keyframing, including the estimation of camera parameters. Develop algorithms for automatic indexing and synopsizing of speech. Develop new neuronet architectures for fusing audio and video cues.
Year 3:: Complete algorithms for video indexing and for speech indexing. Develop algorithms which combines video and audio in speaker identification, speech recognition, and facial expression recognition.

E. Other Ongoing Research

Repository

Several mechanisms for extending HTTP servers have been developed or proposed, including CGI, APIs for several servers, the ILU Requester proposal [ILU-Requester], and our Method Handler Interface [MHI]. The HTTP-NG working group is moving in a similar direction of adopting object-oriented design and distributed object technology [HTTP-NG]. The Corporation for National Research Initiatives [CNRI] is working on a Repository Access Protocol which is a minimal interface for what features should compose a repository. The underlying framework of servers and repositories is not directly relevant to this proposal, but it is one of the building blocks we are dependent on.

Adaptive Timed-Event Protocol

One other University research area in this topic was finished last year at the University of Graduate Studies in Oregon. They built a software feedback toolkit [SFT] with a simple control loop for sending a single video/audio stream and got results that supported our own research results. There are two companies that claim to have control built into their transport schemes but they both offer telephone or low bandwidth service, not the variable type of service we are providing. MPEG 1/2 generates variable bandwidth at given quality of encoding.

Another related effort is the IVS video conferencing tool [IVS] which uses network feedback to adjust its video encoder output rate. The algorithm is similar to an early implementation of VDP, while our current effort focus on generalization of the algorithm to reliable, as well as unreliable transmissions.

There are also some other adaptive schemes [Adaptive] which require the network switches to send back information such as the buffer length and throughput rate; this approach suffers the scalability problem and is not feasible in the current Internet.

RTP [RTP] is another protocol similar to VDP but much simpler. They've done recent research on support for dynamic QoS but it is very primitive and simliar to the adaptation schemes mentioned above. RTP does not address the reliability problem since there is no resend; instead reliable transmission may require TCP which is essentially a greedy scheme that can uncontrollablly affect other concurrent channels.

The major alternative approach to ours is to allocate bandwidth in a quality-of-service (QoS) scheme. The problems with QoS allocated bandwidth are 1) efficiency, (maximum bandwidth must be accommodated even if data streams are variable 2) latency to change allocated bandwidth if bandwidth requirements change-- renegotating QoS service --- we dont even know if this service will be available in next generation QoS implementations, 3) unavailability of services if admission control policy prevents allocation. As far as we know, QoS and radio would be very difficult to implement efficiently because of the high error rate on radio implementations.

Video/Audio Analysis

Almost all existing work on image/video compression aim at achieving high compression ratios only. Our research pays special attention to the development of compression algorithms which use representations suitable for image content-based retrieval, without sacrificing compression. The use of mathematical morphology in locating wavelet coefficient clusters (which correlate closely with actual objects in the image) appears to be a very promising approach. Furthermore, we have recently developed one of the best wavelet image coder in the world.

A number of research groups have been working on video segmentation and keyframing. A much smaller number (mainly CMU and MIT Media Lab) have started to look into the combined use of audio and video. Our proposed research on video segmentation and keyframing is unique in that we shall pay special attention to motion analysis (both camera motion and scene motion), a topic which we have been doing research on for the last 15 years. In speech indexing, we are taking a detection-based approach instead of the conventional transcription-based approach which will fall apart in an un-rehearsed collaborative session type of environment. Various groups are starting to work on bimodal (audio and video) speech recognition with limited success. As far as we know, no work has been done on bimodal speaker identification and facial expresion recognition. For all these bimodal tasks, a key question is: How do we combine the audio cues and the video cues. We believe that the new neuronet architecture we are developing will be a powerful tool for fusing the cues.

Workflow

We plan on using KQML for scalable workflow support and scalable workspace management for the proposed project because KQML provides features that are well suited to scalability across distances, workgroup sizes, and autonomy of those groups. The KQML protocol is asynchronous, supports object transfer, supports exchange of meta (i.e., context) information, and is self descriptive. None of the other protocols that we are aware of support these functionalities.

First, asynchronous protocols are key to developing systems that can scale across the world. Compared to RPC protocols that support synchronous transfer of object references, KQML supports asynchronous exchange of objects - a key requirement for workflow. Second, as workgroups of different sizes and different functional interfactions are assembled together in a rapid fashion, then there is a need to establish and change a graph of workspaces. Each workspace may be set up to manage specific projects or functional interactions. The establishment of such a virtual network of workspaces is supported easily by KQML. Finally, when these groups are autonomous, cross functional interactions between these workspaces need to involved exchanging information about the context operating in the workgroup. The context may include semantic definitions of the set of vocabularies used, a semantic description of the the set of tools used in the workspace and their interfaces, and semantic definitions of information resources (e.g., databases) that can be accessed by the workspace. The exchange of such meta information is supported by KQML. None of the other protocols we know of support these functionalities.

Integration

Many extensions to web servers and clients to support collaboration have been developed [WWW-COLLAB], most asynchronous but some are moderately real-time. Few systems attempt to integrate both extremes.

The World Wide Web Consortium [W3C] has several working groups on collaboration technologies developing specifications for protocols on annotations, notification, and knowledge representations.

F. Previous Accomplishments

NCSA

NCSA has had a long-standing committment to facilitating the work of computational scientists. We thus have a history of development of tools that make access to research facilities, the interactions of researchers, and the results of that research readily available to users. This history includes NCSA Telnet, NCSA Image, NCSA DataScope, NCSA HDF, NCSA Collage, NCSA Mosaic, the NCSA HTTPd web server, and, more recently, the Habanero synchronous collaboration environment project, described in this document, and the NCSA Workplace environment, which is an initial effort at combining tools for asynchronous collaboration in a web framework.

NCSA Collage was the first concerted effort, started in 1991, to target development of shared workspaces for computational scientists. Collage provided for cross-platform sharing of conversations, text, images, data, drawings, and animations, tools for mutual indication and pointing across these media, and floor control for the maintenance of coordinated sessions. The work proposed here builds on that experience, using new methods and software advances such as Java, KQML and CORBA, and our extensive experience in web technology.

Related research spans a number of areas in web technology, including security, protocol extension mechanisms, server and client development, and object integration schemes, where we are working extensively with W3C and IETF working groups for standards development, and have collaborations with web developers in the commercial and academic sectors. NCSA staff have been instrumental in organizing the series of International WWWeb Conferences, developing tutorials, reviewing and presenting papers, etc, where many of these colaborations have been established and developed.

In the area of CORBA technology we have evaluated various implementations and developed and tested prototype elements for integration with on-going projects. We are working, and negotiating with, key vendors to insure that as much as possible of the base technology needed for this and similar projects is made availabe, on the web model, for general use. The UI Digital Library Project [DLI] is an example of a current application point of these efforts, and a focus of research and development of related large-scale structured document management, searching and presentation efforts, and coordination and synchronization of distributed collections.

Examples of research and development collaborations relevant to this project include Sun, Microsoft, SGI, HP, EIT, OCLC, CNRI, the Bootstrap Institute, over a dozen federal agencies, numerous universities and research labs, and literaly scores of independent developers across the web through our browser, server and related tool development efforts.

In the area of CSCW and the application of existing knowledge of on-line collaborative tools to the development of the 'Briefing Room', we are working with Douglas Engelbart of the Bootstrap Institute, and are initiating collaboration with the developers of GroupSystems at the University of Arizona Center for the Mangament of Information, who have over 15 years of experience in the area of electronic meeting support tools. We have earlier helped in the development of Simon Kaplan's 'Worlds' project, and are following its development for insights in shared workspaces. We are discussing similar developments with a number of research teams and labs, including the W3C working groups Annotations, and related subjects , the Franhoefer Institute-Darmstadt Ubiquitous Computing Project. We are tracking a number of related WWW projects involving collaboration [Collab], including synchronous [Sync] and asynchronous [Async]. Recent development on the GroupKit [GroupKit] software, though limited in its cross-platform applicability, parallels some of our software design ideas.

NCSA has supported distributed teams utilizing HPCC assets at its facility on the campus of the University of Illinois since 1985. A variety of supercomputer hardware architectures have been employed during this period. Computational science and visualization teams at NCSA have produced many advances in the state of the art, including the CAVE immersive visualization environment.

To support its distributed user base, NCSA has developed a series of software products which have gone on to enjoy wide acceptance. All of these products were distributed free of charge.

NCSA Telnet, available on UNIX, Macintosh, and Microsoft Windows provided the first free multi-platform access to remote computing facilities.
The platform-independent file system, Hierarchical Data Format (HDF), has been subsequently adopted by both NASA and the Japanese Space Agency for archival storage of huge volumes of imagery data.
NCSA Collage provided real-time collaboration facilities over Internet for users on all three platform types. Numerical visualization facilities were also included.
NCSA Mosaic provided unprecedented ease-of-use in information retrieval to users on all three platform types, and is widely credited with being a critical link in the birth of the World-Wide Web. Several million copies of Mosaic were distributed. A 10,000-fold increase in Web traffic over Internet occurred between the initial offering of Mosaic and the arrival of the first commercial browser software.
NCSA's HTTPD (Web) server continues to be the single most widely used Web server. Traffic on NCSA's own server facility exceeded 4,000,000 hits per week in mid-1994, approximately 18 months after Mosaic was introduced.

NCSA has been working with security technologies for some time now, as exemplified in our development of Web support for Kerberos, Secure-HTTP and PGP (Pretty Good Privacy). Our recent collaborations with W3C refining the security uses of the Protocol Extensions Protocol [PEP] can also be brought to bear in this effort.

A proposal entitled "Java-Based Framework for Collaborating Science Teams" was submitted to E.O. Lawrence Berkeley National Laboratory of the Department of Energy under Request for Proposal 4616610 concerning adaptation of the Habanero design to focus on the needs of distributed teams working in the physical sciences. If funded, that proposal's emphasis on a variety of multicasting technologies, physical science observations and data capture, session authentication provisions, and construction of a prototype Scientist's Notebook (based on a scaled-down version of a generalized object store facility) would enrich the Habanero project herein described.

Vosaic

Vosaic is a web browser supporting continuous media that operates on-demand, in real time, using available Internet bandwidth. It seamlessly integrates video and audio into Web documents and makes the transmission and display of such materials across the network efficient and timely. Video and audio transfers occur in real-time; there is no file retrieval latency.

An adaptive real-time transmission protocol VDP uses IP to reduce inter-frame jitter and accommodate client CPU load, decompression software, and network congestion. The video and audio result in compelling Web pages. Meta-information is transferred along with the video and audio allowing embedded hyperlinks within the video. Extensions to html specify starting and stopping frames and indicate whether the video has controls, plays cyclically, or runs from beginning to end.

Experiments show a forty-four fold increase in received video frame rate with the use of VDP in lieu of TCP, the normal transmission protocol for Web documents. A testbed has been established by distributing Vosaic, its video server, and tools to build continuous media documents. Some 450 sites have copies of Vosaic, 200 sites have copies of the server. Four external sites have set up their own public video servers. Videos with transmission rates that vary from 20kbits/sec to 800kbits/sec are available for evaluation. Some 12 hours of lectures have been recorded and transcribed into Web material.

We have a long history of work with object-oriented systems, one of the core technologies in the proposed project. We have built Choices, a multiprocessor operating system in C++, MicroChoices, a microkernel written in C++. We have an inordinate number of papers on object-oriented systems design including current ICSE conference and previous OOPSLAs.

Crystaliz

During 1994 and 1995, one of the core technologies of the proposed project - Knowledge Query Manipulation Language (KQML) has been worked on by Crystaliz Inc. KQML is a protocol and language for knowledge sharing that was developed in the ARPA Knowledge Sharing Effort and in the Intelligent Integration of Information program. As such, it was a research technology. Additionally, it was developed to support shipping of first order predicate calculus expressions.

During 1994, we were funded to the tune of $100,000 by ARPA (in SBIR funds) to develop a server for the KQML protocol. In addition to the above amount, we invested our own resources (to the tune of $250,000) in 1994 and 1995 to develop a commercially viable technology that would also support shipping of procedural programs. Additionally, we have been working at OMG to release an RFP to properly support mobile code within the context of Object Management Architecture. More recently we have also invested our own resources into investigating the Java virtual machine and the Scheme-48 virtual machine. In addition, companies such as Tandem and IBM are working with us to understand the impact of the KQML technology in their future plans. All this missionary, market development work was done with our own resources. We have tirelessly marketed the KQML technology and will continue to do so.

G. Facilities

NCSA has a vast array of advanced computer hardware, and is committed to staying abreast of the state of the art, particularly in the area of MPP. NCSA also maintains a state-of-the art facility for high speed networking, storage, and filesystems and has migrated toward scalable, distributed systems technologies, like AFS and UniTree, and high-capacity multi-media storage systems such as XFS, which lay the foundation for the development of massive digital libraries. Potentially, even very high-capacity communications experiments are possible utilizing the NCSA T-3 Internet connectivity.

A variety of workstations of all types are available at NCSA for use on this project, either exclusively or shared. Java-capable operating systems are already installed on these workstations. The proposed project budget requests funds for workstations only for personnel to be hired as a result of funding this project.

The work on Video/Audio Analysis will be performed at the Image laboratory of the Beckman Institute. The Lab has state-of-the-art equipment for the acquisition, analysis, and visualization of high-resolution images and video. Facilities include a MasPar parallel computer (4000 Processors), an SGI ONYX, 15 workstations (SUN, SGI, HP), ABEKAS digital video disk, Viewgraphics solid-state digital video memory, Sony D1 digital tape machine, high-resolution monitors. An annex of the Image Laboratory contains speech acquisition and analysis facilities, including the HTN tool kit from Entropic.

Crystaliz Inc. currently has network workstations and PCs installed. These include 3 Pentium class PCs running MS-Windows, MS-NT, and Linux. Additionally, it has a Sun workstations and a NextStep workstation. Furthermore, we will be acquiring workstations during the course of the proposed project to add to this network. The software infrastructure at Crystaliz Inc. consists of development tools, Object Request Brokers from Sun and DEC, and HTTP servers and clients.

For the adaptive video component of the system, we need newer workstations with larger memory and MPEG2 encoding (at least MPEG1) as well as the Teleconferencing standard encoding. We need appropriate hierarchical tertiary as well as secondary storage, e.g. XFS.

H. Qualifications of Key Personnel

Joseph Hardin: (B.A., History, University of Illinois, 1972) is the Associate Director for Software Development at NCSA. He has overseen the development of a variety of connectivity, scientific visualization, and collaboration projects, including NCSA Telnet, NCSA Collage, and NCSA Mosaic. He has been published in the areas of network and WWW systems development, collaboration support systems, digital libraries and hypermedia. He is currently a member of the International World Wide Web Conferences Committee, which he helped found. He is the PI or Co-PI of a number of NSF, NASA, and/or ARPA grants in the areas of web collaboration technology research and development. Joseph will serve as a PI have high level responsibility for personnel allocations and administration of this proposal.
Daniel LaLiberte: (B.S. in C.S., University of Minnesota,1978) Joined NCSA in 1988 working on scientific visualization and collaboration tools. He is currently investigating WWW architecture issues including searching, URIs, repositories, and annotation capabilities and currently a CS graduate student at the University of Illinois, Urbana-Champaign studying software engineering, programming languages, and the evolution of information organization. Dan will serve as a PI for this proposal and be responsible for the technical management of the integrated framework.
Elizabeth Frank: Elizabeth Frank received her BA from Wellesley College, and her Masters of Computer Science from Rutgers. She worked for AT&T Bell Laboratories for 7 years working on Net 1000, System 75, and Technical Support for Sales. After a 2 years as an independent consultant writing Unix drivers and doing project management, she held a one year position at the University of Illinois in the Computer Science Dept. acting as a liason between INTEL and the Pablo Project (run by Prof. Daniel Reed). Elizabeth joined the NCSA, SDG Server Development Team in Feb. 1995, and is currently the Technical Manager of the Team.
Larry Jackson, Senior Research Programmer: ( B.S. in C.S., University of Illinois,1975, M.S. in C.S., University of Rhode Island,1984) Served 5 years as an engineering officer in the nuclear submarine fleet. For 14 years, his subsequent professional efforts were in real-time computer operating systems and simulations for military applications, as a Principal Engineer with Computer Sciences Corporation, Unisys, and General Dynamics (Electric Boat Division). He joined NCSA in the Spring of 1994 as the Technical Manager for Mosaic. In the Fall of 1995, he was named Technical Program Manager of a team to investigate Java-facilitated extensibility of Internet collaborative technologies. Larry will serve as a PI and supervise technical staff responsible for the Java-related collaboration efforts of this proposal.
Thomas S. Huang: (Sc.D. MIT). Is currently a professor of Electrical and Computer Engineering at the University of Illinois in Urbana/Champaign. He is Chair of HCI major research theme at Beckman and an Institute. Fellow of IEEE, OSA, IAPR. IEEE Signal Processing Society Technical Achievement Award and Society Award. He has published 11 books and over 300 papers on image processing, compression, and understanding. He will supervise one RA and be responsible for the Bell Lab consultant.
Rajeev Sharma: Received his Ph.D. from the University of Maryland at College Park in 1993 winning the ACM Samuel Alexander doctoral dissertation award, Washington DC. He currently holds the position of Beckman Fellow at the Beckman Institute and Adjunct Assistant Professor in the Department of Electrical and Computer Engineering at UI. He has published more than 50 papers in refereed journals and conference in the area of computer vision, vision-based robotics, and human-computer interaction.
W. H. (Fred) Juang: (Ph.D., UC Santa Barbara) Is Group Leader for Speech Understanding and Senior Technical Member, AT&T Bell Laboratories and Adjunct Professor of Electrical and Computer Engineering, UIUC. He is co-author of a recent book on Speech Recognition and has published over 100 papers on speech processing, compression, analysis and understanding.
Sankar Virdhagriswaran: (MBA in MIS, course work for M.S. in CS at University of Minnesota and at M.I.T.) Has worked at Honeywell Information Systems and Crystaliz, Inc. Sankar was engineering manager and product manager at Crystaliz, Inc. for a variety of products including expert system tools and collaboration support tools. Sankar will be project manager for Crystaliz, Inc. at 25% for 2 years. Crystalize will employ Mike Webb 75% for 2 years, and Gordon Dakin 100% for 2 years.
Mike Webb, Lead Engineer: Mike Webb received his M.S. in computer science from RPI in 1986. Since then he has worked at Applicon and at Crystaliz Inc. At Applicon, Mike was responsible for working on database engines for CAD tools. As the Mechanical CAD market place changed from a single user orientation to supporting collaboration, Mike was responsible for introducing advanced, database technologies into Applicon's product line. At Crystaliz Inc. he has been responsible for implementations of the core portions of the KQML technology.
Dr. Gordon Dakin, Support Engineer: Gordon is a recent graduate from University of Massachusetts, Amherst, MA. He received his Ph.D in computer science in Sep. 1994. While at University of Massachusetts, he was involved in complex systems development including robotics control units, software programming environments for robots, etc. Gordon was responsible for implementing 3D model extensions to KQML's content language. Currently, Gordon is working on extending KQML's content model language to include HTML concepts, project planning concepts, and VRML concepts.
Roy H. Campbell: Professor in the Department of Computer Science at the University of Illinois. (B.Sc. in mathematics from the University of Sussex, Sussex, England the M.Sc. and Ph.D. degrees in computer science from the University of Newcastle upon Tyne, Newcastle-upon-Tyne, England) He has been with the department of since 1976 and was promoted to full professor in 1985. He is principal investigator of Video Mosaic project that is embedding hyperlinked real-time video delivery into the Web and the Choices project which is constructing class-hierarchical object-oriented operating systems for shared memory and distributed memory multiprocessors. His research interests include distributed and parallel systems, operating systems, programming language design, netowrks, testing, and software engineering. He supervise 4 graduate students working on the adaptive continuous media protocol.

I. Cost Breakdown

Section IV. Additional Information

Bibliography

[AAA]: "Agents, Agencies, Artifacts, and Appliances", Sankar Virdhagriswaran, et. al., World Wide Web Conference, Nov. 1995, Poster Session
[Adaptive]: "An adaptive congestion control scheme for real-time packet video transport", H. Kanakia, P. Mishra, and A. Reibman, In ACM Sigcomm'93, pages 20--31, San Fransisco, CA, Sept 1993.
[Annotations]: LaLiberte, D., Braverman, A., A Protocol for Scalable Group and Public Annotations, http://union.ncsa.uiuc.edu/~liberte/www/scalable-annotations.html
[Carnot]: D. Woelk, W. M. Shen, M. Huhns, and P. Cannata, "Model Driven Enterprise Information Management in Carnot," Enterprise Integration Modeling: Proceedings of the First International Conference, The MIT Press.
[CNRI]: The Corporation for National Research Initiatives, http://www.cnri.reston.va.us/
[Collab]: WWW Collaboration Projects, http://union.ncsa.uiuc.edu/HyperNews/get/www/collaboration.html
[Collab-Eng]: "Collaborative Engineering Based on Knowledge Sharing Agreements", G. Olsen, et. al., http://www-ksl-stanford.edu/knowledge-sharing/papers/index.html
[Collab-WG]: Collaboration, Knowledge Representation and Automatability, http://www.w3.org/pub/WWW/Collaboration/
[CORBA]: Common Object Request Broker Architecture, http://www.omg.org/
[DLI]: Digital Library Initiative, UIUC home page, http://www.grainger.uiuc.edu/dli/
[EncryptedVideo]: "Security Enhanced MPEG Player", Yongcheng. Li and See-Mong Tan and Roy H. Campbell March, 1996, First International Workshop on Multimedia Software Development Web Conference Berlin, Germany.
[FedCon]: NCSA's Federal Consortium, http://www.ncsa.uiuc.edu/SDG/Presentations/FedCon/Shared.html
[GroupKit]: GroupKit, http://www.cpsc.ucalgary.ca/projects/grouplab/groupkit/
[HTTP-NG]: Hypertext Transfer Protocol - Next Generation, http://www.w3.org/pub/WWW/Protocols/HTTP-NG/Overview.html
[HyperNews]: LaLiberte, D HyperNews - a web-based bulletin board system, http://union.ncsa.uiuc.edu/HyperNews/get/hypernews/about.html
[IONA]: IONA's Orbix, http://www.iona.ie:8000/www/index.html
[ILU-Requester]: Paul Everitt, Digital Creations, "The ILU Requester: Object Services in HTTP Servers" March, 1996, W3C Inforamtional Draft, http://www.w3.org/pub/WWW/TR/WD-ilu-requestor
[IVS]: "A rate control mechanism for packet video in the internet", J-C. Bolot and T. Turletti., In Proc. IEEE Infocom'94, pages 1216--1223, 1994.
[Java]: Java, http://Java.sun.com/
[KQML]: Knowledge Query Manipulation Language Specification, http://www.cs.umbc.edu/kqml
[KQMLSEC]: Thirunavukkarasu, C., and Finin, T., "Secret Agents -- A Security Architecture for the KQML Agent Communication Language" October, 1995, Draft submitted to the CIKM'95 Intelligent Information Agents Workshop, Baltimore, December 1995 http://www.cs.umbc.edu/papers/secret.ps
[MBONE]: Multicast Backbone on the Internet (MBONE), http://www.eit.com/techinfo/mbone/mbone.html
[MHI]: Method Handler Interface, http://union.ncsa.uiuc.edu/~liberte/www/repo/mhi.html
[Neighborhoods]: Grossman, E, Kothari, J., Neighborhoods: A Protocol For Facilitating Synchronous Collaboration , http://bombay.ncsa.uiuc.edu/paper/Neighborhoods.html
[OSF]: Open Software Foundation, http://www.osf.org/
[PEP]: Khare, R. "PEP: An Extension Mechanism for HTTP/1.1" Feb 22, 1996, W3 Consortium Working Draft http://www.w3.org/pub/WWW/TR/WD-http-pep.html
[Repository]: LaLiberte, D NCSA Repository Design, http://union.ncsa.uiuc.edu/HyperNews/get/www/repository.html
[RTP]: "Dynamic QoS Control of Multimedia Applications based RTP", I. Busse, B. Deffener, H. Schulzrinne, Computer Communications, Jan. 1996
[SEA]: Khare, R. "SEA: A Security Extension Architecture for HTTP/1.x" Jan 8, 1996, W3 Consortium Working Draft http://www.w3.org/pub/WWW/TR/WD-http-sea.html
[SESSION-ID]: Session Identification URI, W3C Working Draft WD-session-id-960221, http://www.w3.org/pub/WWW/TR/WD-session-id.html
[SFT]: "Adaptive Methods for Distributed Video Presentation", C. Cowan, S. Cen, J. Walpole, C. Pu, In ACM Computing Surveys, Vol 27, No. 4, Dec 1995.
[Sync]: Synchronous Collaboration Projects
[Teamrooms]: Teamrooms, http://www.cpsc.ucalgary.ca/projects/grouplab/teamrooms/
[VDP]: Zingang Chen, See-Mong Tan, Roy H Campbell, and Yongcheng Li, Real Time Video and Audio in the WWW. In Fourth Int. World Conference. Boston, MA, Dec 1995.
[wOrlds]: wOrlds, http://acsl.cs.uiuc.edu/kaplan/worlds.html
[WWW-COLLAB]: WWW Collaboration Projects, http://union.ncsa.uiuc.edu/HyperNews/get/www/collaboration.html
[W3C]: The World Wide Web Consortium, http://www.w3.org/