The Conception of Cooperative Environment for Editing Multimedia Documents with Workflow Technology (CEMT)

CNPq-INRIA Project


Jose Valdeni de Lima(1) Nabil Layaida(2) Nina Edelweiss(1)

Carlos Zeve(1) Manuele Kirsh Pinheiro(1) Tiago Telecken(1)


Instituto de Informática

Universidade Federal do Rio Grande do Sul
Av. Bento Conçalves, 9500
Campus do Vale - Bloco IV

Porto Alegre


Email: {valdeni, nina, telecken}


INRIA - Rhône-Alpes

Institut National de Recherche en Informatique et en Automatique

ZIRST - 655 Avenue de l´Europe, 38330 Montbonnot Saint Martin, France


In the recent years, the Web became a great instrument for publishing and delivering online multimedia documents. Therefore, the authoring of such content became one of the most important aspect for e-learning environments as they are traditionally large producers and consumers of teaching content. In such environments the, Web may be used not only to present the information to students, but also to connect professionals (for example, professors or students) to create material for e-learning courses. This report presents the CEMT project, including the web-based environment defined for editing multimedia documents with workflow technology. The workflow technology provides a group of users scattered across the network the means of guiding the authoring process by controlling the cooperative work. The most modern and standard web technologies, like XML and SMIL, and open tools, like Amaya and Apache server, are used. This guarantees to the proposed environment a very low cost and a wider deployment, specially appropriate to the economic reality of under-development countries. Another aspect, concerning the encoding and presentation of time based multimedia and hypermedia applications is also treated in the CEMT Project, through synchronization languages and tools. These later allows the specification and the rendering of user interactions or the synchronization of pieces of multimedia elements of the presentation. This report presents the defined platform, the chosen strategy and the tasks executed until 10/09/2001 in the CEMT Project.

1. An Introduction to the CEMT Project

This project was proposed by Prof. José Valdeni de LIMA (II-UFRGS - Project SIGHA) and the researchers Cécile ROISIN and Nabil LAYAIDA (INRIA - Project OPERA), with the aim of developing an cooperative environment for the edition of multimedia documents, using Workflow concepts to coordinate the editing and rendering tasks.

The Conception of Cooperative Environment for Editing Multimedia Documents with Workflow Technology (CEMT) Project, begun officially in September, 2000. In fact, its activities were started when Tiago TELECKEN was accepted as a research fellow, at November, 2001. Also, the allocation of one Master of Science student and two graduate students to the project, and the work-mission of Manuele Kirsch PINHEIRO, from July, 2001 to July, 2002, in INRIA, were the important steps for the CEMT Project development.

Recently, in Porto Alegre took place the I Workshop CEMT with several participants as Nabil LAYAIDA, Lionel VILLARD, José Valdeni De LIMA, Tiago TELECKEN, Valter ROESLER, Eidy Guandeline, Carlos ZEVE. The program of this workshop is attached in this document. As a result of this workshop, all the tasks to be devolved by the project members in the second part of the project were discussed, and the project schedule for that period was defined. These reports are being made available in project home-page, at the following address

This paper presents some details of the Project CEMT and some of the activities already achieved in this project. We present along the following sections : Project, where we present details about the environment that we propose in this project; Workflow, where we discuss a little about the Workflow concepts used; Developed Tasks, where we present the activities already developed in this project.

2. The project

2.1 Objectives

In the authoring process, the creation of didactic material is not an easy task. When e-learning is concerned, this task is even more complicated. If we add the use of multimedia and other advanced technologies, the complexity of the system increases substantially. In order to diminish this complexity, the process of didactic material authoring for e-leaning, what we call in this paper "e-learning authoring", often is achieved according to a cooperative process. A group of educators and technical people usually cooperate to define, design and deploy the content on the underlying computer infrastructure.

Besides this, the presence of Web technologies deployed at a world wide level should be highly considered. The use of the WWW for training and e-leaning purposes is unquestionable, and is growing even more with the latest Web technologies, like XML and SMIL. But, in this project the focus on the WWW use should not only be considered as a medium to present e-learning information. We may (and should) use it as an authoring environment, used to connect together groups of people, that may be geographically distributed. This allows better content production support and to reach this goal we have decided to use all the technologies and available products out there in the experimental framework that we are currently developing.

In this section, we present a cooperative authoring environment for multimedia e-learning based on the WWW which is the main goal of this project. In this framework, we experiment the use of modern technologies, like XML, SMIL and WebDAV, and open tools, like Amaya and Annotea server, to create a low cost environment for e-learning. Moreover, on top of these components, we purpose the use of a workflow engine to control the access and creation of courses in this environment. The workflow allows to guide all the authoring process, helping the group to efficiently produce their e-learning material. The result, as is shown in this Section, is an environment divided in four basic elements: one workflow engine that controls the other three elements, a web authoring tool, multimedia authoring tools, and a set of functions supporting the cooperative work (cooperative elements). Those elements are discussed separately in the next sections: Section 2.2 shows how the web pages are constructed (the web authoring tool); in Section 2.3 the production of multimedia is added (the multimedia authoring tools); and Section 2.4 shows the cooperative aspects of the proposed environment. Finally, the Section 2.5 presents the currently used workflow engine, which brings together all the elements; and finally, some preliminary conclusions are presented.

2.2 Constructing web pages

The first element of the proposed environment is a web editor to create the web pages for an e-learning course. We have chosen the editor Amaya, a browser/authoring tool developed by the W3C team in grenoble which partly overlapping with the opera research team. Amaya allows us to publish documents on the Web. "Amaya is a complete web browsing and authoring environment and comes equipped with a 'WYSIWYG style' of interface, similar to that of the most popular commercial browsers." [W3C-Amaya 01] As it is used by W3C to demonstrate and test new developments in Web technologies, it includes very modern features, like XML and CSS support. Thus, by using Amaya, users can generate pages in HTML and XHTML, generate CSS style sheets, MathML expressions and mainly SVG vector graphics. Through the SVG drawings, symbols similar to those used in the "Workflow Process Model of WIDE Project" [WIDE 97] (which will be used in the project) may be created. It generates XML files, which will contain data about the workflow of e-learning. Thus the user, through a visual interface, may construct the workflow of an e-learning course and insert it in the workflow engine controlling the entire cooperation process.

Amaya also implements the Annotea client, a collaborative application based on the state of art technologies like RDF, XLink and XPointer. Using this client, users involved in a e-learning authoring process can insert annotations in theirs courses' documents. Therefore, using Amaya with this Annotea client, the users have a cooperative function and a communication channel, which makes it possible to them to insert and to exchange comments, ideas, experiences, etc.

In complement of that web-authoring tool, the proposed framework uses a specifically configured HTTP server. This server permits the users to publish their pages and make these pages available for the co-authors, allowing the construction of the cooperative environment needed for the e-learning authoring process. This same server, is enriched to become a genuine cooperative environment by the addition of one Annotea server. This later will be discuss in the Section 2.4.

2.3 Adding multimedia

As mentioned earlier, the importance of multimedia for e-learning is unquestionable. There is no better manner to support e-learning without including multimedia in the courses material. In addition, there are many tools to create media elements, like images, sound and video, and thousands of media elements are already available over the Internet. But, just putting all these media items together in the same document will not necessarily enhance the course quality. Those media should be synchronized to make a good presentation of a document and reflect over time the content used by the teachers like gesture, demonstrations and examples. We concentrated our work here in the use of multimedia synchronization tools that give to the users the possibility of coordinating and constructing rich multimedia presentations as close as possible to the different media used in a traditional classrooms.

One of most recent technologies for multimedia synchronization is the SMIL language. The SMIL (Synchronized Multimedia Integration Language) is an HTML-like language proposed by W3C that permits the integration of many different media types, like streaming audio and video, images and text. Generally speaking, it "enables simple authoring of interactive audiovisual presentations" [W3C-SMIL 01]. With SMIL, other useful technologies have been born. It is the case of XHTML+SMIL. This one includes a subset of the SMIL 2.0 specification, providing support for animation, content control, media objects, timing and synchronization, and transition effects, with XHTML and CSS features [XHTMLplusSMIL 01].

What is purposed in our environment is the use multimedia-authoring tools that understand and produce SMIL and SMIL+XHTML documents. These documents could be linked to and from normal web documents, such as the documents created by Amaya, and presented by Web browsers (like Microsoft Internet Explorer 5.5 presents XHTML+SMIL2.0) or by its plugins (like Real Player plug in, in Netscape browser, presents SMIL1.0).

As a consequence, the users may add the results of these multimedia synchronization tools with the documents created by Amaya, and doing so, they may create a complete set of hypermedia documents for their e-learning courses.

2.4 Working cooperatively

The idea of a group work is to gain efficiency to solve complex problems and to reach a better final product than if each group member was working separately. We expected that the work in groups will develop products with better quality and productivity, through a cycle of enhancements and cooperative interaction from the group members [DIA 97]. So, in order to integrate this aspect in the e-learning authoring process, we provide a support for people to form groups for the design and creation of e-leaning courses. Thus, it's possible to the authors to divide the process in smaller pieces, to enhance productivity, to take profit of the authors' expertise, and also to discuss, exchange points of view, etc. Finally, working in a group, the authors may enrich the content of one another and reach a better final product.

In the proposed environment, we chose to adopt a cooperative approach, but we don't use a bigger and complex groupware system. We chose to work with a few cooperative elements, that we think necessary for a successful deployment of the framework. Thus, we have chosen to improve our tools with cooperative elements, like awareness and annotations, while we remained closer to simple applications design that work with open and widely known protocols and technologies. The cooperative elements used by us are detailed below.

2.4.1 Awareness

As we said before, we expect the group work to produce better results by the harmonic interaction of the group members. To reach this harmonic interaction, users need to be aware of the colleagues' activities, because, according to Gutwin and Greenberg [GUT 99], being aware of the colleagues and of their activities is important to make the tasks of the workflow natural and fluid.

This led us to the concept of awareness. Awareness is the knowledge on group activities, including past, present and future activities. It's also the knowledge about the group itself and its overall status. Without this knowledge, a coordinated and structured cooperative work becomes almost impossible [PIN 01a]. It is the awareness support that makes it possible to transform irregular interactions in consistent interactions over time, allowing the members updated on the important events for the group [DIA 97], [ARA 97].

So, to help a group of professionals to create an e-learning course, it's necessary to give them some support for awareness. To achieve thos goal, we enriched the tools used in the proposed environment with a small, but useful, awareness module. This was done by the addition of a framework, called BW Framework, designed to provide awareness support, specially in asynchronous way (like it is the WWW) [PIN 01b]. Combining this framework with the other authoring tools, users could receive the notification of many events and even perform queries to filter them Such notifications are related to document changes or new annotations inserted in a document.

This combination between the chosen tools and the BW framework is possible because this framework was design not only to create new groupware systems, but to be integrated with existing ones. It works together with the application, and it structures the awareness information in such a flexible way that every application may describe the activities or events that are significant to the workflow. The BW looks at this description made by the application and based on this, provided the notification of the groups' activities to the subscribed users in the workflow.

2.4.2 Using annotations with Annotea

Annotations are an interesting resource in cooperative environments. Using annotations, the co-authors of a document may exchange experiences, opinions, comments, questions, etc, and enrich the cooperation process. It acts, in fact, as a communication resource, which gives to the group the possibility to asynchronously discuss about the work they are doing together.

We chose to use this resource of annotations in our environment to make it truly cooperative, and for that, we chose to use the Annotea server. "Annotea is a web-based shared annotation system based on general-purpose open RDF infrastructure, where annotations are modeled as a class of metadata" [KAH 01]. It treats annotations as external statements made by an author about a Web document, and it stores this annotations in central RDF bases in one or more annotation servers [KAH 01].

As Amaya implements also an Annotea client, the users could easily use this resource without changing the application, doing theirs annotations directly on the documents they are constructing. By using annotation, users exchange ideas, comments, and questions. The annotation serves not only for authoring but also as a communication channel between the teachers and the students reading the courses.

2.4.3 WebDAV

In the sections above, we said that using Amaya to create and publish web pages requires a well-configured HTTP server. However, just this combination Amaya/HTTP doesn't cover the question of "lost update"(when a user's update is overwritten by another user's update). To treat this question, we planned to introduce in the Amaya editor the Internet protocol design for authoring purposes, the WebDAV. The WebDAV _ WWW Distributed Authoring and Versoning - "is a set of extensions to the HTTP protocol which allows users to collaboratively edit and manage files on remote web servers" [WEBDAV 01]. It aims to extend HTTP protocol to give place to an open architecture at the protocol level to develop new distributed authoring tools into the web. WEBDAV especially emphasize on the collaborative authoring of web pages [SALCEDO 98], [WEBDAV 99].

2.5 Putting all together

To connect and organize all those features presented in the sections above, we introduced in the proposed environment a workflow component. This element controls all the e-learning authoring process. The workflow defines the tasks and its circulation between the different users for the creation of the course. The workflow is not involved in the control of the presentation of the course, to define what tasks the students should do to follow the course, but it is designed to help professionals to create a course, to create e-learning material. The data about this workflow will be generated by SVG drawings as showed in Section 2.2 and stored in a specifically designed DTD conforming XML file.

Our proposal is to introduce a workflow engine inside the server (the same server HTTP and Annotea), and a client of this engine in each co-author machine. This engine may read, in the server, a description of the workflow made in a predetermined language, a DTD XML that we are currently defining. By interpreting this description, the engine will understand what task, resources and constrains the workflow have, and will be capable to control it. Besides this description, the workflow engine server may use the awareness information to better control the authoring process. For example, he could see which documents are completed by the awareness events such "final document save". These awareness information are maintained by the framework BW in a centralized base. This base could be periodically consulted by the workflow engine to make some decisions on the workflow progression.

In the other hand, the client element of this engine will act like a front-end for other applications, like Amaya. Using this client, the users may request a task, following the workflow sequence and constrains, and this client will call for them the appropriate tools (for example, if the task is edit a web page, it will call the Amaya tool). Another feature planed in the client side of the workflow is to give to the users the possibility of interactively supplying information about the realization of tasks manipulated by the workflow engine.

Thus, the Figure1 below shows the complete environment that we have proposed:

Figure 1: CEMT Environment

This workflow engine, the client and the server, are still under construction. We already have a preliminary version of the workflow definition tool (see [TIS 00]), but there is much more to accomplish in order to have it functional, especially in the server side. This represents the most complex and sensitive part of the proposed environment, but we expect to obtain a proof of concept version during the year 2002.

3. Workflow

A workflow management engine can be integrated with others technologies, in particular as a support technology for the management of a cooperative authoring. A convergence between workflow management and cooperative authoring occurs in order to make a specification of cooperation at the conceptual level, where we can formalize the cooperation authoring model, and also at the execution level, where the workflow engine controls de execution of the cooperation edition model.

At the conceptual level, a user may provide a sequence of phases to be followed in the authoring of documents, and he/she can refine a phase into more detailed phases. Such view of the specification authoring process reflects what activities are needed to model the construction of the document. This requires adapting the document structure to the activities construction model. On the fly revision of the workflow is also to be studied in order to provided more flexibility to the overall system.

At the execution level, the application controls the activities that involve coordinated execution of multiple tasks, which can be coordinated in different instances of the same specification model.

4. Developed Tasks

Besides the planning and definitions showed in the sections 2 and 3, further activities were executed in the project.

Initially, a detailed study of the tools to be used in the Project was made. Among them are:

A series of experimental tools was installed for creating a platform for developing and implementing the project framework. In this effort, was installed and tested softwares as Byzance, Amaya, Madeus and Big Watcher. We also installed a Web server dedicated to the project (see, and over this server, we installed features as WebDAV publishing modules and an Annotea server. For the installation of all those systems it was necessary to establish the communication between all the machines involved, at UFRGS and at INRIA, and the installation of many tools and software components, such as libraries, databases and server modules (on linux).

After the tests realized with Byzance and the study of the source code of this software, an initial integration of the framework "Big Watcher"[PIN01a] in the software Byzance was implemented.

However, after many studies and tests, the members of the CEMT concluded that these system (Byzance) was more complex and has limited support for new document standards (SVG, MathML, XML) and protocols. Also, the software Byzance is no more supported by the Opera group, despite of its cooperative capabilities, and it demonstrated to be too heavy to be maintained within this project. Because of this, the implementation of the workflow within the Byzance software was replaced by Amaya. Amaya offers to this project the possibility to work with up to date technologies, like we have seem in Section 2.2, and also there are many similarities in the code, because a major portion of Byzance source code is, in fact, build on older version of the Thot library which is at the heart of Amaya. So, all the knowledge acquired and the experiences gained with Byzance was used for Amaya and also in the workflow definitions.

In the Amaya system, we added the framework BW, using the same structure used before in the Byzance implementation. The structure of this work is shown in the Figure 2.

Figure 2 - BW-Amaya Architecture

The software Amaya was written in C language and software BW was be written in the Java language. To make the communication between the two systems, a TCP/IP communication module was attached in the BW, and a TCP/IP client in the Amaya system.

The client module feeds the Amaya system with the data about events and orders, and sends these information to the server module by a protocol defined by the group. The server sends the information to the BW system and the BW system sends the data or the orders to get data to the awareness server. This later stores the information or sends to BW the required information (and the BW sends the information to the Amaya system by the same way).

After the definition of how to use of the Amaya system in replacement of the the Byzance system, further changes were also made to accommodate this new environment. In particular, we decided to take advantage of the availability of newer technologies and systems such as the Webdav, Annotea and Madeus systems even if they were not included in the initial submission of this project. Consequently, the installation and configuration of these technologies and of their environments of development were made. Finally after the installation of the new modules, many tests about their functionality and integration were performed. More details about these technologies was presented in the section 2.

A model of the workflow of the tasks implemented in the Byzance system was also constructed, using the representation proposed by Casati in the Wide Project, the "Workflow Process Model"[WID 97]. The workflow among the tasks were supplemented by a description of each activity, using a template which allows to define the pre and post-conditions, the agents, the super tasks and others data about the task, as proposed by Sizilio [SIZ 2001]. The templates give to the workflow more flexibility and add more information. The workflow of the Byzance system work is available at the address:

5. Conclusions

So far, the main results of the CEMT project can be summarized as follows: (i) the integration of BW to both Byzance and Amaya, (ii) the study of the Workflow within Byzance, (iii) the expertise in manipulating, developing and modifying most of the Amaya features, and (iv) the design and implementation of the servers which are already in use.

The BW integration with Amaya and Byzance systems allows to check the extensibility of both Amaya and the BW module. After this checking, it was concluded that it is viable to include:

- New resources such as a richer set of events, not only past events but also present and future ones.

- New and more complex functionalities of Amaya, such the possibility to edit workflow diagrams through graphical interfaces based on SVG.

With the modeling and study of the Workflow in Byzance was obtained a reference model for the workflow of the new system, Amaya. The Byzance is the result of many studies cariied out by INRIA in cooperative work and this theory can be brought in the CEMT project. The group has also carried out a profound study to design an efficient and flexible model : "Workflow Process Model" plus Templates [SIZ 2001]. This model was chosen as the official representation of the workflow in the project.

Using the implemented servers (Web and Annotea servers), we published all those results on the Internet, and we also used the resources of Web publishing and annotations to improve the cooperation inside the project group.

As a conclusion, based on the already gained experiences and achievements the CEMT framework, configurations and tests of almost all technologies and systems, the CEMT project already has a complete development environment to reach its goals. A group of developers and analysts trained in this environment are currently carrying out the remaining developments, and a complete and well defined planning of the future work has been agreed upon. This indicates that the project has good chances in reaching its initial goals.

6. References

[ARA 97] Araújo, R.M.; Dias, M.S.; Borges, M.R.S. Suporte por Computador ao Desenvolvimento Cooperativo de Software: Classificação e Propostas. In: XI Simpósio Brasileiro de Engenharia de Software. Anais. Fortaleza, CE, outubro, 1997. P. 299-314

[DEC 98a] DECOUCHANT, Dominique; SALCEDO, Manuel Romero; QUINT, Vincent. Structured Cooperative Authoring on the World-Wide Web. Available at: <>. (set.1998).

[DEC 98b] DECOUCHANT, Dominique; SALCEDO, Manuel Romero; SERRANO, M. Principes de Conception d'une Application d'Édition Coopérative de Documents sur Internet. . Available at: <> (set.1998).

[DIA 97] DIAS, Márcio; XEXÉO, Geraldo. Editor Cooperativo para Diagramação de Software OO. In: SIMPÓSIO BRASILEIRO DE ENGENHARIA DE SOFTWARE SBES, 11., 1997. Fortaleza, CE. Anais... Fortaleza: SBC, 1997. p. 499-502

[GUT 99] Gutwin, C.; Greenberg, S. A framework of awareness for small groups in shared-workspace groupware. Technical Report 99-1, Department of Computer Science, University of Saskatchewan, Canadá. Available at: <> (set., 1999).

[KAH 01] KAHAN, José; Koivunen, Marja-Riitta; Prud'Hommeaux, Eric; R. Swick, Ralph. Annotea: An Open RDF Infrastructure for Shared Web Annotations.

[KOI 01] KOIVUNEN, Marja. Annotea Project . Available at: <> (jun., 2001)

[OPE 99] Opera Group. Byzance. Available at: <> (jun.1999).

[PIN 01a] PINHEIRO, Manuele Kirsch. Mecanismo de suporte à percepção em ambientes cooperativos. Dissertação de mestrado. Porto Alegre, Brasil : PPGC/UFRGS, 2001. Available at: <> (jun.,2001).

[PIN 01b] PINHEIRO, Manuele Kirsch; Lima, José Valdeni; Borges, Marcos R.S. Awareness em Sistemas de Groupware. In IV Jornadas Iberoamericanas de Ingenieria de Requisitos y Ambientes de Software. Memorias. Santo Domingo. Costa Rica : CIT, 2001.

[PIN 01c] PINHEIRO, Manuele Kirsch; Lima, José Valdeni. An Adaptable Framework for Past Events Awareness Support. Available at: <> (jun.,2001).

[QUI 99a] QUINT, Vincent;VATTON, Irène. Thot Tool Kit API. Available at: <> (fev.1999).

[SAL 98] SALCEDO, Manuel Romero. Alliance sur l'Internet: support pour l'édition coopérative des documents struturés sur um réseau à grande distance. Grenoble, France: INPG, 1998.

[SIZ 2001] SIZILIO, G.R.M.A.; EDELWEISS, N. Modelo de Autoria de Cursos de Ensino à Distância. Revista Brasileira de Informática na Educação, nr. 8, abril/2001. ISSN 1414-5685.

[SOH 99] Sohlenkamp, M. Supporting group awareness in multi-user enviroment through perceptualization. Dissertation. Fachbereich Mathematik-Informatik der Universität - Gesamthochschule - Paderborn. Fevereiro, 1998. Available at: < poliawac/ms-diss/> (jul., 1999)

[TIS 00] TISSOT, Michaël. Envirionnements d'édition de workflow. Grenoble, France : INRIA, 2000. Rapport de Stage. Available at: < rapport.v2.pdf>. Access : sep. 2001.

[WEBDAV 99] WEBDAV: WWW Distributed Authoring and Versioning. In: INTERNET ENGINEERING TASK FORCE MEETING, 44., 1999, Minneapolis, Minesota. Proceedings... Available in: <>. Access : jan. 2000.

[WEBDAV 01] Welcome to WebDAV Resources. Available in: <> Access : ago. 2001.

[WIDE 97] The WIDE Workflow Modeling Language. Available at:

[WWW10 01] INTERNATIONAL CONFERENCE, 10., 2001, Hong Kong. Proceedings... May 2001. Available in: <> Access : August, 2001.

[W3C-Amaya 01] Vatton, Irène. Welcome to Amaya. Available in <>. Access : August, 2001.

[W3C-SMIL 01] Michel, Thierry. Synchronized Multimedia. Available in <>. Access : August, 2001.

[XHTMLplusSMIL 01] Newman, Debbie; Schmitz, Patrick; Patterson, Aaron. XHTML+SMIL Profile. W3C Working Draft 07 August 2001. Available in <>. Access : August, 2001.

I Workshop CEMT – Program

Date: 05/09/2001

Session 01 - 10.30 - 12.00

Lionel: Presentation of the Tools

Nabil : Set of Activities of the Design Operates

Nabil : Presentation of the Language Smil 2.0

Session 02 - 14.00 - 18.00

Valter: ALM: Adaptative Layering Multicast

Zeve: Features of CEMT Workflow Model

Tiago: Workflow do Editor Cooperativo Byzance / Implementações no Editor Amaya

Eidy: Data Extraction in Semistructured Document.