Keywords: presentation technology, persuasion, mixed media, dialogs, virtual worlds, rich media.
{eliens,claire,huang,ctv}@cs.vu.nl } \date{version 0.1} \maketitle -->
Some phenomena in the media are older than is generally recognized.
As observed in
In other words, mixed media have their origin in the medieval era,
where both visual material and textual material were part of the rethorics of
Recently we included dialogs
using speech balloons (and possibly avatars) to display the text
commenting on a particular presentation.
The characters and dialog text may be used to enliven the material.
In this way, the students' engagement with the material may be increased,
Clearly, there is a tension between engagement and immersion.
Immersion, understood as the absorption within a familiar
narrative scheme (in our case the lecturer's presentation),
may be disrupted by the presence of (possibly annoying)
comments, whereas the same comments may lead the attention
back to the material, or
provide a foothold for affective reactions to the material,
In one of her talks, Kristina Höök observed that some users get really fed up with the comments delivered by Agneta and Frieda. Nevertheless, it also appeared that annoyance and irritation increased the emotional involvement with the task.
For our presentations, we may ask how mixed media may help in increasing the emotional involvement of the audience or, phrased differently, how dialogs may lead to emotional enhancement of the material,
As a consequence, the tension between immersion and engagement may be partially resolved, since the characters delivering the dialog may be placed in
their 'natural' context, that is a virtual environment
as in example (a).
dialog
style parameters
DLP+X3D
We have described a framework for mixed media that allows for the superposition of (text) dialogs delivered by humanoid avatars and/or speech balloons, on arbitrary rich media objects and virtual environments.
We have looked at the design space of mixed media presentations, by discussing a number of representative examples, each illustrating a particular level of complexity. Also authoring issues were discussed, and an indication was given of the style parameters needed to develop effective presentations.
We have further described the implementation platform used to realize the mixed media presentations and explored what new applications and extensions are feasible.
observerStructure
Background and motivation
example -- promotion video in virtual environment
\hspace{-0.2cm}
\hspace{-0.7cm}
(a) dialog in context (b) dialog on video
General perspectives -- narrative structure and persuasion
Mixed media, that is the combination of dialogs
with rich media and virtual environments,
may endanger the narrative structure of a presentation.
Each of these media entities may have a distinct narrative structure.
Slides are sequentially organized, and each slide may have levels
that are displayed in a sequential fashion.
The narrative structure of digital video may be
arbitrarily complex, and may make full use
of cinematographic rethorics.
Navigation and interaction in virtual environments
may be seen as a weak narrative structure, which may however
be strengthened by guided tours or viewpoint transformations,
taking the user to a variety of viewpoints
in a controlled manner.
Finally, dialogs have a well-defined temporal structure,
with alternating turns for the two (virtual) speakers.
consonant or dissonant comments
engagement versus immersion
emotional enhancement
Design space -- the juxtaposition of mixed media
In this section, we will explore in a more
systematic way what options we have in creating
mixed media presentations.
We have a division according to levels of complexity,
with at level 0 the basic material, that is dialog
text, rich media objects such as digital video
and virtual environments.
The other levels arise by adding dialogs to either
the media object or the virtual environment.
We also allow for the virtual speakers of the dialog
to change attributes of the presentation, for example
by depositing objects in the (virtual) environment,
The ordering of these levels is not unique, but admittedly depends
on our intuition of complexity.
\hspace{-0.2cm}
\hspace{-0.7cm}
(c) context in dialog (d) dialog in context
level 0: basic material
level 1 -- combined:
When combining the media object with the dialog,
we may simply superpose the dialog on the movie,
in a similar way as shown in figure (b), or we may
project the video onto the surface of the speech balloon,
as illustrated in figure (c), which has a rather surprising effect.
level 2 -- with avatars:
It makes quite a difference whether humanoid characters are used
to deliver the dialog or only the plain text balloons.
The use of avatars seems to enhance the recognition of
a particular role, that is the kind of comments the character makes.
Once the role is established, the avatars may dissappear
and the (color and position) of the speech ballons suffice.
Another difference is that the ballons should be positioned
differently without the presence of an avatar.
level 3 -- with attributes
Apart from speaking their dialog text,
the avatars may undertake autonomous actions.
They might get bored and display ambient behavior, like looking on
their watch.
In addition, before or after speaking their phrase, they
may change their position and modify the environment,
for example by depositing (3D) objects to illustrate
their comments.
These actions may be arbitrarily complex, and for example
result in going to the next (level of the) slide.
level 4 -- with context:
The ultimate context of a slide may be a virtual environment,
as in our examples (a) and (d) a virtual environment of our university campus developed by one of our students.
Having such a context, the difference between
a presentation and an information space becomes blurry
since the user may interact with the environment and, for example,
start a guided tour.
In such cases the developer must decide whether
the dialog takes place in a fixed
position of the virtual environment or is fixed relative
to the viewpoint of the user/audience.
Authoring issues -- style parameters
In the next two sections, we will look at the implementation
of the dialogs, respectively from an authoring perspective
and a system perspective.
The phrases are (textually) included in a slide,
which is itself indicated by appropriate begin and end tags.
The alternation between speakers is indicated by the
attributes left and right.
Although detailed indications of (among others)
when a phrase should be uttered are possible, these
advanced options are hardly ever used, except for
defining complex actions.
Apart from phrases, we also allow for gestures, taken from
the built-in repertoire of the avatars.
In Implementation -- the DLP+X3D platform
In our group we have developed a platform
for intelligent multimedia,
that is a platform
for virtual environments based on agent technology,
supporting embodied conversational agents,
The control points are actually nodes in the VRML scenegraph
that act as handles which may be used to manipulate the scenegraph.
Research directions -- conversational agents
In the examples discussed, the agent avatars entered
a dialog with one another to comment on a particular scene or slide,
to augment a presentation.
As a next step, we would like to extend our approach
to allow for interaction in a virtual
environment, that is to augment information spaces.
virtual musea
cultural heritage
Conclusions
references
Appendix: The slides format
The first slide contains some text and a 3D object.
The second slide contains only a 3D object.
Inbetween the slides there may be arbitrary text.
The slides are converted to VRML using XSLT,
the XML transformation language.
VRML PROTOs
The slides contained in a document constitute a slide set.
A slide set is a collection of slides that may contain lines of text
and possibly 3D objects.
For displaying 3D objects in a slide
we need no specific PROTO.
annotated slides