topical media & game development
[] readme course(s) preface I 1 2 II 3 4 III 5 6 7 IV 8 9 10 V 11 12 afterthought(s) appendix reference(s) example(s) resource(s) _

talk show tell print

multimedia authoring

It is tempting to identify a presentation with the information space it presents. This is what users often do, and perhaps should do. When that happens, the presentation is effective. But you must remember that the actual presentation is just one of the many possible ways to engage a user in exploring an information space. Making the choice of what to present to the user is what we understand by (multimedia) authoring.

Authoring is what we will discuss in this section. Not by giving detailed guidelines on how to produce a presentation (although you may look at the online assignment for some hints in this respect), but rather by collecting wisdom from a variety of sources.

...

1

visualization

 Let's start with our explorations by looking at the problem of visualisation with a quote from David Gelernter, taken from  [User]:

visualization


Grasping the whole is a gigantic theme, intellectual history's most important.

Ant vision is humanity's usual fate; but seeing the whole is every thinking person's aspiration.

David Gelernter, Mirror Worlds 1992


Now, consider, there are many ways in which the underlying information space may be structured, or speaking as a computer scientist, what data types may be used to represent the (abstract) information.

data types


The visualisation problem then is to find a suitable way to present these structures to the user. Basicall, following  [User], there are two paradigms to present this information:

Storytelling may be very compelling, and does not force the user to interact. On the other hand, storytelling may lead to information consumerism alike to television enslavement.

An interaction paradigm that combines 'storytelling' with opportunities for interaction, as for example in the blendo approach discussed in section 3.2, would seem to be most favorable. Interaction then may result in either changing the direction of the story, or in the display of additional information or even transactions with a third party (for example to buy some goodies).

persuasive technology

Whatever your target audience, whatever your medium, whatever your message, you have to be convincing if not compelling.

In the tradition of rethorics, which is the ancient craft of convincing others, a new line of research has arisen under the name of persuasive technology. In the words of my collegue, Claire Dormann, persuasion is:

persuasion

  • a communication process in which the communicator seeks to elicit a desired response from his receiver
  • a conscious attempt by one individual to change the attitudes, beliefs or behaviours of another individual or group individual through the transmission of some messages.
In other words, the purpose of persuasion is to accomplish one of the following goals: to induce the audience to take some action, to educate the audience (persuade them to accept to accept information or data), or to provide the audience with an experience. In the area of multimedia, one may think of many applications. Quoting Claire Dormann, in interactive media, the field of application of persuasive technology ranges from E-commerce, social marketing (like an anti-AIDS campaign) to museum exhibits. Also E-commerce provides an obvious example. To convince people to buy more, more persuasive messages and technologies are developed through the use of humorous and emotional communication, agents (such as price finders) or 3D representations of products and shops. For health campaigns (or any campaign of your choice) one can imagine 3D information spaces with agents presenting different point of views and where users are given different roles to play. In a museum you might want to highlight key points through innovative and fun interactive exhibits. Although the subject of persuasive technology is far less technology-oriented than the name suggests, multimedia (in a broad sense) form an excellent platform to explore persuasion. As concerns multimedia authoring, set yourself a goal, do the assignment, explore your capabilities, convey that message, and make the best of it.

(re)mediation

 What can you hope to achieve when working with the new media? Think about it. Are the new media really new? Does anyone want to produce something that nobody has ever seen or heard before? Probably not. But it takes some philosophy to get that sufficiently clear.

In  [Remediation], the new media are analyzed from the perspective of remediation, that is the mutual influence of media on eachother in a historical perspective. In any medium, according to  [Remediation], there are two forces at work:

(re)mediation

  • immediacy -- a tendency towards transparent immersion, and
  • hypermediacy -- the presence of referential context
Put in other words, immediacy occurs when the medium itself is forgotten, so to speak, as is (ideally) the case in realistic painting, dramatic movies, and (perhaps in its most extreme form) in virtual reality. Hypermediacy may be observed when either the medium itself becomes the subject of our attention as in some genres of modern painting, experimental literature and film making, or when there is an explicit reference to other related sources of information or areas of experience, as in conceptual art, many web sites, and also in CNN news, where apart from live reports of ongoing action, running banners with a variety of information keep the viewers up to date of other news facts.

Now, the notion of remediation comes into play when we observe that every medium draws on the history of other media, or even its own history, to achieve a proper level of immediacy, or 'natural immersion'. For example, Hollywoord movies are only realistic to the extent that we understand the dramatic intent of cuts, close-ups and storylines, as they have been developed by the industry during the development of the medium. As another example, the realism of virtual reality can only be understood when we appreciate linear perspective (which arose out of realistic Renaissance painting) and dynamic scenes from a first person perspective (for which we have been prepared by action movies and TV).

Even if you may argue about the examples, let it be clear that each (new) medium refers, at least implicitly, to another medium, or to itself in a previous historic phase. So, what does this mean for new media, like TV or virtual reality?

Let's start with virtual reality.  [Remediation] comment on a statement of Arthur C. Clarke

Virtual Reality won't merely replace TV. It will eat it alive.

by saying that ... he is right in the sense that virtual reality remediates television (and film) by the strategy of incorporation. This strategy does not mean that virtual reality can obliterate the earlier visual point-of-view technologies, rather it ensures that these technologies remain as least as reference points by which the immediacy of virtual reality is measured. So, they observe "paradoxically, then, remediation is as important for the logic of transparency as it is for hypermediacy". Following  [Remediation], we can characterize the notions of immediacy and hypermediacy somewhat more precisely.

immediacy

  • epistemological: transparency, the absence of mediation
  • psychological: the medium has disappeared, presence, immersion

hypermediacy

  • epistemological: opacity, presence of the medium and mediation
  • psychological: experience of the medium is an experience of the real
Now, sharpen your philosophical teeth at the following statement, taken from  [Remediation], p. 224:

Convergence is the mutual remediation of at least three important technologies -- telephone, televison and computer -- each of which is a hybrid of technical, social and economic practice, and each of which offers its own path to immediacy.

The telephone offers the immediacy of voice or the interchange of voices in real-time.

Television is a point-of-view technology that promises immediacy through its insistent real-time monitoring of the world.

The computer's promise of immediacy comes through the combination of three-dimensional graphics, automatic (programmed) action, and an interactivity that television can not match.

As they come together, each of these is trying to absorb the others and promote its own version of immediacy.

Let us once more come back to virtual reality and its possible relevance in our information age,  [Remediation], p. 225:: in the claim that new media should not be merely archival but immersive, the rhetoric of virtual reality finally enters in, with its promise of the immediacy of experience through transparency. . So, with respect to the new media, we may indeed conclude: what is in fact new is the particular way in which each innovation rearranges and reconstitutes the meaning of earlier elements. What is new about media is therefore also old and familiar: that they promise the new by remediating what has gone before. The true novelty would be a new medium that did not refer to the other media at all. For our culture, such mediation without remediation seems to be impossible.

...

2

example(s) -- mobius

 Rurger van Dijk, a former student of mine, has implemented an interactive story in flash. The story is a romance, told with images displaying scenes from the life of the players, a young man and a young women. The user can choose perspectives, either the man's or woman's, and watch the story from that point of view. The story is both non-linear and circular. The scenes can be connected in various way, and order is not compulsory.

research directions -- narrative structure

 Where do we go from here? What is the multimedia computer, if not a new medium? To close this section on multimedia authoring, let us reconsider in what way the networked multimedia computer differs from other media, by taking up the theme of convergence again. The networked multimedia computer seems to remediate all other media. Or, in the words of  [Hamlet]:

convergence

(p. 27) ... merging previously disparate technologies of communication and representation into a single medium.

The networked computer acts like a telephone in offering one-to-one real-time communication, like a television in broadcasting moving pictures, like an auditorium in bringing groups together for lectures and discussion, like a library in offering vast amounts of textual information for reference, like a museum in its ordered presentation of visual information, like a billboard, a radio, a gameboard and even like a manuscript in its revival of scrolling text.

In  [Hamlet], an analysis is given of a great variety of computer entertainment applications, varying from shoot-em-up games to collaborative interactive role playing.  [Hamlet] identifies four essential properties that make these applications stand out against the entertainment offered by other media, which include books and TV. Two key properties determine the interactive nature of computer entertainment applications:

interactive


  • procedural -- 'programmed media' ...
  • participatory -- offering agency
All applications examined in  [Hamlet] may be regarded as 'programmed media', for which interactivity is determined by 'procedural rules'. With agency is meant that the user can make active choices and thus influence the course of affairs, or at least determine the sequence in which the material is experienced.

...

3

Another common characteristic of the applications examined is what  [Hamlet] calls immersiveness. Immersiveness is determined by two other key properties:

immersive


  • spatial -- explorable in (state) space
  • encyclopedic -- with (partial) information closure
All applications are based on some spatial metaphor. Actually, many games operate in 'levels' that can be accessedonly after demonstrating a certain degree of mastery. Networked computer applications allow for incorporating an almost unlimited amount of information. Some of the information might be open-ended, with storylines that remain unfinished. Closure, then, is achieved simply by exhaustive exploration or diminishing attention.

multimedia authoring

Coming back to the question what the 'new medium', that is the networked multimedia computer, has to offer from the perspective of multimedia authoring, two aspects come to the foreground:

multimedia authoring


  • narrative format
  • procedural authorship
The narrative format is incredibly rich, offering all possibilities of the multimedia computer, including 3D graphics, real-time sound, text. In short, everything up to virtual reality. But perhaps the most distinguishing feature of the new medium is that true authorship requires both artistic capabilities as well as an awareness of the computational power of the medium. That is to say, authorship also means to formulate generic computational rules for telling a story while allowing for interactive interventions by the user. Or, as phrased in  [Hamlet], the new cyberbard must create prototypical stories and formulaic characters that, in some way, lead their own life and tell their stories following their innate (read: programmed) rules. In section 8.3 and appendix C, we will present a framework that may be used as a testbed for developing programmed narrative structures with embodied agents as the main characters.
[] readme course(s) preface I 1 2 II 3 4 III 5 6 7 IV 8 9 10 V 11 12 afterthought(s) appendix reference(s) example(s) resource(s) _

(C) Æliens 04/09/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.