topical media & game development
PDF
PANORAMA -- explorations in the aesthetics of social awareness
A. Eliëns &
D. Vyas
VU University Amsterdam
resource(s)
abstract
In this paper we reflect on our experiences in developing PANORAMA,
a playful application meant to promote and support social awareness
in a work environment, through art-inspired visualisations
of social processes and personal contributions.
With respect to the design of PANORAMA, we found common notions of
visual semiotics helpful in determining the overall composition of the
screen layout. More in general, however, the development of PANORAMA
proved to be an exercise in interaction aesthetics, which as we
will argue in this paper may greatly benefit from common notions in
interactive video game play.
In this paper we will only briefly discuss technical and deployment issues,
since our main contribution here is to establish the relation between
the aesthetics of interaction and game play.
Keywords and phrases: social awareness, visual semotics, interaction aesthetics, video games
INTRODUCTION
Since the 1970's, Dutch universities have enormously grown in size,
due to the ever larger number of students that aim at having university
level education.
As departments become bigger, however, staff members no longer know eachother
personally. The impersonal and anonymous atmosphere is increasingly an
issue of concern for the management, and various initiatives have been taken,
including collective trips into nature, as well as cultural events, not to much
avail for that matter.
An additional problem is that more and more members of the staff
come from different countries and cultures, often only with a temporal contract and residence permit.
Yet, during their stay, they also have the desire to communicate and learn about
the other staff members and their culture(s).
In september 2006, the idea came up to use a large screen display in one of the public
spaces in our department, to present, one way or another, the 'liveliness' of the work place,
and to look for ways that staff members might communicate directly or indirectly with eachother
through this display.
Observing that communications often took place during casual encounters at the coffee machine or printer,
we decided that monitoring the interactions at such places might give a clue about the liveliness
of the work place.
In addition, we noted that the door and one of the walls in the room where the coffee machine
stood, was used by staff members to display personal items, such as birth announcement cards
or sport trophees.
In that same room, mostly during lunch time, staff members also gathered to play cards.
Taking these observations together, we decided to develop a system, nicknamed PANORAMA,
to present these ongoing activities and interactions on a large display, which was to
be positioned in the coffee room.
The name of our system is derived from the famous
Mesdag Panorama
in The Hague, which gives a view on (even in that time nostalgic rendering of) Scheveningen.
However, it was explicitly not our intention to give an in any sense realistic/naturalistic
remdering of the work place, but rather, inspired by artistic interpretations of
panoramic applications as presented in [Grau (2003)], to
find a more art-ful way of visualizing the social structure and dynamics of the work place.
At this stage, about one year later, we have a running prototype (implemented in DirectX),
for which we did perform a preliminary field study as well as a first user evaluation, [Vyas et al. (2007)],
and also we have experimented with a light-weight web-based variant, allowing
access from the desktop.
In this paper, our primary focus, however, will be to establish
the relation between interaction aesthetics and game play, as inspired by our experiences
in developing PANORAMA.
structure
The structure of this paper is as follows.
First we will give a brief sketch of the PANORAMA system, that is the ideas underlying it
and the realization of a first prototype.
Then we will present, in a more general fashion, possible guidelines for the design of PANORAMA,
followed by a discussion of common issues in interaction aesthetics and game play.
We will then introduce the notion of dialectics of awareness,
and in conclusion identify the primary dimensions of aesthetic experience.
|
|
|
| Fig 1. (a) context | (b) self-reflection |
BEING SOCIAL @ WORK
The PANORAMA system is meant to support social awareness,
in non-work related ways, using a large screen display
in a public room in our faculty.
To achieve social awareness, we ask the staff to
contribute items of self-reflection,
such as holiday postcards or birth announcements.
In order to reflect the liveliness of the workplace,
we monitor places where occasional encounters
may take place,
for example during a break at the coffee machine
or in the printer room, waiting for the printer queue.
Encounters in such places are often of an informal,
personal nature, but may be mixed with work-related
interests.
As an experimental feature, we consider to allow for
direct interaction using the system, for example,
to play a game, possibly with a mobile phone as an
input device.
In summary, the PANORAMA system is determined by the following
contributions of its users,
contributions that are not necessarily direct
or even do require explicit activity.
PANORAMA
- self-reflection(s) -- e.g. picture/postcard(s)
- casual encounter(s) -- at coffee machine or printer
- occasional battle(s) -- optional direct interaction
For a deeper understanding of what
role the system would play in the
(working) life of the staff,
we engaged in several field studies
and used cultural probes
to determine what could be valuable contributions
to ask for and how to display these on
the PANORAMA screen, [Vyas et al. (2007b)].
We have developed a first prototype implementation in DirectX 9,
using ViP technology, based on the system described in [Eliens (2006)].
In this realization, we deploy a moving virtual gallery,
containing video and image feeds.
The gallery acts like a moving scroll,
displaying information in a continuous manner, in a panorama-like way.
The images in the gallery are fed by channels, containing
information that is either due to explicit
contributions (self-reflections) or ongoing activity
in the work place (casual encounters or occuring events), monitored
by cameras or other sensors.
Obviously, as we will discuss later, the PANORAMA system is subject to
a dialectic of awareness, that is it will be present, but the staff will
only occasionally pay atention to it, dependent
on their interests and also on what visual cues and effects
the system presents to draw attention to ongoing activity.
Although we would like the system to be autonomous
in the decision how to present information, we cannot hope
to do this by computational means only, [Eliens (1988)], and
hence we need to provide
interaction markers to
invite the users to contribute actively to the system,
or influence the way information is displayed
according to their preference.
For the display of information, we provide a rich context of material, including
videos showing the faculty and its surroundings, fragments from video clips,
and of course the material resulting from the occassional encounters and self reflections.
In PANORAMA we use particle systems displaying the information in a pictorial way by
images flowing according to the rules of the particle system chosen to represent
that particular type of information.
To organize this material we took conventions governing
our interpretation of 2D displays as a guideline in designing the flow of particle systems.
A more detailed discussion of these conventions will be given in the next section.
Identifying bottom with plain, top with ideal, left with
given and right with new, we arrived at the following identifications.
semiotic rules
- self reflections: plain ideal/new
- casual encounters: plain ideal/given
- contextual stories: ideal/given plain/new
- personell faces: ideal/new plain/given
- occurring events: ideal plain
For example one may remark that people's faces become more familar in time,
and that in the process of getting to know them we see more of the plain reality of people.
Naturally, different interpretations and different designs are possible.
Apart from the spatial characteristics of these flows of information we
also used the speed with which the images move accross the screen as a parameter of design.
For example events and occurrences move very fast, while both casual encounters and self reflections
move slowly. Faces come across the screen with intermediate speed.
To give self reflections more visual salience, the images are displayed in a non-transparent way,
whereas all other flows of images merge with the background due to transparency.
Although it is debatable whether the interpretations given above hold,
we found the heuristics given by semiotic theory extremely helpful in deciding
how to represent the information as flows of images in space/time.
MEANING OF COMPOSITION
Aesthetic awareness is common to us all.
Having an understanding of aesthetic awareness,
can we isolate the relevant design parameters and formulate
rules of composition that may help us in developing
interactive applications?
According to our philosophical credo, [Eliens (1979)], no!
However, the history of art clearly shows the impact
of discoveries, such as the discovery of perspective,
as well as conventions in the interpretation of
art, as for example in the iconic representation
of narrative context in 17th century Dutch painting.
Moreover, the analysis of the visual culture of mass media may also
give us better understanding of the implied meaning
of compositional structures.
The notion of perspective, described in [Alberti (1435)],
is an interesting notion in itself,
since it describes both the organisation of the image
as well as the optimal point of view of the viewer.
The normal perspective as we know it is the central
perspective.
However, there are variants of perspective that force the
viewer in an abnormal point of view,
as for example with anamorphisms.
Perspective had an enormous impact on (western) art
and visual culture.
It defines our notion of naturalist realism, and allowed
for the development of the panorama as
a mass medium of the 19th century, [Grau (2003)].
Art that deviated from central perspective, such
as cubism or art from other cultures, was often
considered naive.
Photography and its pre-cursors had a great impact
on the perfection of perspectivist naturalism,
and what is called photorealism
became the touchstone of perfection for early
computer graphics, [Bolter and Grusin (2000)].
Apart from perspective, other conventions regulate
the composition of the 2D image,
in particular, following [Kress and van Leeuwen (1996)],
the information value related to where
an object is placed in the image,
and the salience of the object,
determined by its relative size, being foreground or background,
and visual contrast.
Also framing is used to emphasize meaning,
as for example in the close-up in a movie shot.
In analysing a large collection of image material,
[Kress and van Leeuwen (1996)], somewhat surprisingly found
that lef/right positioning usually meant
given versus new,
top/bottom positioning ideal versus real,
and centre/margin positioning important
versus marginal.
It is doubtful whether these meaning relationships
hold in all cultures, but as a visual convention
it is apparently well-rooted in western visual culture.
For 2D images, [Kress and van Leeuwen (1996)] further identify
narrative elements, that is relations between
objects in the image that suggest a story,
such as a diagonal line from a person to a door,
or a relation of an object to the viewer,
such as a gaze towards the viewer,
a technique that has been used only since late
renaissance painting.
More than paintings or 2D images, film is the medium
for conveying narrative structures.
The art of storytelling in film has been perfected
in such a way that Hollywood films may seem more real
than life.
However, as emphasized in [Bolter and Grusin (2000)], this is not due
to any inherent form of naturalism, but to
the fact that we have got accustomed to the conventions
applied, that is the techniques of cutting, montage, camera
movements, close-ups, etcetera.
In a highly recommended book, [Arnheim (1957)], Rudolf Arnheim
gives an extensive analysis of the
principles of montage and film technique, and he explains why film is such
an effective medium:
frames of reference
It is one of the most important formal qualities of film
that every object that is reproduced appears simultaneously in two
entirely different frames of reference, namely the two-dimensional
and the three-dimensional, and that as one identical object it fulfills
two different functions in the two contexts.
Due to the subtle play between these two frames of reference
film may be considered an art form,
and as such perhaps the dominant art form of the 20th century.
As a mass medium, film may be characterized by
what Arnheim, following Benjamin, called the aesthetics of shock,
replacing reflective distance with immersive thrill.
As an art form, however, it is the dominant paradigm
for aesthetic awareness, lacking however still one dimension,
interactive dynamics.
As observed in [Bolter and Grusin (2000)], interaction is what
distinguishes video games from film.
Current day technology allows for high-resolution
photorealist graphics, that make video games or
virtual applications almost indistinguishable from film.
Virtual reality technology as applied in video games
adds arbitrary choice of perspective,
as exemplified in first-person shooters or fly-overs,
as well as an arbitrary mix of
the imaginary and real, as in CG movies, in an interactive
fashion.
Now, should we take the aesthetics of interactive video games
as the standard for interactive applications?
Not necessarily, since the naturalism strived for
in most games may at best be characterized
as naive realism, mostly photorealism.
As observed in [Kress and van Leeuwen (1996)], realism is a social construct,
and hence the program for developing an aesthetics for
interactive applications should perhaps
include the development of appropriate realisms.
Again with an eye to the history of art,
where we have for example impressionism,
cubism, expressionism,
as a guideline in the
design of interactive systems, it might be even better
to look for appropriate interaction-isms,
styles of developing interactive systems and games
from a particular perspective. Not excluding provocative perspectives! Cf. [Burger (1981)].
AESTHETICS OF GAME PLAY
Where an arbitrary interactive system may differ from a game played
for entertainment is obviously the actual outcome, that
is the value attributed to using the system in
the real world, and probably
the effort required and the possible consequences.
You would not like to run the risk to die a virtual
death when answering your email, would you?
However, when interactive systems replace task-bound
functionality with fun, the difference becomes less clear.
As we indicate in [Eliens & Chang (2007)], one element not sufficiently captured by a
classic game model, as introduced in [Juul (2005)], is the narrative aspect of the game play.
To quote [Juul (2005)]:
rules vs fiction
Game fiction is ambiguous, optional and imagined by
the player in uncontrollable and unpredictable ways, but the emphasis
on fictional worlds may be the strongest innovation of the video game.
We may observe that many games already have a
strong relation to reality in what narrative context
they supply, or else in the realities of the media industry,
in particular Hollywood.
For serious interactive systems, we may assume an even stronger
and in some sense more straightforward relation
with reality, by the use of media content
that is relevant for the life of the individual.
All these aspects of playing games are clearly relevant for
the new interactive systems, which appeal more to play
than task-oriented behavior.
For example rules may be used to describe the visual characteristics of
a system (e.g. the display of images as a flow in a particle system),
outcome may be regarded as the benefits of the system (e.g. social awareness),
value may include the risks of the system (e.g. a transgression of privacy),
efforts is important when asking for contributions from the user (e.g. as image material
to be displayed in the system), attachment may result when the system is installed
(e.g. when people look forward to find new information), and finally consequences must be
considered when a system is installed and used (e.g. interaction between people may actually change
when they get to know eachother, for better or worse).
interaction markers
Where in game playing the variety of interaction modes
seems to be well understood within each community
of game players, for the development of more
general interactive systems we will have to think seriously
whether the target user will be able to learn the
various modes of interaction,
either by explicit instruction or during play.
And as designers we must be concerned with the rules of interaction as well as
issues of visualisation and
interaction mappings, that is in other words which
affordances the application offers for a particular
group of users.
DIALECTICS OF AWARENESS
In the course of our field study for the PANORAMA system,
[Vyas et al. (2007)],
we tried to establish what relation users would have
to the system, not only in the way they interact
with it, but also in terms of what role the system plays
in their lives, and when and how they would be aware of
the system.
Due to the intrinsic properties of the PANORAMA
system, as a system meant to support social awareness
in a work environment, we could not assume
direct focussed attention.
Instead, we must take
the various forms of awareness or attention into account.
Our thoughts in this direction were triggered
by a lecture of Linda Stone (former vice-president
of Microsoft) at the Crossmedia Week
September 2006 in Amsterdam,
entitled Attention -- the Real Aphrodisiac.
In that lecture Linda Stone made a distinction between
applications popular before 1985,
applications which were in general meant for
self-improvement, for example language-learning,
applications that were popular between 1985 and 2005,
applications that she characterized as supporting
continuous partial awareness,
such as email and news-feeds, and applications
of the period thereafter, from now into the future,
which may be characterized as applications
that allow the user to be creative, take part in a community,
and are in other words more focussed and less dependent
on the external environment.
Admittedly, it takes a few more steps to formulate
a theory of the dialectics of awareness.
However, with the function of the PANORAMA
system in mind, we may make, following [Benjamin (1936)],
some interesting distinctions between
the experience of art and architecture.
Where art is usually experienced in a delimited
time span, and is similarly delimited in space,
that is the position of the observer,
architecture is everywhere and always there.
As a consequence, art receives focussed attention
and may be appreciated with reflective distance,
whereas architecture is often not perceived
consciously, but merely present and subject to
an almost sub-conscious sensibility,
which is only brought to the focus of attention
when it is either aesthetisized, for example
when taking photographs, or when something
surprising is sensed, for example in the change
of skyline in New York.
As argued in [Hallnäss and Redström (2002)],
many of the new interactive systems,
whether in the category of ambient media,
ubiquitous computing or calm technology,
will fall somewhere inbetween the spectrum spanned
by art and architecture, or more likely even
alternate between the forms of awareness associated
with respectively art and architecture.
In designing the new interactive systems and games, we need to be explicitly concerned with
the actual phases of awareness that occur, simply because it
is not clear what role these systems play in our life.
When introducing a new system or artefact, we may distinguish
between the following phases:
phases of awareness
- initiation -- appeal to curiosity
- promotion -- raising interest
- progression -- prolonged involvement
As designers we must ask ourselves the following questions.
How do we appeal to the users' curiosity, so that our system is noticed?
How do we get a more sustained interest?
How de we get the user to interact with or contribute to the system?
And, how do we obtain prolonged involvement, and avoid boredom?
These questions are not simple to answer, and require also an understanding of the
actual context in which the system is deployed as well as an understanding
of the level of (aesthetic) literacy of the user(s).
AESTHETIC EXPERIENCE
In [Hallnäss and Redström (2002)] the notion of expressional is introduced,
to convey the expressive meaning of objects, that is the shift of attention
from use to presence,
subject to a dialectic process of appearance
and gradual disappearance,
when objects gradually become more familiar.
For the design of presence, aesthetics
is then considered as a logic of expressions,
in which expressions act as
the presentation of a structure in a given space
of design variables.
However appealing the notion of expressional,
following idealist aesthetics, [Kant (1781)],
where a distinction is made between aesthetic awareness
as a given, or a priori, sensibility and aesthetic judgement
as being of a more empirical nature, we would prefer
to consider aesthetics as a logic of sensibility,
which includes a dimension of self-reflection
in the sense of its being aware of its own history.
Put differently, to characterize the contextual aspect
of aesthetics, as it certainly applies to art,
we may speak of aesthetic literacy,
that is aesthetic awareness that is self-reflective by nature.
Assuming a notion of aesthetics as a logic of sensibility,
we may distinguish between three dimensions of form,
extending Kant's original proposal, as indicated below:
dimensions of aesthetic experience
- spatial -- topological relations, layout of image
- temporal -- order, rhythm, structure
- dynamic -- interaction, reflection, involvement
The dimension of dynamics clearly is the great
unknown, and more in particular it is the dimension we
have to explore in the context of interactive
systems, not in isolation but in relation to the other
dimensions,
not so much to establish definite criteria,
but to understand the forces at work, or in other words
the relevant parameters of design.
[Sartre (1936)] gives an existential foundation
for the dimension of dynamics, by observing
that the human body is instrumental in gaining awareness,
as the centre of both obscurity and reflection from
which consciousness emerges, through selection and action.
It is in the existential dimension of aesthetic awareness that we come most close to the
experience of the new digital artefacts, since it concerns both involvement and human action.
Interestingly, and in apparent contradiction with [Hallnäss and Redström (2002)],
cited previously, to establish
a foundation for the aesthetics of interactive systems [Graves-Petersen et al. (2004)]
seek refuge with pragmatist aesthetics
as it promotes aesthetics of use rather than aesthetics of appearance.
Again, although we agree with the gist of [Graves-Petersen et al. (2004)],
we wish to emphasize that the contribution of pragmatist
aesthetics is not its focus on use, but the role
of experience in understanding and appreciating
aesthetic artefacts and for that matter games, that is the active role
of the subject in becoming aware of its meaning.
Reflecting on the
epistemological value of game playing in [Climate], we observed following [Magic],
that the game player enters a magic circle akin to a complex social system, where
actors, rules, and resources are combined
in intricate (game) configurations:
game as social system
| actors | rule(s) | resource(s) |
|---|
| players | events | game space |
| roles | evaluation | situation |
| goals | facilitator(s) | context |
Leaving the interpretion of the elements of such a (game) system,
indicated in the table above, to the reader, we may wonder what meaning games have,
and looking at the fantasy items and visual effects of current day video games,
we may wonder not only what is the meaning of meaningful elements,
having a logical place in the narrative, but also what is the meaning or function of the
apparently meaningless elements.
The answer is simple, involvement and more in particular emotional involvement
due to the in-born playfulness of humans.
DEPLOYMENT
The original version of PANORAMA was developed in DirectX, and was meant to be displayed on a large screen,
with a static viewpoint on a dynamically changing scene, reflecting the activity in the workspace and
the self-reflective contributions in an artful way.
In [VUSL], we speculated about using Second Life for the realization of PANORAMA.
Moving from a central large screen in first life to a visualisation embedded in Second Life,
which is moreover subject to first-person viewpoint dynamics, is quite a challenge.
The most simple solution would be to project the large PANORAMA screen onto a display object
in Second Life using live video streams but this would leave that challenge unanswered.
Embedding PANORAMA in Second Life would allow us to observe, in more detail than in a previous
user study, the behavior of users, that is, to be more precise, the proximity to particular objects
of interest, the duration of their presence, and, using mechanisms of recommendation, their interest in
related items.
We also explored the use of AJAX and web services in an X3D/VRML implementation
of PANORAMA,
using the google GWT toolkit
to allow users to contribute their image material, and a PHP server for storage and retrieval of images, [Si & Eliens (2007)].
In addition, we consider to retarget the promotional game we developed for our faculty, using the
Half Life 2 Source SDK, into a (virtual) platform for social encounters and awareness, [VULife].
CONCLUSIONS
In an earlier study on interaction aesthetics, we came along a report of how the Belgium curator
Jan Hoet organized the Documenta IX, a famous yearly art event in Germany,
and we were struck by the phrase art and the public sharing accomodation, [Documenta],
which in a unique way expresses the intuition
we have with respect to the role the new interactive systems and games may play in our lives.
The PANORAMA system, as presented in this paper, may be regarded as one
of the new interactive systems, with game playing -- in the form of occasional battle(s) --
as an intrinsic element.
PANORAMA, and similar systems alike, presents us not only with a technical challenge,
but more importantly also with a design challenge, which requires a new way of
looking at the aesthetics of interaction, or perhaps we should say the meaning of such systems
in our day to day experience, amplifying our awareness.
As our initial prototype was received with much interest,
we see as important targets for future research, firstly the deployment of alternative platforms, [Si & Eliens (2007)],
and secondly the development of suitable games, that fit within
the aesthetic framework determined by the primary raison d'etre for PANORAMA,
to promote and support social awareness.
REFERENCES
- [Perspective] Alberti L.B. (1435),
- On painting and on sculpture,
Phaidon, edited by C. Grayson, 1972
- [Film] Arnheim R. (1957),
- Film as Art,
University of California Press
- [Reproduction] Benjamin W. (1936),
- The Work of Art in the Age of Mechanical Reproduction,
Online Archive -- http://www.marxists.org
- [Remediation] Bolter J.D and Grusin R. (2000),
- Remediation -- Understanding New Media,
MIT Press
- [Avantgarde] Burger P. (1981),
- Theorie der Avantgarde,
Edition Suhrkamp
- [Creativity] Eliens A. (1979),
- Creativity: reflection and involvement,
Ninth Int Conf of Aesthetics, Dubrovnic, August 1979
- [Computational] Eliens A. (1988),
- Computational Art,
First Int Symposium on Electronic Art, Leonardo Supplementary Issue, Pergamon Press 1988, pp. 21-26
- [Odyssee] Eliens A. (2006),
- Odyssee -- explorations in mixed reality theatre,
In Proc. GAME'ON-NA 2006, P. McDowell ed., Eurosis-ETI, pp. 62-64, Sept 19-21, Monterey, USA
- [Serious] Eliens A. and Chang T. (2007),
- Let's be serious -- ICT is not a (simple) game,
In Proc. FUBUTEC 2007, April 2007, Delft
- [VirtualArt] Grau O. (2003),
- Virtual Art -- From Illusion to Immersion,
MIT Press
- [Interaction] Graves-Petersen M., Iversen O.S., Gall Krogh P.,
- Ludvigsen M. (2004), Aesthetic Interaction -- A Pragmatist's Aesthetics of Interactive Systems,
Proc. DIS2004, Cambridge Mass. USA, pp. 269-276
- [Presence] Hallnäss L. and Redström J. (2002),
- From Use to Presence: On the Expressions and Aesthetics of Everyday Computational Things,
ACM Trans. on Computer-Human Interactions, Vol. 9, No 2, pp. 106-124
- [HalfReal] Juul J. (2005),
- Half Real -- Video Games between Real Rules and Fictional Worlds,
MIT Press
- [Kritik] Kant E. (1781),
- Kritik der reinen Vernunft,
Felix Meiner Verlag, ed. 1976
- [Semiotics] Kress G. and van Leeuwen T. (1996),
- Reading Images: The Grammar of Visual Design,
Routledge
- [Imagination] Sartre J.P. (1936),
- L'imagination,
Presse Universitaire de France, ed. 8, 1971
- [PanoramaWeb] Si Yin and Eliens A. (2007),
- PANORAMA -- A Rich VRML Application Platform For Online Gaming,
Workshop Web 3D Games, Web3D Symposium 07, Perugia Italy, April 15-18
- [Panorama] Vyas D., van de Watering M., Eliens A.,
- van der Veer G. (2007), Engineering Social Awareness in Work Environments,
Proc. HCI International 2007, 22-27 July, Beijing, China
- [Social] Vyas D. van de Watering M., Eliens A.,
- van der Veer G. (2007b), Being Social @ Work: Designing for Playfully Mediated Social Awareness in Work,
Proc. HOIT 2007, Chennai, India in August 2007
(C) Æliens
04/09/2009
You may not copy or print any of this material without explicit permission of the author or the publisher.
In case of other copyright issues, contact the author.
