Immersive Virtual Environments
Application in Museum Exhibition Design
Garth Paine - Activated Space
Paper presented to the Museum of Victoria - October 21, 1997
I have been invited today to give a presentation outlining some of my work in Immersive virtual environments.
Immersive virtual environments present us with a wonderful opportunity to develop exhibition approaches that, in keeping with the high level of public presentation expectations, provide a very successful mechanism to engage the museum visitor in the exhibition material, delivering that material in an entertaining way, facilitating both efficient and memorable information delivery through interaction with the exhibit itself.
There are two key points to consider;
The use of interactive technologies and therefore the experience of interaction for the visitor.
Immersion, the ability for this kind of technology to immerse the visitor completely in the material of the exhibition itself.
There is of course a fundamental point here which is the broader one with respect to interactivity.
I believe it is clearly indicated in the exhibition design principles of the last five or more years that interactive displays are a very successful way of involving the visitor in the material of the exhibit.
A parallel can perhaps be drawn with computer games which are set apart from film and television by three things:
◦ Engagement
◦ Control
◦ Experience.
Film and television require a passive audience.
Watching something has a different impact than acting upon something. With a computer game or interactive multi-media the big difference is the level of control.
Control puts the user or the player at the centre of the activity.
The distinction then is between passive viewing of an exhibition and active viewing of an exhibition.
A successful Immersive virtual environment will develop what is called direct or active engagement with the subject.
Back to The Top of This Document
This paper is divided into two sections.
Part One
Defines immersive virtual environments and then goes on to discuss their role in exhibition design. This discussion includes consideration of the following points.
The implications of encumbering the visitor with technology, or allowing them to be unencumbered.
The motivation for the use of this technology, whether it is for the engagement of the visitor in existing material, or it is being applied as a platform to replace other delivery methods.
I will also address the issue of individual or communal experience through interaction and explore the communication focus. That is, wether the design focus is from a human perspective or a perspective other than the human.
Part Two (not relevent for printed version) - demostration of tracking technologies
I will then finish with a summary.
An immersive virtual environment, is a space or room where normal sensory stimulus is augmented by the addition of sound, images and/or other sensations introduced through interaction.
Interaction may be facilitated through video sensing, tactile triggers, heat or light sensors and may use devices such as the Very Nervous System, the ADBIO, the I-Cube system and various others.
The visual content of an immersive virtual environment is created through the projection of non-linear video or multi-media presentation. It may use technologies such as Quick Time, AVI files, Director presentation files or Mtropolis files.
The vision is augmented and supported by multi-speaker sound systems.
The Sound system is used not only to deliver the audio into the environment but to control the spacialisation and perspective of that sound.
There are a number of systems available to do this and some of them are the Level Control System from LC Audio, and the interactive sound system from Wildlife Sanctuary, both of which are capable of delivering integrated show control of digital audio, graphics, text and video components and monitoring and altering performance characteristics for the level of sound the content based on the crowd densities and other programmable variables.
Activated Space has also developed applications using the VNS and Macintosh based playback systems that provide interactive visual and audio output.
Other systems that are substantially more limited in their interactive input include Dataton, Crestron and AMX.
There are a number of issues that need to be considered when designing an Immersive virtual environment.
Much of the virtual reality technologies that exist today require the user to don goggles, data-gloves, perhaps even body-suits and head mounted displays in order to interact with the system.
These systems are quiet clearly not practical for wide scale use in Museums and tend to be applied where normal sensory input is not wanting to be augmented but completely replaced by an artificial cyborg world in which the person may take on a virtual form that is quite different to their own.
The systems that I am describing today do not require the user to wear such technologies.
These systems are sometimes referred to by the hard-core virtual reality people as non-immersive systems and require no personal hardware.
In this situation normal sensory cues are not cut off but are supplemented by the addition of sounds and images.
The next point of consideration is to determine whether the immersive virtual environment is being used to engage the visitor in existing exhibition material or to replace existing delivery platforms.
An Immersive virtual environment could be used to generate for instance the underworld environment of a whale and a series of movement patterns related to the animal that could be tied to the movement patterns of people through the exhibition.
It could however on a visual and or possibly just audio basis be used to cause a sculptural figure or other element of the exhibition to talk directly to the visitor.
It might for instance at a certain distance call the visitor to come over -
"HEY COME OVER HERE, I'VE GOT A STORY TO TELL YOU".
When the visitor approaches it may again ask for the visitor to come closer -
"COME ON, A LITTLE CLOSER".
Within a certain distance, lets approximate that at say two metres it could then be made to say;
"OKAY, NOW LISTEN I'VE GOT A STORY TO TELL YOU. DO YOU KNOW WHAT HAPPENED HERE?...".
In this case the interactive element has been used to replace text based presentations. Audio based dialogue presents the information that would otherwise have been presented in pictorial or text based formats.
The next point of consideration is whether the interactive experience is to be an individual or communal one.
Interactive systems that sense a particular threshold, for instance the closest body to an exhibit, or the presence of a body within a certain area will present the visual and audio information into that area regardless of the behavioural patterns of the other people in that space.
The material can of course be focused in a variety of ways within the exhibition design so that the audio does not spread or interfere with other exhibits and the visual material may be contained.
However, if the interactive technologies require the user to wear a head-set display or data glove, then the interactive element is limited to the individual wearing the technology. With the head-set video display unit the presentation is experienced by the wearer only.
It is also important to consider whether interactive feed back is to be delivered to the participant, that is;
do you want the visitor to know that they are interacting with the installation ? or,
is the interactive technology being used to map visitor behaviour patterns to exhibition information delivery patterns ?
In this case it may not be necessary for the user to realise that their presence and movement patterns are actually causing interactive output.
If it is the intention of the designer to set up a direct interactive relationship between the individual and the space, then it is important to understand that this does not work particularly well on a communal basis. In a clearly interactive environment, people want to see a direct and immediate relationship between their input and the interactive system through the visual or audio output.
They wish to be able to manipulate and control the space quite separately from the in-put stimuli of others.
This is not of course a situation that is easily set-up or controlled within a museum environment, which is essentially a communal space.
Whilst systems such as the VNS do present head tracking and individual movement analysis tools, there is a problem in an exhibition environment to present the audio and video playback in such away that the individual is able to observe and isolate their impact upon the space.
I suggest, therefore that it is more beneficial in an exhibition environment to concentrate on programming these systems in such a way that represent communal behaviour patterns within the space.
That is, to concentrate on mass movements or mass variations in movement and to broadcast the interactive out put in a dispersed fashion.
The last point that I want to raise here is the necessity for the designer to consider whether the communication focus of such an environment is from a human perspective or a perspective other than human.
I mean by this, do we wish the interactive output to be representative of the perspective of human observation.
That is, are we seeing the landscape of the Himalayan peaks as if we were Sir Edmond Hilary on top of Everest, or does the visitor feel that they are strapped into a hang-glider flying some distance above the earth? These are of course both human perspectives, however within virtual environments it is possible to give the visitor a virtual body.
That is, we may wish to take them into the body of a whale swimming through a see of plankton. We may wish to take them through the perspective of being a spaceship landing on Mars. There could be any number of these applications but the perspective from which the interactive is actually communicating must be carefully defined as part of the design process in order for the technologies and the programming approaches to support that design perspective.
A further consideration for the designer is of course the contextualisation of the interactive immersive environment. In order for the visitor to find it a useful, entertaining and informative delivery method the interactive immersive environment must be contextualised.
It is important therefore firstly to look at the environment.
What qualities do you wish to represent within the visual and audio environment that has been created? What are the points of interest that you wish to draw the visitors attention to and how are you going to represent those?
Therefore, what are the communication objectives that are embodied in the brief that has brought about the development of such an environment?
It is also of course important to consider the characterisation of any people, animals or objects that may be used to deliver information within this environment.
In doing so I think it is important to consider the emotional context for that character?
What is their investment in the environment that our visitor is experiencing?
What is their position and why are they reporting it.
It is also important to consider the psychological journey of discovery that the visitor is making within this environment.
How will it change their perception of the phenomena that is being represented?
What is it that they will learn?
It is important to map out this journey as part of the concept design.
One thing that I haven't mentioned yet in this paper is that the trigger systems utilised by an immersive virtual environment can very easily be built into the exhibition.
It is possible in fact to use the iconography of the exhibition material itself to hold the triggers so that the visitors interaction with these icons causes interactive playback as desired.
As part of the summary I would like to touch upon something that I have not mentioned earlier and that is the context for interactive immersive virtual environments in the current technological and social environment.
I am currently doing research at RMIT into human movement as a controller within interactive immersive virtual environments and I have a paper on-line called Immersive Virtual Environments - A Social Perspective Link to the Essay.
This paper explores the context in which immersive virtual environment installation work has arisen in recent years and its parallel with developments in scientific thinking, the paradigm shift towards systems thinking, chaos theory, quantum theory, fractal mathematics.
Essentially an interactive immersive environment is a closed feed back loop between technology, the human condition, that is emotion, relationship behaviour, movement and the definition and experience of a physical space.
Each of these enacts upon the other and an immersive virtual environment cannot exist without all three.
Norbert Wiener a key proponent of cybernetics noted during his work on modelling social systems that
"it is certainly true that the social system is an organisation like the individual that is bound together by a system of communication and that it has a dynamic in which the circular process of the feedback nature plays an important role"
Fritjof Kapra comments
"the discovery of feedback as a pattern of life applicable to organisms and social systems helped social scientists observe many examples of circular causality implicit in social phenomena. The dynamics of these phenomena were made explicit in a coherent unifying pattern."
I believe that these quotes represent an important observation about the way in which immersive virtual environments reflect social patterns and community interaction in a way that is readily mapped into exhibition design.
Finally in summary I would to say that;
The interactive design process should be considered from the very out set,
immersive virtual environments can in fact drive other design considerations and this should be taken into consideration.
The contents of an immersive virtual environment are very powerful communicators themselves. That is,
moving images are a dynamic way of presenting information and maintaining attention, and
sound communicates in a more immediate and abstract way than other media.
My final point is;
budgeting properly for the production from the very outset is of paramount importance in order not to jeopardise full realisation of the design intention.
Application in Museum Exhibition Design
Garth Paine - Activated Space
Paper presented to the Museum of Victoria - October 21, 1997
INTRODUCTION
I have been invited today to give a presentation outlining some of my work in Immersive virtual environments.
Immersive virtual environments present us with a wonderful opportunity to develop exhibition approaches that, in keeping with the high level of public presentation expectations, provide a very successful mechanism to engage the museum visitor in the exhibition material, delivering that material in an entertaining way, facilitating both efficient and memorable information delivery through interaction with the exhibit itself.
There are two key points to consider;
The use of interactive technologies and therefore the experience of interaction for the visitor.
Immersion, the ability for this kind of technology to immerse the visitor completely in the material of the exhibition itself.
There is of course a fundamental point here which is the broader one with respect to interactivity.
I believe it is clearly indicated in the exhibition design principles of the last five or more years that interactive displays are a very successful way of involving the visitor in the material of the exhibit.
A parallel can perhaps be drawn with computer games which are set apart from film and television by three things:
◦ Engagement
◦ Control
◦ Experience.
Film and television require a passive audience.
Watching something has a different impact than acting upon something. With a computer game or interactive multi-media the big difference is the level of control.
Control puts the user or the player at the centre of the activity.
The distinction then is between passive viewing of an exhibition and active viewing of an exhibition.
A successful Immersive virtual environment will develop what is called direct or active engagement with the subject.
Back to The Top of This Document
OUTLINE
This paper is divided into two sections.
Part One
Defines immersive virtual environments and then goes on to discuss their role in exhibition design. This discussion includes consideration of the following points.
The implications of encumbering the visitor with technology, or allowing them to be unencumbered.
The motivation for the use of this technology, whether it is for the engagement of the visitor in existing material, or it is being applied as a platform to replace other delivery methods.
I will also address the issue of individual or communal experience through interaction and explore the communication focus. That is, wether the design focus is from a human perspective or a perspective other than the human.
Part Two (not relevent for printed version) - demostration of tracking technologies
I will then finish with a summary.
DEFINITION
An immersive virtual environment, is a space or room where normal sensory stimulus is augmented by the addition of sound, images and/or other sensations introduced through interaction.
Interaction may be facilitated through video sensing, tactile triggers, heat or light sensors and may use devices such as the Very Nervous System, the ADBIO, the I-Cube system and various others.
The visual content of an immersive virtual environment is created through the projection of non-linear video or multi-media presentation. It may use technologies such as Quick Time, AVI files, Director presentation files or Mtropolis files.
The vision is augmented and supported by multi-speaker sound systems.
The Sound system is used not only to deliver the audio into the environment but to control the spacialisation and perspective of that sound.
There are a number of systems available to do this and some of them are the Level Control System from LC Audio, and the interactive sound system from Wildlife Sanctuary, both of which are capable of delivering integrated show control of digital audio, graphics, text and video components and monitoring and altering performance characteristics for the level of sound the content based on the crowd densities and other programmable variables.
Activated Space has also developed applications using the VNS and Macintosh based playback systems that provide interactive visual and audio output.
Other systems that are substantially more limited in their interactive input include Dataton, Crestron and AMX.
POINTS OF CONSIDERATION
There are a number of issues that need to be considered when designing an Immersive virtual environment.
Much of the virtual reality technologies that exist today require the user to don goggles, data-gloves, perhaps even body-suits and head mounted displays in order to interact with the system.
These systems are quiet clearly not practical for wide scale use in Museums and tend to be applied where normal sensory input is not wanting to be augmented but completely replaced by an artificial cyborg world in which the person may take on a virtual form that is quite different to their own.
The systems that I am describing today do not require the user to wear such technologies.
These systems are sometimes referred to by the hard-core virtual reality people as non-immersive systems and require no personal hardware.
In this situation normal sensory cues are not cut off but are supplemented by the addition of sounds and images.
ENGAGE/REPLACE
The next point of consideration is to determine whether the immersive virtual environment is being used to engage the visitor in existing exhibition material or to replace existing delivery platforms.
An Immersive virtual environment could be used to generate for instance the underworld environment of a whale and a series of movement patterns related to the animal that could be tied to the movement patterns of people through the exhibition.
It could however on a visual and or possibly just audio basis be used to cause a sculptural figure or other element of the exhibition to talk directly to the visitor.
It might for instance at a certain distance call the visitor to come over -
"HEY COME OVER HERE, I'VE GOT A STORY TO TELL YOU".
When the visitor approaches it may again ask for the visitor to come closer -
"COME ON, A LITTLE CLOSER".
Within a certain distance, lets approximate that at say two metres it could then be made to say;
"OKAY, NOW LISTEN I'VE GOT A STORY TO TELL YOU. DO YOU KNOW WHAT HAPPENED HERE?...".
In this case the interactive element has been used to replace text based presentations. Audio based dialogue presents the information that would otherwise have been presented in pictorial or text based formats.
INDIVIDUAL/COMMUNAL
The next point of consideration is whether the interactive experience is to be an individual or communal one.
Interactive systems that sense a particular threshold, for instance the closest body to an exhibit, or the presence of a body within a certain area will present the visual and audio information into that area regardless of the behavioural patterns of the other people in that space.
The material can of course be focused in a variety of ways within the exhibition design so that the audio does not spread or interfere with other exhibits and the visual material may be contained.
However, if the interactive technologies require the user to wear a head-set display or data glove, then the interactive element is limited to the individual wearing the technology. With the head-set video display unit the presentation is experienced by the wearer only.
It is also important to consider whether interactive feed back is to be delivered to the participant, that is;
do you want the visitor to know that they are interacting with the installation ? or,
is the interactive technology being used to map visitor behaviour patterns to exhibition information delivery patterns ?
In this case it may not be necessary for the user to realise that their presence and movement patterns are actually causing interactive output.
If it is the intention of the designer to set up a direct interactive relationship between the individual and the space, then it is important to understand that this does not work particularly well on a communal basis. In a clearly interactive environment, people want to see a direct and immediate relationship between their input and the interactive system through the visual or audio output.
They wish to be able to manipulate and control the space quite separately from the in-put stimuli of others.
This is not of course a situation that is easily set-up or controlled within a museum environment, which is essentially a communal space.
Whilst systems such as the VNS do present head tracking and individual movement analysis tools, there is a problem in an exhibition environment to present the audio and video playback in such away that the individual is able to observe and isolate their impact upon the space.
I suggest, therefore that it is more beneficial in an exhibition environment to concentrate on programming these systems in such a way that represent communal behaviour patterns within the space.
That is, to concentrate on mass movements or mass variations in movement and to broadcast the interactive out put in a dispersed fashion.
The last point that I want to raise here is the necessity for the designer to consider whether the communication focus of such an environment is from a human perspective or a perspective other than human.
I mean by this, do we wish the interactive output to be representative of the perspective of human observation.
That is, are we seeing the landscape of the Himalayan peaks as if we were Sir Edmond Hilary on top of Everest, or does the visitor feel that they are strapped into a hang-glider flying some distance above the earth? These are of course both human perspectives, however within virtual environments it is possible to give the visitor a virtual body.
That is, we may wish to take them into the body of a whale swimming through a see of plankton. We may wish to take them through the perspective of being a spaceship landing on Mars. There could be any number of these applications but the perspective from which the interactive is actually communicating must be carefully defined as part of the design process in order for the technologies and the programming approaches to support that design perspective.
CONTEXT
A further consideration for the designer is of course the contextualisation of the interactive immersive environment. In order for the visitor to find it a useful, entertaining and informative delivery method the interactive immersive environment must be contextualised.
Environment
It is important therefore firstly to look at the environment.
What qualities do you wish to represent within the visual and audio environment that has been created? What are the points of interest that you wish to draw the visitors attention to and how are you going to represent those?
Therefore, what are the communication objectives that are embodied in the brief that has brought about the development of such an environment?
Characterisation
It is also of course important to consider the characterisation of any people, animals or objects that may be used to deliver information within this environment.
In doing so I think it is important to consider the emotional context for that character?
What is their investment in the environment that our visitor is experiencing?
What is their position and why are they reporting it.
It is also important to consider the psychological journey of discovery that the visitor is making within this environment.
How will it change their perception of the phenomena that is being represented?
What is it that they will learn?
It is important to map out this journey as part of the concept design.
SUMMARY
One thing that I haven't mentioned yet in this paper is that the trigger systems utilised by an immersive virtual environment can very easily be built into the exhibition.
It is possible in fact to use the iconography of the exhibition material itself to hold the triggers so that the visitors interaction with these icons causes interactive playback as desired.
As part of the summary I would like to touch upon something that I have not mentioned earlier and that is the context for interactive immersive virtual environments in the current technological and social environment.
I am currently doing research at RMIT into human movement as a controller within interactive immersive virtual environments and I have a paper on-line called Immersive Virtual Environments - A Social Perspective Link to the Essay.
This paper explores the context in which immersive virtual environment installation work has arisen in recent years and its parallel with developments in scientific thinking, the paradigm shift towards systems thinking, chaos theory, quantum theory, fractal mathematics.
Essentially an interactive immersive environment is a closed feed back loop between technology, the human condition, that is emotion, relationship behaviour, movement and the definition and experience of a physical space.
Each of these enacts upon the other and an immersive virtual environment cannot exist without all three.
Norbert Wiener a key proponent of cybernetics noted during his work on modelling social systems that
"it is certainly true that the social system is an organisation like the individual that is bound together by a system of communication and that it has a dynamic in which the circular process of the feedback nature plays an important role"
Fritjof Kapra comments
"the discovery of feedback as a pattern of life applicable to organisms and social systems helped social scientists observe many examples of circular causality implicit in social phenomena. The dynamics of these phenomena were made explicit in a coherent unifying pattern."
I believe that these quotes represent an important observation about the way in which immersive virtual environments reflect social patterns and community interaction in a way that is readily mapped into exhibition design.
Finally in summary I would to say that;
The interactive design process should be considered from the very out set,
immersive virtual environments can in fact drive other design considerations and this should be taken into consideration.
The contents of an immersive virtual environment are very powerful communicators themselves. That is,
moving images are a dynamic way of presenting information and maintaining attention, and
sound communicates in a more immediate and abstract way than other media.
My final point is;
budgeting properly for the production from the very outset is of paramount importance in order not to jeopardise full realisation of the design intention.