Dustin Chertoff has pulled experiential design insights from the advertising world to come up with a more holistic theory of presence in virtual reality. In 2008, he was in graduate school and was dissatisfied with the major theories of VR presence. His gaming experience showed him how much of his feeling of immersion was related to the content of the game. He wrote an essay published in the journal Presence where he laid out what he saw were the two major limitations of VR presence theory at that time, “First, many models tend to focus heavily on perceptual issues while focusing less attention other facets of virtual experiences, such as cognition and emotion.” Second, “these models fail to provide an interpretable, extensible framework with which to understand and apply the theoretical principles to practical applications.”
Chertoff finished his Ph.D. thesis titled “Exploring Additional Factors of Presence” in 2009, and then published his Virtual Experience Test questionnaire in the 2010 proceedings of the IEEE Virtual Reality Conference. Presence researcher Richard Skarbez first alerted me to Chertoff’s work after I asked him if he’d seen any prior research into presence looking at the different dimensions of my elemental theory of presence, which breaks down the subjective quality of an experience of VR into different combinations of Embodied Presence, Emotional Presence, Active Presence, and Social & Mental Presence. I was encouraged to see that Chertoff had independently come to an identical framework through his survey that was designed to holistically “measure virtual environment experiences based upon the five dimensions of experiential design: sensory, affective, active, relational, and cognitive.”
I had a chance to catch up with Chertoff in San Francisco during GDC this year, and we each concluded that our experiential design frameworks are functionally equivalent. We talked about his FearlessVR company that he co-founded where they design VR exposure therapy experiences for different phobias, but we spend the bulk of our discussion exploring how he came to looking to looking at the field of experiential design to inform presence theory. We also compare and contrast how each of our experiential design frameworks create tradeoffs and amplify different qualitative dimensions of an experience.
LISTEN TO THE VOICES OF VR PODCAST
Chertoff’s Ph.D. thesis “Exploring Additional Factors of Presence” has a great survey of presence research (see his summary graphic down below), as well as inspiration he’s drawn from flow theory and video game design frameworks like GameFlow. He summarizes Forlizzi and Battarbee’s definition of “experience” by saying it’s “something that can be articulated, named, and schematized within a person’s memory. Experiences of this type have beginnings and ends, but anticipation of, and reflection on, the experience may take place before or after the event.”
The idea of experiential design is that deeply immersive experiences form stronger memories, and that it’s easier for us to store new information along when we’ve had related experiences. Chertoff says, “Experiential designs are successful when they encourage people to create meaningful emotional and social connections—personal narratives that involve episodic memories, and positive associations with the artifacts of that experience.”
Chertoff cites Joseph Pine & James Gilmore’s book The Experience Economy as his source of inspiration for the five dimensions of his experiential design framework that make for a memorable and immersive experience, which map nicely over to my elemental theory of presence.
Embodied Presence corresponds to Chertoff’s sensory dimension which he says “includes all sensory input—visual, aural, haptic, and so forth — as well as perception of those stimuli.” I’d also include different virtual body representations, as well as the virtual environmental components which help transport you into another world and trick your perceptual system into believing that you’re in another world. Most presence researchers research embodied presence as virtual reality uniquely stimulates our sensorimotor aspects of perception beyond what any other communications medium can do.
Emotional Presence corresponds to the affective dimension which “refers to a participant’s emotional state. For simulation, this dimension can be linked to the degree to which a person’s emotions in the simulated environment accurately mimic his or her emotional state in the same real-world situation.” For me, this includes the storytelling and narrative components as well as music, colors, symbols, and deeper myths that all engage the emotions. Emotional presence can also come into the experience through social engagement with other people.
Mental Presence corresponds to the cognitive dimension, which “encompasses all mental engagement with an experience, such as anticipating outcomes and solving mysteries. For simulation, much of the cognitive dimension can be interpreted as task engagement, [which] is related to the intrinsic motivation, meaningfulness, and continuity (actions yielding expected responses) of an activity.” Game designers often talk about mental friction, and if there isn’t something in an experience to stimulate your mind then you’ll risk getting bored.
Social Presence corresponds to the relational dimension, which is “composed of the social aspects of an experience. For simulation, this can be operationalized as co-experience — creating and reinforcing meaning through collaborative experiences… Experiences that are created or reinforced socially are usually stronger than individual experiences and they further enable individuals to develop personal and memorable narratives.” I combine mental & social presence into the air element, because they both deal with the abstractions of thought and communication. But it also emphasizes the fact that not every experience has to have a social dimension to it, and that solitary experiences can be just as immersive and engaging to your mind.
Finally, Active Presence corresponds to the active dimension, which Chertoff described in the interview the degree to which you can express your agency, and physical engagement through taking action within the experience. He also sees it as a form of subjective engagement by saying “Does he or she incorporate the experience into his or her personal narrative; does he or she form meaningful associations via the experience?”
Chertoff assigns a few things to the active dimension that I would categorize elsewhere. For example, I think empathy is more of a function of emotional engagement, and that connection to the environment, avatars, and identity are more related to embodied presence. I tend to think of active presence being primarily as an expression of agency and will that includes exploration, curiosity, creativity, physical or virtual locomotion, and any type of interactivity.
There are qualitative dimensions of an experience that are sometimes hard to clearly schematize into a single category, and I believe that all of these different dimensions are happening at the same time all the time. But I do see that there are tradeoffs between active presence and emotional presence that I explore in much more detail in this introductory essay about elemental theory of presence..
A Survey of Presence Research
Chertoff has a lot of other references to presences research, the evolution of experience economies, insights from the user-system-experience (USE) model of user-centered design, as well as combining game design theory with flow theory with GameFlow.
Here’s a summary graphic of Chertoff’s survey of presence research as of 2009, where he summarizes the major components of presence in the different major models:
Chertoff argues that a lot of these models don’t have a good conceptual framework that can account for role of content in cultivating a sense of immersion and presence, and he sees that Experiential Design and Flow Theory can fill in those gaps.
Flow theory shows the relationship between the objective skills of a user and the objective challenges within an experience that result in a variety of different internal subjective states in the user.
When the skill and challenge is low, then that can lead to apathy or boredom. When the challenges are high, but the skill is low, then it can create anxiety or arousal. Flow states happen in a sweet spot of high skill of the user and high challenge within the experience.
Flow theory connects the objective aspects of an experience to the internal states of a subject, and it’s also helping to evolve existing user-centered design models. Most user-centered design research has focused on the usability of an interface with a specific productive output, but for gaming the experience of playing the game being the reward itself and so it’s more about the playability of the game and whether or not it helps the player achieve a flow state. Flow theory has been applied to Game Design in the GameFlow theory, which shows the role of content in creating a sense of immersion and presence.
Anyone interested in different frameworks and models of experience will find a wealth of references in Chertoff’s three papers: Improving Presence Theory Through Experiential Design (2008), “Exploring Additional Factors of Presence (2009), and Virtual Experience Test: A virtual environment evaluation questionnaire (2010).
Moving to An Experience Economy
Finally, I think it’s important these two experiential design frameworks are independent of virtual reality, and they can be applied to creating any human experience. Pine & Gilmore wrote a prescient article “Welcome to the Experience Economy” in the Harvard Business Review in 1998, and they say, “We expect that experience design will become as much a business art as product design and process design are today. Indeed, design principles are already apparent from the practices of and results obtained by companies that have (or nearly have) advanced into the experience economy.”
— Kent Bye VoicesOfVR (@kentbye) June 6, 2017
I’ve talked before about how Snapchat shows how we’re moving from the Information Age to the Experiential Age, and this article from 1998 lays out some of the economic trends that led experiential design paradigms used by companies like Starbucks and Apple. Here is a graphic from Pine & Gilmore’s article that describes the fundamental characteristics of this new experience economy that is moving from delivering intangible & customized services on-demand to staging memorable & personal experiences that are revealed over a duration of time.
Virtual and augmented reality technologies are driving our culture towards having more and more immersive experiences, and learning how to holistically create immersive and memorable experiences is going to continue to become a vital part the future of our economy and our culture.
For a more in-depth discussion about my elemental theory of presence, then be sure to check out this interview with Jessica Brillhart, an early discussion with Alex Schwartz, my SVVR 2017 keynote talk, and this No Proscenium podcast interview with me talking about how it could be applied to immersive theater.
Donate to the Voices of VR Podcast Patreon