Rob Morgan is the writer of The Assembly, and he spoke last year at GDC about narrative design in VR and the importance of accurate body language in NPCs to maintain a sense of presence. He returned to GDC this year to talk about some of his lessons that VR can learn from writing stories in AR. He has a lot of interesting insights about how to use ambiguity and implication to drive narratives within a first-person story to allow a user to have a more immersive and personalized experience.
LISTEN TO THE VOICES OF VR PODCAST
My interview with Rob last year was one of my top 10 favorite podcasts that I’ve done because he has so many interesting insights about what’s unique and different about VR as a storytelling medium. He was the first person to point out to me the importance of body language cues in NPCs to maintain a sense of immersion within a story, and that the lack of believable social interactions with artificial characters can break presence and disrupt your suspension of disbelief.
Part of the magic of storytelling is that we allow ourselves to believe in the overall context of a story as well as the individual motivations of each of the characters. But the problem within an immersive story where the user is playing the protagonist is that you actually know very little about who the identity that user and what’s really motivating them. So instead of creating a well-defined set of motivations for the main character within the context of the story, Rob suggests using the mechanism of ambiguity to allow enough spaciousness within the story for the viewer to project their own identity into what they’re doing and why.
In my interview with Devon Dolan about the four different types of stories in VR, he makes the distinction of whether the character is a spectator and ghost within the experience or whether the character is a part of the story and has some agency within it. If you are a character in the story, then Rob says that some of the essential questions that the user will be asking are “Who am I?,” “What is my relationship to what is happening around me?,” and “How much can I affect it?” In that way, becoming the main protagonist within an interactive story becomes an interface to the experience.
One useful mechanism that Rob has discovered to help immerse you deeper within a story in VR is to have other characters implicate you in some way. You may have a secret or be accused of a murder, or they may just tell you, “Just act normal.” As soon the other characters make an assumption or assertion about your motivations as to why your doing something, then it allows you to clarify your own internal sense of what you’re doing and why. Then if you start to internalize or express that sense of agency within the story, then it allows the user to take their roleplaying to the next level and to get really emotionally hooked within the story.
Virtual and augmented reality also opens up new types of genres and stories that were previously impossible before. Conflict and growth is the heart of drama, and a lot of films and video games use the trope of external combat to express this. But Rob sees the potential to explore stories in VR and AR that are much more about internalized emotional conflicts within the main protagonist / player of the experience.
There’s a number of moral dilemmas within The Assembly including a specific moment where the player has to decide whether or not to press a metaphoric “evil” button. His goal is to not have the player use their rational mind to calculate the costs or benefits of the action, but to get to the point where the user makes more of gut decision as to whether or not it feels good or bad. It’s a much more visceral experience and memorable experience if it’s felt in their emotional body rather than a result of an abstracted mental process.
Rob believes that the canvas for all storytelling is within the space between the user’s ears, and that VR has the capability to evoke a level of immersion within a story that goes beyond what was possible before. He says, “We are ALWAYS dealing with mixed reality, whether it’s AR or VR, because the reality at the center of the experience is always the player.” And by using the mechanism of ambiguity and implication, then you’re able to both have more of an impact on allowing players to express their identity within the story but also exert some sort of direction or control in how the story unfolds. Leaving the story open-ended enough will allow the user to fill in the blanks with their own story, and in the end create an experience that will be more personal and leave a deeper impression.
Rob is a freelance writer for immersive technologies, and you can read more of his thoughts at @AboutThisLater or on his blog.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast.
[00:00:12.015] Rob Morgan: My name is Robert Morgan. You might remember me. I was on the show about a year ago. It's a pleasure to be back. I am a writer and narrative designer, whatever that is. And right now I primarily work in VR and also in AR. And I write dialogue scripts. I design stories and I design experiences for cutting edge technology, I suppose. I've also worked in the past on console games. I used to work for PlayStation and now I'm a freelancer.
[00:00:42.377] Kent Bye: Great. So tell me a bit about the talk that you're giving today here at GDC.
[00:00:46.480] Rob Morgan: So my talk was about what I learned writing for augmented reality and what I took from augmented reality and took into virtual reality in terms of how to design experiences, how to get players to bring a little bit of themselves into the experience, and how to create emotional immersion. This is what I've been talking about. I was talking about this last year. So last year, I talked about Creating NPCs who felt real, even if you didn't necessarily have the budget to get over the uncanny valley in visual terms. but instead having a lot of emotional effects that are, well, cheaper than visual perfection, but also, in my opinion, more effective because they're emotional effects. So, for example, getting NPCs who make eye contact in a way that we might expect them to, but also, if you can't afford to get them to make eye contact in a way that's predictable, then finding ways around that. This year I've really extended out stuff that I've talked about previously, stuff that I talked about last year. I'm talking about the way that augmented reality has taught me a new approach, I suppose, to the way that the player identifies with the person that they are in the simulation. I got my start in augmented reality. I worked for PlayStation, first of all, as a writer at London Studio. And what I worked on was an augmented reality title called Wonder Book for the PlayStation 3. It was an augmented reality book, and the book was a physical cardboard book that the player interfaced with. The really key thing about that experience was that the player, the user, was the center of the experience. They were themselves in their own living room and the experience was mediated through their TV, through a camera. So the experience was themselves watching themselves on the TV, watching themselves interact with the book, watching themselves become a wizard and interact with the graphical, magical elements that we were able to create. And it was a great learning experience because It taught me how to design dialogue, how to write dialogue for experiences where you have very little control over the player. And paradoxically, I think that's really valuable in VR. Because in VR, I don't think the revolutionary thing is that we have a huge amount of control over what the player sees. I think the revolutionary thing about VR is that we have a closer relationship with the player than ever before. But that means we have a closer relationship with reality than ever before. Because the player is the central reality that you are trying to influence. In augmented reality, the player is the central reality that you're trying to augment. But in VR, you can't get away from the fact that the player is at the center of the experience. And so when we talk about embodiment and the embodiment problem, when we talk about identity in VR, I think what we're talking about is the fact that we need to try to get players to want to embody characters. We need to be able to create characters that players want to embody, but that doesn't mean doubling down on things that we've been doing in games, flat screen games, for decades. because the types of protagonists, the types of characters that we see in flat-screen games are partly a function of, frankly, demographic assumptions about who gamers are and what they want, but also a function of a kind of arms race of thrills and effects that were necessary when we were working in a flat-screen medium. When we were working in a medium that was over there, it was on the other side of the room, it might be the size of a playing card on your TV. And so all the bonkers physics and all of the high ground speed and all of the thrills and the emphasis on combat, all of that stuff was a function of the fact that we were working in a flat screen medium. And now we're finding that VR is encouraging us to create entirely different types of experiences. And I think we need to create entirely different types of protagonists. and bring players in in a different way. It's not like making an action figure, trying to make an ideal action figure that we know that our players will want to grab off the shelf first. It's not like trying to design the perfect formula for a character that players will definitely want to slot themselves into. Because frankly, in the past, that's always resulted in fairly formulaic characters who are easy to predict. They're brown-haired white guys, and we see a lot of that in games. And that's absolutely fine until it becomes an over-saturating effect, which frankly it has in certain strains of flat-screen games. That's fine, and flat-screen games aren't broken in VR. isn't a fix for flat-screen games. VR is its own beast. But I think that means we need a completely different approach to getting players to identify with who they are. And that means ambiguity. That means, for me, implication. Not as in, like, you just imply things. That's important as well, because as a writer, that's a great way of writing less. Writing less is very important in VR, because VR users have a lot on their mind. I mean implication, like you get implicated in a murder. I mean, getting the player involved by getting them implicated, giving them a secret that they're keeping, giving them something that they are engaged in, telling them that it's already too late, and telling them that kind of thing before they even know who they are in the game, and getting them on the side of the simulation, getting them to feel like, oh, okay, I have a secret, I'm invested, I have to pretend like everything's normal. And I've found working in AR, creating those effects is a great way of getting players on side when the player and the protagonist are the same thing. There's nothing in between them. I think it's a really powerful way of being implemented in VR, where we might ask the player to roleplay as a character, and they're embodying that character. There's nothing wrong with roleplaying. But we also have to leave a bit of ambiguity for the player to be themselves and draw their own conclusions about what they're doing and sustain their own motives for why they're doing what they're doing rather than trying to give them the whole character sheet and tell them that that's why they're doing what they're doing. Because that, I think, pops you out of immersion just as fast as a texture pop or just as fast as, you know, a physics failure or something else. That's why identity is important.
[00:07:02.925] Kent Bye: Yeah, as I'm listening to you, one of the things that comes to mind is one of my own kind of personal theories about the difference between 2D media and VR and some of the differences. And what I think is that in 2D films, we see kind of a hero's journey where there's this going out and exerting your will into some sort of action adventure. And I feel like that's a very young, external type of energy. And VR tends to be something where you're actually more receptive. You're taking in the environment and it seems to be a slower pace. It doesn't lend itself to have that same type of action adventure just at a fundamental technological standpoint from being able to have locomotion issues and all the things that come with that. but you're talking about perhaps having a limited mobility, but you're kind of receiving story in a different way or receiving information in a scene and kind of taking it all in. And so I personally wonder if that's going to actually change the traditional stories that we have. Let's say in the future, we have like sci-fi dystopian stories because that's just more interesting from a dramatic point of view when things go horribly wrong. So I imagine with virtual reality, we may be able to kind of paint a picture of a more optimistic future, but In terms of story, I think there still has to be like tension and growth and change and so I'm just curious from your perspective when you look at all that, how you see the stories that are told within VR are going to be something that's unique and different and something that would really not really work in any other medium.
[00:08:28.943] Rob Morgan: I think you're absolutely right. I think the key for me is there's a difference between conflict and combat. Conflict is the core of storytelling and I'm a writer so that is my business, that's my bread and butter. Combat is something slightly different and I think the emphasis on combat, I think you're right, that is a function of a type of game that we happen not to be working in. I like hardcore games, I like shooters as much as the next person, but I think it's a function of the formal qualities of VR that we do need different experiences. slower ground speed, the necessity to have smaller but more detailed environments. So the game that I finished most recently is the Assembly for Endreams. And one of the distinctive things about that is it's a point-and-click. And we had to re-engineer the point-and-click, but also we kind of had to rely on the old rules of the point-and-click. Detail in the scene, getting you involved in a single scene, having to make something that was so interesting that the puzzles could sustain the fact that you were just looking at the same one thing the whole time. And what we did with the assembly was we found that the traditional way of telling stories, the way we were used to in flat screen mode, that it wasn't quite working the same way. It was like putting paintings in a gallery. So this is my analogy. If you put the paintings up on the wall of a gallery, traditionally, theoretically, you know how the user is supposed to consume that. They're supposed to walk up to the picture with their hands behind their back, look at it, cock their head to one side and go, hmm, and then move on. And we were finding that VR users just weren't consuming it like that. They weren't theoretically ideal gallery patron. They were running around like kids in a gallery. Because the story was there on the wall. Hey, guys, we've got all this great story for you on the wall. Come and consume it. But they were just as interested in the rest of the room. They were interested in the reality of the simulation, because that's what VR does. It makes the user interested in the reality of everything they're seeing. And so our traditional way of delivering chunks of story had to be completely adapted. I really love the fact that you think that that's going to kind of result in a more optimistic style of storytelling. I would love to think so, too. I think we're always going to need conflict. But I also think it's really interesting that, frankly, I used to worry that VR would be kind of a ghetto for hardcore experiences after console and other mainstream gaming experiences had moved on from what we currently think of as hardcore. and the cultural limitations of that medium. I used to worry that VR would actually be a bastion of really, really hardcore experiences and that the early adopter phase of the medium would kind of continue on and people would still be basically firing shotguns at one another in the medium long after other experiences had become mainstream on the other formats. But now we know that actually hardcore experiences either don't work or have to work very, very differently in VR. And it means that VR is moving the ball forward in terms of culture, in terms of culturally and generically what people are doing. And VR is actually being a pioneer. in terms of completely different experiences that we had never thought we could see in gaming or that have only existed in indie gaming. And I think it's a fantastic opportunity because what's fantastic about VR is not the whole idea that we are in the business of tricking the player into thinking that they are somewhere. That I don't think is what it's about. I think it's the fact that we have a really unprecedented access to certain emotional centers and certain visual centers. And we can create experiences that aren't necessarily like an overwhelming combat experience. We know that that overstimulates VR players, or at least it does right now. But instead we can create contemplative experiences. really intense experiences, but ones which are more conceptual or they're more visually playful. They're less based on the fundamental principle of combat and they can explore different kinds of conflict. Emotional conflict, personal conflict, conflict within yourself and your own goals, ambiguity within yourself and your own goals. So an example of what I'm working on from the Assembly. And I think I talked about this last time, but it really dawned on me in a new light in the course of this year. We essentially have an evil button in the Assembly. At one point, you're told to do something really, really unethical. Whether or not you actually have to do it, I don't want to give any spoilers. But you are told and you have the opportunity to do something really unethical. And if you go ahead and do it, then you keep pressing this evil button. And I found, okay, I've got to write some dialogue for this character that you're embodying, the role that you are role-playing. And they're strongly characterized characters. And, okay, what am I going to write? What am I going to be having you say when you're pressing the evil button? More and more I was just pulling it back and having you do and say less and less while it was happening, because I wanted to take that character out from between the player and the experience. I didn't want you to have role-playing to hide behind and instead I wanted you to be responsible for your actions and so without firing a shot there's a conflict in the this example in the assembly, but the conflict is something we're trying to create within the player. We're trying to create a sense of a sustained action that they're doing while we're also giving them plenty of reasons not to do it, or suggesting reasons that they might think that they shouldn't, and then forcing them, challenging them to justify why they're doing it, even just mentally, and then doubling down on that by having other characters in the simulation. speculate or accuse them of doing it for a certain reason. Oh, I bet you're doing it for this reason. I bet you're doing it for that reason. And as soon as we start challenging users and creating supposed motivations for them, but not locking them down and saying, you're doing this because it's your quest. You're doing this because your wife was killed by orcs. You're doing this for this and this and this reason. Instead, we're creating an ambiguous, a possibility space And then we were accusing the user of something that they may or may not feel that they were doing. And then in user testing, we suddenly found players in the headset going, no, that's not why I'm doing it. I'm doing it for this reason. I'm doing it for that reason. We were able to bring the user's identity and force them to roleplay. Suddenly they were roleplaying, even if they didn't think that they were. And that's a magical moment for me. Like, that's a moment that I know I've got them emotionally. I know I've got the player hooked because Instead of trying to visually overwhelm them, or in addition to giving them a beautiful place to be, and for me as a writer, trying to create someone they want to be, trying to create a story they want to be a part of, I also force them to bring their identity with them into the simulation and embody themselves and justify their actions. Just that process of challenging them takes the user on a next step to the point where they're not only role-playing, But there's a bit of slippage between their own identity and the identity that they're roleplaying and the person that they are. And they're forced to take a bit of responsibility for what they're doing. Suddenly, the impact of what's happening and their choices is more than just a roleplaying roll of the dice. It's something that hopefully will stay with them.
[00:15:36.496] Kent Bye: Can you expand a little bit more about what you mean about using ambiguity or giving a character a secret and how that would be used to kind of drive a narrative forward?
[00:15:46.478] Rob Morgan: So what's informed my VR talk this year is the work I've been doing in AR. So I'm currently working on a piece of site-specific audio augmented reality theater. What this functionally means is that we usually envision, we imagine augmented reality to be a visual medium. We need to have a glass or a contact lens or something so that we can superimpose graphics into the visual field. Now, we're not really there yet, but the idea of having a layer that sits on top of reality and that gives you a different context to reality, gives you a different context digitally mediated for what you would be doing anyway, creates a different imaginative context to your own neighborhood, that's something I've found exciting and have found exciting for years. And I'm also finding in the medium that I'm working in at the moment, you can do it with nothing visual, you can do it with only audio, because frankly, Even if 90% of the augmentation that you're doing, it takes place in the user's head. As long as you can change their relationship to reality, even a little bit, then it's a really powerful effect. Because then you just add an extra ingredient to something that the user experiences really powerfully. Reality. So, in this experience that I'm developing, the first thing that the player hears is going to be something which implicates them. Before they find out who they are, they're going to be told something like, just try to act normal. Hey, hey, are they still watching? Something like that. Because A, in augmented reality, it's usually a defining characteristic that the player and the protagonist are the same person. You are the protagonist, which means A, you can't assume anything about the player demographically or motivationally or in role-playing terms. You can't say who they want to be or what they want, what they want out of it. You can't even say if they're still gonna be playing in two minutes time. Or if they're going to exploit the flaws in your simulation and wander off in a direction that you don't expect. Augmented reality trains you to write for players that you cannot control. And that level of implication is a way of handling the fact that you know very little about your protagonist, i.e. the player. You know very little about them and you can't control them. So what you're forced to do is to implicate them and get them engaged as best you can, not in traditional storytelling methods of just creating a character and saying you are them and saying, OK, go. This is your quest. Vengeance and vengeance is over there. Go. And you got to gain 30 levels before you can get there. Again, there's nothing wrong with those experiences, but I don't think they're going to have the same impact in VR compared to experiences which take into account the fact that the user feels more there. In VR, just like in AR, you are more there, you are more yourself, you're ineluctably present in the simulation in a way that you just aren't in flat screen games. So I found, personally, This process of implication, this process of getting you secretively involved, or that you have some dire thing that you're holding against everybody, or something is amiss. Something like saying, just act normal. There's no more evocative sentence in the English language than just act normal, because the minute you say it to somebody, their whole behavior, their entire relationship to reality changes. And you can see it, the set of their shoulders changes. You go up to somebody at a party and you just go, it's okay, just act normal. You have screwed up that person's evening. And it's a great way to get people on board with your simulation. They might not like it, it might not be a comfortable sensation, but they are on board. And you can do that before you give any world building, before you tell them who they are. And so it's a trick that we tried to pull in the assembly as well. One of the characters is railroaded into the gameplay in a really classic way. It's a dual character experience. One of the characters is kidnapped, and their gameplay is a process of escaping. Classic video game conceit. You don't have to ask, why am I doing this? Because you have a very clear goal. You're escaping. But the other character has a mystery plotline, and their relationship to events around them is much more ambiguous. They're a conflicted person. And so the way we got you involved was not to deliver a load of exposition about your motivations, how you got to here, what you want, what you maybe don't want, what you're conflicted about. All that stuff we could do later. The first thing we said was try to act normal, because then that accesses an emotional center of the user's brain really directly, in the same way that VR directly accesses the visual centers of their brain. It gives the user emotional information, not expositional information, and it grabs them. Hopefully, at least, that's the aim. The whole idea, basically, is that we try and make you feel guilty for something that you haven't done yet. We tell you that it's already too late. And then you start finding out who you are.
[00:20:40.872] Kent Bye: And so, you know, I've been doing a lot of different interviews about narrative and storytelling, world building, you know, I went to Sundance and I've been thinking about this a lot in terms of what is it about the virtual reality medium that makes it unique as a storytelling medium. So I'm curious from your perspective, you know, it seems like a lot of things that you're talking about, I hear you go down these plot lines that you're putting these people into these moral dilemmas where they have to make a decision that may be against their own ethical value system. And so I start to wonder, like, what is the deeper intent of why would people want to do that? Then I realized, well, a lot of people go into horror or very scary experiences because it's exhilarating in a certain way. But I'm just wondering from your own perspective, what you see the deeper thing that's going to be drawing people to experience these types of immersive stories in VR?
[00:21:27.190] Rob Morgan: So I think the ethical dilemma thing, we traditionally treat it as a metric. And that's the way that it's often been presented to us in flat screen games. And I think that's because choice in flat screen games is very closely related to consequence. The point of having a choice in a flat screen game typically is to display the consequence, to make the player feel empowered by showing them your choice had this effect. And so whether that's because you're suddenly dark side and then you develop dark side powers and you take it in a very logical final way to, this is a very powerful way of expressing a certain metric by throwing a load of Jawas down a well or something. But I think in VR, because you are more there, because you're radically embodied, I think we're going to find that in a lot of simulations, Presenting you with an ethical dilemma is going to be an exercise in leaving traces behind in the user's brain, not necessarily playing out the consequences, because the consequences are a way of empowering the player. But in VR, you have such a radical level of empowerment. I think that it's going to be much more important to leave a level of ambiguity which says you did something evil, and you're going to have to live with that. No, I'm not saying that you should just be left hanging with that. I'm saying that in VR the really intriguing things... is not necessarily seeing your skin turn from, you know, light side to dark side. I mean, the moral choice in VR, for me, the real point is the moment that you press the button. We have to set that moment up. It's almost the preamble that's as important or more important than the consequences, because the moment you press the button has to not be a calculated choice about choosing between various consequences. It has to feel evil or good at the time because that's what VR gives us, that's what radical embodiment gives us. It gives us the opportunity to have this really charged moment rather than a process of role-playing where you are choosing between two or three or four different definitions, different formats of your character. As for what I think is going to draw users into VR, I don't want to pretend like action experiences aren't going to draw users into VR. I think that's absolutely going to be a really key thing. Really in-depth simulators. You look at Elite and it is literally the game you dreamed of playing when you were a kid. And what's going to bring users to VR? I think it's experiences that stay with them after they've taken the headset off. For me, that's the key thing. Because VR gives us a completely different way to access the user's brain and emotional centers, and a really intense way to do that, I think experiences that stay with you is the thing that we should be aiming to create. Not experiences that are exhilarating at the time, although that's absolutely something that there's an art to that, and people still go on roller coasters, that's amazing. What I personally find really interesting is the idea of an experience which, when you take the headset off, has changed the way you relate to reality. Because the point is... When we talk about virtual reality, it's not the virtuality that's important. It's the reality. We have a closer than ever access to the player's reality. We can change the way that they relate to reality, the way they walk around, the way they see the world, the way they feel empowered or disempowered. And we have a way of making them feel reminded of what they did in our simulations when they're walking through their own neighborhood. For me, that's the key experience. It's not the moment that you pull the trigger. And it's not necessarily the moment that you see the outcomes of pulling the trigger. It's the moment that you wake up the night after remembering pulling the trigger. because flat-screen games have been able to get at that at their absolute best. VR has a direct line to that stuff in a way that flat-screen doesn't. That means we have to be very responsible as developers, but it also means there's an amazing opportunity to create experiences the likes of which we haven't seen before. That's why it's still so fascinating to me, and that's why I think that I continue to believe that augmented reality, where the majority of the user's experience is reality, and there's a little bit of augmentation, it still has a huge amount to teach virtual reality, where the majority of the experience is virtual, and there's a little bit of reality left. What's left, the reality that's left, that's the player. and we have to be able to work with the player and bring them in and cooperate with them, rather than trying to compete with their own sense of reality. Because we're not in the business of overwhelming people into immersion. Immersion is a pact that we have to make with the player, and that means helping them, intriguing them, and encouraging them to develop their own sense of willing suspension of disbelief. And just because we can create really powerful, really convincing images, doesn't mean that willing suspension of disbelief is going anywhere. Because I still think that that is just as much the core of our medium, just as it was for Shakespeare.
[00:26:35.240] Kent Bye: Yeah, I think that your insights that you told me last year about adding a lot of social behaviors within NPCs, to me, really spoke to the process of cultivating and preserving presence and what is required for that. And after I talked to you, I went to the IEEE VR and talked to different academic researchers about presence and the two main illusions of presence, which are the place illusion and the possibility illusion, that you're in a place, you know, VR can take you to all different worlds, but making sure that that world is coherent and makes sense and is believable. I think a lot of the things that you were talking about, having NPCs and all the body language, is something that speaks to that possibility of maintaining that presence. And one of the things that came up also was that Oculus Story Studio did Henry, where they found that eye contact was super important, and that was what people were finding really compelling, and that people felt really present. And I was like, in the back of my mind, I was like, yeah, I know. After I talked to Rob, I kind of had figured that out. I think it's one of these things where creating a narrative experience where you feel like you're kind of a ghost or maybe you're a character, but yet still having the sense of a body presence. To me, I think there's a lot of different nuances of how to cultivate and preserve that presence. And I'm just curious from your perspective as a storyteller, as a narrative designer, what kind of things that you've done in terms of how to maintain and cultivate that presence?
[00:28:00.584] Rob Morgan: So for me, without getting too buzzwordy, I genuinely believe there's the place illusion, the present illusion, and then there's the personhood illusion. And that means how you relate to the NPCs in your environment, but also how you as the person, the key person in the experience, relate to your environment, relate to other people within it. I think that even in the most straightforward 360 live action VR experience, and the most sophisticated VR role-playing experience game, there has got to be a question. Who are you? Who is the spectator? And how far are we making them or letting them be a spectator? And how far are we bringing them in as a protagonist? The example of Henry is really interesting because, yes, the eye contact was really, really interesting and was immensely powerful effect. But also, Matt Burdett of the Story Studio then talked about, in a blog post, he talked about something that they had started calling the Swayze effect. Because in Henry, they added in the fact that Henry the Hedgehog made eye contact with you. They added it in because it was a really powerful effect because it's a social element and social elements create an emotional weight in the mind of the player that forces them to fill in the gaps in the reality or in the virtuality. The limitation of it in the specific case of Henry was that there were formal problems with the demo that meant it didn't make sense to have eye contact because the point of the demo was that Henry was on his own on his birthday. So the fact that he kept making eye contact with you meant that there was a dissonance within the scene, because the loneliness that was being enacted was being undermined by the fact that he kept looking at you. So it was an interesting effect, and I think one that, you know, it's a brilliant demo, it's one that they should be really proud of. But it also created a weird emotional effect, because it made you feel like you were silently watching this sad hedgehog celebrate a sad birthday, because he knew you were there. Why weren't you doing or saying anything? Why weren't you wishing him a happy birthday? And it suddenly crystallized the fact that we might be accustomed to the idea that if we put that demo on and watched him be lonely and he didn't make eye contact with us, we might be content to think of ourselves as spectators. But we have an identity in the scene because we're radically embodied, because that's what VR does. It's very difficult for us to conceive of ourselves as floating cameras. We know what CCTV looks like. And if the camera, our view, was positioned in the upper right corner of the room, then we might be able to imagine ourselves as spectators. But if we're at eye level and we can look around, we are enacting personhood. That's what the camera is doing for us. And it's very hard to imagine ourselves as a camera when we can move around fully because we know how cameras behave. So it was a brilliant example that really showed us that the way that you interact with the scene is not just about the affordances of interface. Because in VR, you are a protagonist. You can't get around that. And a protagonist is an interface. You have to understand your relationship with what's happening around you in a way that a spectator simply doesn't. A spectator understands that they have a neutral relationship with what's happening around you. That's what implication means. Accidentally, or rather, you know, because they created an interesting effect. In Henry, Oculus created a demo which implicated you in the loneliness of a really cute hedgehog. And they created a really uncomfortable, dissonant experience in the process of doing it. Which is a brilliant example for all of us. If you wanted to set out to create that effect, you couldn't have done better. It was a Pixar-esque masterpiece. It's like the first 10 minutes of Up. It's heartbreaking stuff. And I think that shows us that we have this new kind of access to the player's mind, but that comes from the fact that your identity is in flux. The minute you put the helmet on, you're trying to calibrate who you are and how you relate to the scene. If you are identified as a ghost, and if you can find a way to make sense of that to the player, you are a spectator. You are involved in the sense of being able to float around, but you are a ghost. If that's communicated mimetically to the player, they understand that they're a ghost, they understand how that feels, and they learn to embody that, I don't think there's a problem with that. But if you don't allow the player to feel part of that, then you might as well just put them as a webcam on the end of a Dalek and have them just glide around, like weirdly experiencing the scene without any human interaction whatsoever. Because I believe it's inherent in being in a VR simulation that there's a possibility space around the player. Who am I? What is my relationship to what is happening? How much can I affect it? That's the essence of the fact that you're more there than you are in a flat screen experience. And if what you can do in the simulation is nothing at all, that's fine, but that has to be clarified to you. Because otherwise, there's been amazing documentary VR work being done, actually a really interesting demo, sadly I can't remember the name of it, one of the most interesting VR demos I've ever seen. put you in a headset, gave you a phone-like peripheral to hold, and it made you a witness to an atrocious beating by a group of, I believe, border guards. And it played on the fact that being there implicated you, because there's no such thing as a passive witness. You can be a passive spectator, but I think in VR you are a witness, whatever's happening, because you're more engaged. And I think that gives us a great opportunity, but it also means that When we misstep, we can create effects that we never intended to, and our task is to learn from those effects and then learn to manipulate them next time. I mean, hell, I've written for a horror game where you experience the process of being operated upon, and it was a pretty harrowing scene to write, but just the idea of envisioning it in VR, when the user was implicated in the body that was being operated upon, It was a pretty horrifying experience and in the end we didn't make it. At one point the guy kind of came at you with a knife and he operated on your eye and popped your eye out of its socket. Okay, possibly I should have given her some sort of trigger warning before this. Anyway, yeah, it was a way of radically interfering with the way that the player was present in the simulation and it was unpleasantly effective.
[00:34:40.899] Kent Bye: I think there's a trade-off there between what they call the bat test. If you have any things that are going to be as extreme as that, it's going to break presence because you're going to have to question to yourself, is this real? No, it's not real. It's OK. Because it's a fine line to put the protagonists in danger with these different contrived situations. But yet, if you put their lives too much in danger, like pointing a gun at their head, then that, in my experiences, had the side effect of just having my rational brain really kick in and say, you know, yeah, this is not really real. You're OK.
[00:35:11.167] Rob Morgan: Right, and that interfaces with your identity as well, like Endreams are doing really amazing work and they've done some really interesting talks and I encourage everybody to go and have a look at the design postmortems or design update talks they've done on the assembly because they talk about how One of the original things that we conceived of was to mimetically simulate the experience of having vertigo. Whether or not the player had vertigo, the character had vertigo. So you were going to go through this walking along a balance beam experience and we were going to simulate nausea, vision effects, vision blurring, all the things that were associated with vertigo. And what we found was it was in effect, too effective. Because it had a nausea impact and, you know, obviously we're very sensitive about that in VR and we don't want to play around with those buttons too much. But also, it did interfere with the player's identification. And what we found was that we could do 90% of that effect by having voice that covered the fact that the character was experiencing vertigo. But we didn't do the vision blur. We just have What's essentially your voice, the character that you are roleplaying, the character you're embodying, is having a really hard time. They're walking along this balance beam and it's hell. But also it's ambiguous and it allows the player to experience that in their own way. And so the horrifying experience, like you were talking about, is still something that they can accommodate within themselves rather than something that we're performing to them. And we could do 90% of the emotional impact without trying to simulate the physiological impacts of Vertigo. We could just do the emotional side in terms of putting a voice to the fact that you were feeling horrible. And we still have players taking their headset off and going, that was horrible. after the experience. And that's what we were shooting for. But we can do it without pressing that nausea button because we can play with the fact that, are you them? Are you not them? And the fact that the player is kind of taking a step back and going, this isn't me. I don't have to panic as much as the player that I'm role playing is panicking. I think that's fine because they're still in the helmet. They're still in the distraction-free zone of VR. And players are better role players than we give them credit for. If we've set them up right, players can interface with their identity in the game. They can merge in and out. Because what we're talking about is mixed reality. We are always dealing with mixed reality, whether it's AR or VR, because the reality at the center of the experience is always the player. And there's no point in trying to compete with their reality. We have to find ways to get them or bring them or mix their reality into the simulation.
[00:37:55.392] Kent Bye: Yeah, I feel like the whole realm of telling stories and storytelling in VR is something that is still a little bit of the Wild West. And there's people that are out there, like yourself, and people that are creating 360-degree videos, creating interactive, dynamic, virtual reality experiences. But it's still early days in that, you know, five, ten years when we look back, we'll see how early it is. I'm curious, from your perspective, as you're listening to the podcast and listening to other people's insights, what are some of the things that really stuck out to you in terms of key insights that you've had in terms of storytelling in VR?
[00:38:30.853] Rob Morgan: Honestly for me, and I know that I might be talking myself out of a job here, the really, really innovative things I've seen have been emerging narratives in terms of games which are designed to be participant. Because one of the strange things about our business is that It's kind of lonely. You're inside the headset and we can deliver content to you, but it's just you. And A, it's a shame to miss out on the same sofa co-op experience because that's how I grew up playing games. I have an older brother and that's a core part of my experience. But it's also, it's impossible to ignore that Twitch and other streaming services are a big part of our ecosystem. It's hard enough to sell VR, to sell the VR experience as it is. If we can't display the experience and show how much fun it is, then it's difficult to sell. I think we need to keep hold of the fact that the earliest VR narratives, per se, in the current generation, were the YouTube reaction videos of people seeing early Oculus demos. Because to me, that is what sold the format. Those faces of people going, oh my god, it's a roller coaster and it's real. Those people are telling themselves a story. They're filling in the simulation with an emotional content. And so, For me, the things that stick out and the things that kind of give me faith and show me that the medium is going places that we've never even imagined before are not just a great script or a great experience, because I think those are on the way. But you're right. Exactly how storytelling works in VR, it's totally up for grabs. I myself, really, I spend more time identifying the problems than the solutions. And I think in five years, we'll look back on some masterpieces and go, well, of course. But now it's not clear how storytelling works. But the storytelling of the other people in the room, the other people on the other end of Twitch, that's stuff that we can manage right now. And so things like Unseen Diplomacy from TriPixels, they're doing amazing work. They found a way to make VR social in a way where because the experience of you experiencing the game. That's something very private. But then you take the headset off and people tell you what it looked like when you did it, and they show you poorly shot phone footage of yourself awkwardly clambering around. That's part of the story. And that's an interface between public and private that really brings the headset out of the bedroom, out of the darkened basement, and into an experience that's there for everybody and for you to retain a story about. That's a personal drama. That's what you're going through with Unseen Diplomacy. And you're forced to kind of create a space for that personal drama that still makes sense of the fact that it was really serious what you were doing, even though you also know that on the outside you look ridiculous. That's the point where you have flipped the switch that makes the user's brain tell themselves a story about what they're doing. And frankly, that is always going to be a better story than anything that we can tell because the user is filling in the blanks. And that's always going to be a story that suits them better than anything we can design.
[00:41:36.424] Kent Bye: Yeah, one of the things that Eric Darnell told me after I stopped recording the podcast that I did with him at Sundance, and he's from Baobab Studios, he said that in film it's kind of like one individual telling you a story of their experience, and in VR it's kind of like you're giving somebody an entire experience that then they generate their own stories from. And I think to me, that's one of the best distinguishing differences between this new medium is that it's less about telling a direct narrative, but giving someone the full experience that allows them the freedom to create their own stories from it. And so, you know, in talking to VRChat and Jesse Jodry, having a group of 20 people go through an experience of like Dante's Inferno with maybe a guide who's giving you a tour of things and The story is amplified and changed as more and more people are experiencing it and sharing their stories together. And, you know, it's like the story is told by the multitude of perspectives. And that's something that I think that Alex McDowell, who wrote the Minority Report but also did the Leviathan Project, where he's talking about this process of world building in VR. In the beginning, many generations ago, before the Gutenberg press, before the technology came about to be able to have authors give a singular perspective about their stories, stories were told in an oral tradition where they were passed along from many generations. they would change and adapt and evolve given all the different perspectives and that with technology we've sort of fractured out into this singular perspective that we've just been living with since like 1455 and now we're getting back to VR where we're able to bring back those collective stories together again.
[00:43:12.394] Rob Morgan: My motto, my touchstone, especially when I go and do like a public access or a new scientist talk about VR, and I get the inevitable question about, oh, well, what happens when people, these kids with their VRs, they get lost in the simulation and they go off into their wish fulfillment fantasy and they lose touch with reality? What I always say is writers have been able to create total immersion using nothing more than ink on a paper for thousands of years. And that's because the canvas for all storytelling is not the canvas, it's the space between the user's ears. And so in VR we have a different relationship to the user's brain, the user's kind of story tabular than we did with any previous medium. But the business is still fundamentally the same. Because the minute you can get them telling themselves a story, or getting them telling a story amongst themselves and amongst their friends, that's the story that will stick with them even if, from my perspective, this is a poorly structured story. Or, you know, this is a story that doesn't adhere to the verities. It doesn't matter because it's their story. And for me as a writer, my job is not to override or overwrite the conclusions, the stories, the narratives, the emotional effects that the player brings with them out of the simulation. Like you said, oral storytelling is not about the story, it's about the minor variations. It's about hearing something familiar and then getting something extra out of it. It's not about hearing the story because you don't know how it ends. It's about hearing the story because you can fill in the gaps and you can fill in the gaps with what you want. Which is why I think we're going to see a new breed of storytelling with VR which both has a closer relationship to the player, but as a result has a more ambiguous relationship between the player and the story, and leaves a lot of space for the player in the story. I think that's the kind of experiences that we're going to start seeing.
[00:45:10.726] Kent Bye: And finally, what do you see as kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:45:17.070] Rob Morgan: So I said this last year and I believe it just as hard. I think that VR and AR are going to converge probably sooner than we would now think. I think that the point where you are able to select your own level of reality is the point that we're going to find ourselves in. Not before we know it, but before we're quite ready for it. And I think that in a few years time we're going to look back on the idea that VR and AR were separate technologies and separate disciplines. And instead we're going to look at the difference between getting up and going and making some popcorn and watching something on TV. If you want to flip into a channel where you're experiencing a completely opaque superimposition onto reality, what we would now call virtual reality, you can do that in bed. But if you need to get up and go to the loo, you simply flip back into a higher reality, lower simulation level or experience. So for me, the virtual realities that we're seeing now, you know, they are the children of flat screen gaming and they will continue to relate to that really strongly because that's what the audience will continue to want. There's nothing wrong with that. But I think over time we're going to see virtual reality evolve into something which is designed to fit around the user's life. So we might flip into a virtual reality experience if we're sitting on the sofa at the end of a hard day. But we can also dial it back, mix that choice of reality with another choice of reality, with another base level of reality which is theoretically the real one. But those distinctions are going to mean less and less. It might sound science fiction. But I believe we're already working with mixed reality because we're already augmenting the player. Whether you're in augmented reality or virtual reality, that's what you're doing. In terms of what that means for virtual reality experiences, well, I don't think it means we'll see more horror or less action. I think it's too early to make those genre decisions because I'm not a marketer or a demographic expert. But what I'm hoping for is that we'll see not just strictly contemplative experiences, but really conceptual experiences which are based on playing with the player's relationship to reality itself. That's what I really want to see and I think the technology is giving us a really amazing opportunity to make that kind of stuff and I'm excited.
[00:47:41.977] Kent Bye: Okay, great. Well, thanks again, Rob.
[00:47:44.159] Rob Morgan: Thanks for having me again. It's great to be back on.
[00:47:47.193] Kent Bye: And thank you for listening. If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash voicesofvr.
