Wolves in the Walls is a narrative VR piece based upon a Neil Gaiman novel that explores object interactions and cultivating a connection to a virtual character. It’s a project that originally started as a project within Oculus Story Studio, but was then transferred to Fable Studio, which is now known as Fable. Inspired by the Oculus Toybox demo, the team originally wanted to explore the secret life of objects, but then their focus shifted to cultivating intimacy and connection to virtual characters after seeing the immersive theater experiences of Sleep No More and Then She Fell in New York City.
They were so struck with the embodied interactions with the immersive theater characters in Then She Fell that they actually hired the production company Third Rail Projects to help with the embodied communication, body language design, and motion capture of the main virtual character of Wolves in the Walls. I had a chance to catch up with director Pete Billington and creative director Jess Shamash at their Sundance premiere last year in 2018 to talk about they combined insights from film, video game design, and immersive theater in the creation of Wolves in the Walls, which is now available.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye and welcome to the Voices of VR podcast. So at Sundance 2018, there's an experience called Wolves in the Walls, which originally started from Oculus Story Studio. And then once they folded, then they actually passed that over to Fable Studio with a lot of the same people from Oculus Story Studio who were able to continue and actually complete the first chapter of this project, which debuted at Sundance 2018. So, this was coming from a Neil Gaiman novel, and they actually collaborated with Third Whale Projects in order to do some immersive theater acting, and so I had a chance to talk to both the director and head of creative production, Pete Billington and Jess Shamosh, to talk about the process of creation that Wolves on the Walls, as well as some of their biggest insights for immersive storytelling. So we'll be covering all that and more on today's episode of the Voices of VR podcast. So this interview with Pete and Jess happened on Tuesday, January 23rd, 2018 at the Sundance Film Festival in Park City, Utah. So with that, let's go ahead and dive right in.
[00:01:16.246] Pete Billington: My name is Pete Billington. I'm the director of Wolves on the Walls and co-founder of Fable Studios.
[00:01:21.348] Jess Shamash: And my name is Jess Chemosh. I'm the producer on Wolves on the Walls. I'm also working as a creative producer, head of creative production at Fable Studio.
[00:01:29.422] Kent Bye: Great, so yeah, maybe you could tell me a bit about the backstory of Wolves on the Walls and how it came to be here at Sundance now.
[00:01:36.334] Pete Billington: It's actually one of the original properties that Oculus Story Studio was interested in exploring. completely appropriate for the medium, you know, a character-driven narrative, an old creepy house that made a lot of interesting sounds, just a very rich environment to sort of explore some of the unanswered questions. But it really consolidated around this first idea of Henry making eye contact with us. And about the same time we were getting prototype touch controllers and we knew that sometime in the near future we would have to start to solve for how to use our hands in a narrative experience. And so we really sort of doubled down after the first time we saw Toy Box. I went into this R&D facility at Facebook and a person gave me a couple of touch controllers and said they'd be right back and the next time I saw them they were a floating head and hands. We did a quick fist bump. And then they picked up a cube and handed it to me. And that was kind of a mind-blowing moment for me, because it was so natural and intuitive. I just grabbed it without thinking. I wasn't thinking about an interface. I had this complete freedom of movement. And we went through that amazing demo and did all these fun carnival-like things. But it was really that first object transaction that had me sitting in my car for about 30 minutes afterwards, thinking about how every aspect of storytelling, everything I had known, for the last 20 years was about to change. And so, you know, we kind of sprinted back up to the studio and we started working on this very early prototype of Lucy with a tin can telephone attached between, you know, two strings and she handed you one side of it, you know, people took it right away and then she would sort of probe the wall for a sound and you could listen spatially in your side of the can and hear what she was hearing in the wall. And we just got super excited as soon as we felt like this connection through a device with our hands to a character. And then it was really six months of, okay, what do we need to do to make this flow with narrative? And we made so many mistakes, you know, just trying to pursue object interaction and novelty. But it was this amazing time to really talk to as many people in the game industry as we could about how to make character connection, how to build a thin AI layer where a character can address you and make eye contact and locomote towards you. So we talked to Jordan Thomas from Bioshock, and we talked to Sean Vanneman from Firewatch, and just wanted to process as much of the game side of it as possible. Because Story Studio originally was comprised mostly of filmmakers, and we understood that we needed to smash those two worlds together, but that was sort of the first genesis of Wolves and how to adapt a source material for VR in that way.
[00:04:23.173] Jess Shamash: So I started at Surrey Studio a little bit after Pete did and I remember him working on that early prototype of probing the walls with the cans and we had these early prototypes of the touch controllers. There was only two in the studio and every single day they'd go missing because everyone wanted their hands on them. I stole them a few times I think. And no, we were just such a big fan of Neil Gaiman's work, so we were so excited. You know, Pete and I and Sashka and Kendall Cronkite and Goro Fujita, we were the early kind of skeleton crew on the project, and then other people from the team join. And we were such a fan of Neil Gaiman's work, and as Pete said, it was all about character and interacting with Lucy. And we wanted to give Lucy the complexity in VR that Neil's words give her on paper. And so, like he said, the development process was working and chatting to all these different experts from different disciplines and getting these people in a room and just seeing what we can do. And it was just... It was fun, it was, yeah, and we made a lot of mistakes, but I think, really, when we started to combine gaming and film and immersive theater, that's the sweet spot, and that's really what the experience is reflective of.
[00:05:38.378] Kent Bye: Yeah, and I'm curious to hear your perspectives on what components that you're taking from gaming, immersive theater, and film, and what you're leaving behind.
[00:05:47.895] Pete Billington: So, you know, our development process early on started with traditional film process. So character design the way that you would in an animated feature, script writing the way that you would approach a film, so beats and outlines, and then going into final draft and writing a script. But the whole time you have the gaming side of our team sort of saying, hey, you really need to be testing things, you need to be prototyping things, you can't lock a script without really experiencing something real time. And we sort of arrived at this concept We unfortunately called proto-viz, which is sort of a smash-up of prototyping and pre-visualization, trying to combine those two techniques. And one of the things we found is, in VR especially, you cannot just assume that something's going to work in its crudest form. You really need to take it to a certain level of polish. And so we would do these intervals we're calling rough assemblies or just to fully light a character, animate them, make them have eye contact, so we know that character connection is there. And the game side of that is all of the node infrastructure in Unreal and the blueprints to make her address you, to make her come to your location, to branch, whether you've Interacted with her or not. We never wanted to stop the flow of the story and so there's a huge aspect of just The heavy lifting of creating this character in Unreal Engine and and what we're taking from the game industry in that also things like what we learned from Sean on Firewatch is, you know, when to make the story go wide in terms of ranch, and when to keep it more narrow, and how to sort of curate the audience in a way that they feel they have free choice, but also that you're taking them for a ride, and that's a very delicate balance, which I think we're very much taking from game. You know, the film I mentioned, we're taking sort of the traditional design process and the storytelling component and character development components of that And then this last piece, which was really the key differentiator for Wolves, which was immersive theater and the subtlety of motion and the nuance and natural intuitive interactions where, you know, Lucy and the experience will hand you an object but won't ever wait for you. She won't even maybe say, here take this. She'll just subtly gesture in a way or make an eye dart to an object. And that's something that I've only experienced in immersive theater, just how naturalistic the interaction is. When we animate characters in film or in a feature animation, Often that emotion is over-the-top exaggerated because we are conditioned to want this sort of explosive entertainment format. But when you actually are present with another person, if someone behaved that way, it would feel incredibly unnatural. It's actually a new component of the Uncanny Valley in my mind because There's a fidelity of look and quality, but there's also a fidelity of behavior that has to be consistent with occupying a space and intimacy. And if you betray that in any way, you get pulled out of the experience immediately and feel uncomfortable, or you won't participate in a way that we want you to. You won't see the emotion in Lucy. You won't feel the world the way that she sees it. All of those things.
[00:09:00.500] Jess Shamash: Yeah, just to add to that, I like to describe this experience as curated intimacy. And so it's all about this bond with Lucy. And so all of our creative decisions were filtered through that lens. And so, you know, that then all of our Film background, game background, immersive theater background, any collaboration or creative decisions that we're moving forward with, we always ask ourselves, what brings us closer to this character? And so that's really how we made those decisions and how we weave all those various disciplines into this experience. And it's funny, our production schedule was such a mashup. All three disciplines, where we're using language, rough assembly, I came from Pixar, so that's what we used to call these kind of midway reviews, prototypes, proto-vis, stage rehearsals, kind of everything, but it was really fascinating. It was a fascinating experience and we learned so much.
[00:09:59.980] Kent Bye: I'm curious to hear each of your first initial experiences with immersive theater because, you know, I saw Sleep No More back in October of 2011. I just happened to see it, you know, in the first year it was open and so as I entered into the VR space I had sort of a bar or threshold of like the level of immersion that I had for this type of, you know, immersive storytelling of what was possible and VR just feels like it's, in so many ways, trying to catch up with what's being done in the immersive theater space. But at the same time, when it comes to branching narratives and player agency and interactivity, I think the gaming world actually is sort of pushing the bleeding edge in terms of how far you can take it in terms of your expression of your agency in terms of those branching narratives. And it's very kind of on rails and you're more of a ghost or a witness to a lot of these immersive theater experiences. Except for when I saw Then She Fell, I felt like, you know, in Sleep No More you have these, you know, like you can win the lottery and get a one-on-one interaction. And I felt like Sleep No More was kind of like completely architected to just focus on that level of those intimate one-on-one or one-on-three connections. such that you're able to have that, you know, instead of acting big, you're able to have these more subtle, nuanced, present interactions with the character. So I'm just curious to hear, you know, each of your, you know, experiences entering into the immersive theater space, and then sort of the things that you feel like are really applicable to VR.
[00:11:20.323] Jess Shamash: Yeah, so Pete and I, I think about a year ago, was it a year ago? I've lost track of time in VR. We went to see both Sleep No More and Then She Fell and we saw Sleep No More first and that was an amazing experience. I love to be immersed in experiences and I love 1920s jazz so I I had gone through and I spent the first hour in the experience but then I found my way back to the bar because there was this amazing singer there and I just wanted to live in that world. And it was interesting because when we were in the bar, you're talking about those winning the lottery tickets, we actually, that's where we had our one-on-one moment. And then the next night we saw Then She Fell and it was 90% that, that intimacy with another character. And what I was referring to earlier is that curated intimacy. They kind of guide you on this path. And then that's kind of when we knew that's what we wanted to do in Wolves. And we were so fortunate to be able to work with the third rail. They ended up choreographing the entirety of Wolves. We went to New York, and this was after we saw it, but a couple months later we met with them, we went through the treatment, and then a few months after that we went to New York, and with all of their performers, it was people from the experience, so I saw Alice. I saw the Mad Hatter, and they were actually cast in wolves. They were cast as wolves in our experience, and it was kind of like an out-of-body experience seeing them again. I think you and I, we talked a couple days ago, and you know, when you're at Then She Fell, you fall in love with these characters, and then you think you have these moments with them, but they have no idea who you are. And so, yeah, then we got to work with them and it was such an amazing, incredible experience. Just the thoughtfulness and how they prototype, how they prototype, how quickly going through rehearsals and seeing what's working and putting the audience, putting us in the audience's shoes for this rehearsal. So, yeah.
[00:13:26.328] Pete Billington: Yeah, I can't say how much of an impact Immersive Theatre, those first two experiences, you know, we went with an intention to explore sort of the secret life of objects because we thought, okay, we have to use our hands and maybe interaction with objects is about delivering backstory and exposition through, you know, exploration. And we'll open a drawer and we'll get this letter and we can read this letter and Sleep No More is that, you know. But it always came at the price of sort of this FOMO, what am I missing in the narrative? And that was a really important lesson. I've done Sleep No More twice now and the first time I resisted it because I wanted to explore all of the cool interactive objects. And I was unwilling to sort of chase the narrative in the way that you do in that experience. And I'm absolutely not going to spoil anything for either of these experiences. But the second time, when I let myself play the role that they wanted me to, I really understood it better. And that actually unlocked a little bit of how we think about the role of the audience now and how you get cast in a piece, especially in immersive theater. But the contrast of then going to Then She Fell the next day, as Jess mentioned, like you really do fall in love in the space of two hours, but you're also watching a dance. You know, there's always motion happening around you, and that was something that we wanted to capture in Wolves. You know, Lucy is persistent in the experience, and she is always moving. And we want to make sure that you're led around the space in the same way that we felt we were led in Then She Fell. I kind of talk about, you know, VR as this thing that you can do in your home that you don't understand why you should be doing it right now in the same way that when I first got an NES it meant that I could play Super Mario Brothers in my house and it was this amazing, like, I don't have to ask my mom for a dollar, and I can play all day. And I had something that I was trying to replicate, and now I had that device. And the same way a Blu-ray player or high-quality video in your house on a large screen sort of gets you the cinema experience in your home. And I feel like VR is getting us the immersive theater. experience in our home a little bit, just a little glance at it, but so few people have actually had the pleasure and opportunity to see immersive theater that everyone I talk to, you know, I say you have to go do these experiences because we walked out of Then She Fell saying that was the best piece of entertainment we have ever seen. How did they do it? And this is what we need to strive for. So that was a huge motivator for making this piece.
[00:16:00.276] Kent Bye: Yeah, to me, after I saw Vinci fell back in 2017, after I went to Tribeca, I had seen, you know, Supino Moore a number of years previous, but I was really struck by the level of intimacy that you can gain by looking into someone's face in their eyes and to see their full facial expression. It's like when you go into VR, it's a little frustrating because like any social VR experience you have, I can know that there's a certain bar or threshold of what it's like to be face-to-face with another person. And I'm like, yeah, but it's not going to be anything near like the real thing. But I think with the animated character, you actually start to do maybe more of a high-level symbolic representation of that eye contact and that body language. And so it sort of translates probably better than any other social VR experience I've had. to sort of like capture all that high-fidelity motion capture. And it's not like a real-time interaction. It's like I'm a ghost within a story that's unfolding. And so I'm not able to kind of like interact in real-time like in Then She Fell where I could sort of like, there's more of a dance there between the body language where I'm giving off cues and they're giving off cues. But I'm just really struck by how in film there's like a visual storytelling language that has a rhythm and all sorts of different methods of building dramatic tension. And I'm struck with how both in Immersive Theater and in VR, you're starting to use space and spatial language and symbolic language of body language and distance of how close you are to someone as well as all the different facial features. That's another level of storytelling to build dramatic tension, but a lot of that is actually happening at an unconscious level. when I watched Wolves on the Wall, I didn't say, oh yeah, they obviously used professional immersive theater actors to embody it. It was totally, I didn't even notice it. It was invisible, but yet it was able to sort of communicate at that subtle way, this kind of new language that's being, I think, kind of cultivated with these two experiences of immersive theater where there's not a lot of dialogue, and so they're forced to kind of use their bodies to communicate. And I think that you're kind of translating a lot of those lessons for how do you start to add a layer of narrative and story on top of the body language.
[00:18:05.307] Pete Billington: Yeah, we absolutely wanted all of that to be as invisible and natural as possible. If we ever felt like something was obvious, it broke presence, and so we would find a different way. When we talk to the immersive theater performers, they have a certain amount of improv. You know, there's a certain plan for, you know, this scenario, this scenario, but then there's always adaptation, and these are amazing performers, very intelligent people so they can adapt and maybe the barrier or challenge in VR is that you don't have that improbability really. We're not at a sophisticated level of AI yet where we can just sort of algorithmically respond to you in a natural way. So that means predictively like being intuitive and natural which is even more challenging. So under the hood, you know, With the burden of never wanting the narrative to stop, never sort of idling for you to do the right thing at the time, we want you to feel that you're always making a choice but never being penalized for a choice. That we have to sort of run through a hundred rehearsals of an interactive experience before we ever even start the motion capture process. We have to plan for every possible outcome and then hope that those also just feel like they're happening on the fly. You know, you can interrupt Lucy at any moment throughout the piece, you know, when you're doing something and she'll just, oh, you know. Not that. Do this instead, please. And then she'll resume right where you are. And you could have interrupted sort of organically at any point. And we always wanted to make that feel like, yeah, this is just like being with your friend when you were eight years old, hanging out in their room. What are the kinds of things you get up to as a kid? Putting you in that character.
[00:19:47.521] Jess Shamash: Yeah, making you feel that everything, you know, when you're eight years old, everything feels like an adventure. Anything is possible. And the little noise in the attic is a monster in the walls. And so putting you in Lucy's shoes, and that's what we did kind of by establishing you at her height. So you see the world through her eyes, through her emotional perspective, through her lens. And then also the way that she talks to you and she communicates to you, she addresses you, you heard it too, you did, you did. So you have some agency there, but the story can still go on. So yeah, for us, I think just putting you in that space, making you feel like you're a child again who has agency, that was important.
[00:20:32.298] Kent Bye: Yeah, I had a chance to talk to one of the lead actors of Vinci Fell at the Immersive Design Summit that was here recently, and the thing that he told me was that after doing over 900 performances, after every single performance there was something that he had never seen before, that someone had done. And I found that really striking, just from an experiential design perspective, like, how do you design for, like, basically an infinite potential of what humans are going to do? And to me, I see that there's like a couple of ways to kind of break it down. One is sort of like, from Chinese philosophy, the yang or the yin, whether or not they're going to be willing to take action or take no action. That's sort of like, either they want to just sort of be completely passive and receptive, or they're going to be willing to sort of engage within the experience. but within that there's sort of like different variations of like the active presence of being able to want to perhaps do more of a live-action roleplay and like really embody a character and really sort of like have a dialogue with somebody or somebody who wants to sort of either have social interactions or solve puzzles or have sort of more of like a mental stimulation with sort of a challenge of like make it more of a game in terms of like see how people react to something you know it might be more of a trollish behavior to kind of like try to disrupt the experience to see what happens just because you're curious as to how they're going to react. Or in terms of the more receptive, the embodied presence, it could be more of a kinesthetic experience, a sensory experience. And I know that within Then She Fell, there's a whole sensory component of taking different shots throughout the experience. But you have that experience to really focus on your body and your inner experience of being present within yourself. And then there's the sort of emotional presence component of the story and the narrative that's there, of you really sort of being engaged in that. So I see that there's sort of like different temperaments that people are gonna have in terms of like, they're maybe gonna have like a center of gravity going towards one of these different types of interactions and that as experiential designers, there's a little bit of like trying to adapt the story to kind of like, you've kind of phrased it twigging or to have these light branching to be able to not break that flow and still be able to continue the story.
[00:22:33.947] Pete Billington: Yeah, absolutely. And everything you mentioned, I think we found that to be true along the way. You know, there's at one point, part of Wolf's story as chapters two and three is dad, you know, is rehearsing tuba down in the basement. And one of our early prototypes, we were doing interaction and we had character animation. And then when we just added this subtle layer of dad playing tuba, like two floors down, it dropped you so much more into the space. And so every time you can plug another sensory port into the experience, whether that's haptics or just a general sense of spatialized audio or being able to pick an object up or having an emotional connection. Every layer of that bonds you even closer to what this character is experiencing because you can get closer to their emotional state. You alluded to it earlier with the use of space, but we are sort of coining this term of emotional lens or emotional POV because we are restricted in our vernacular. We don't have you know, a set of prime lenses that we can put on a camera. We can't even really control where the camera's dutching, or if we're zooming or not, because the audience is the camera, and we've talked about that for a long time in VR, but the attempt at solving that was to actually physically change the space to match her emotions. So throughout the piece, you know, Lucy will actually transform the space just based on her projection of what she's feeling, and we say that we always want you to see the world the way that she's feeling that world. just to sort of get in that emotional connection. So yeah, I can't wait for the day that we have this full tactile sensory, being able to eat and smell and all of that. But until then, we use all of the parlor tricks we can to get you to feel that way and make those connections.
[00:24:17.627] Jess Shamash: Yeah, just to add to that, that's exactly it. We did everything that we could think of by immersing you, for example, as Pete was saying, the tuba that plays in later chapters. And then you'll hear that tuba, and you're like, oh, when I was in the attic, that's what I was hearing. And then also, just to talk a little bit more about the emotional POV, again, you are not only seeing the world that Lucy feels the world, you're hearing it. So yeah.
[00:24:44.028] Kent Bye: I'm curious to hear what big lessons that you're taking from like interactive gaming and narrative gaming. Yeah, just curious to hear how narrative gaming has sort of influenced how that gets translated to VR and what works and what doesn't work.
[00:24:57.472] Pete Billington: Yeah, I think we had the opportunity to work with just some amazing people. We spoke with Doug Church and Jordan Thomas and Sean Vanaman, but we also had this influx of people working on the project that had worked at Telltale and Double Fine. And so there was just some DNA, you know, right there that had worked on a lot of narrative gaming and the moments of choice, whether those are subtle or intuitive in the case that For Wolves we really wanted to emphasize people having moments of choice where they felt like they had agency but also sort of keeping that more invisible than I think a narrative game. So in some ways Wolves is a response to narrative gaming to differentiate the format because that was Story Studios mission and now I think Fable has a slightly different mission but At the time, I think it was important to say gaming is already sort of established within VR. We know it's going to have a very successful lifespan and that is like the early adopters are mostly from the gaming community. And so with Wolves, we wanted to say, OK, what is it? that makes you feel conscious of choice in gaming and how do we make that more subtle and intuitive for this type of experience where we never make the audience feel like they're inadequate because an element of a game is that challenge component is that reward component and that dopamine hit and I think we wanted to feel everything that we did as an interactivity or a branch or a twig really was at the service of drawing you closer to this character and we had this goal, we had a lot of goals, but one of the goals was how fast can we get you to connect to her? You know, can we do that in 15 seconds? Because you have a limited amount of space to tell the story and we're asking a lot of our audience in terms of participation and so we needed a way to figure out First, how are we going to earn the audience, get their permission to participate? And then, how are we going to get you to go on this ride with her? And ultimately, how do we get you to stand in front of this little girl and protect her later on down the story?
[00:26:55.494] Jess Shamash: Yeah, and I will just say that, you know, on the team, our team was made up of so many different experts from these various places, from filming, from games, and one of the proudest moments for me on the project was, you know, we started off in different corners, and when everyone started speaking the same language, and we're like, this is what the project is about, and we came together, And Pete and I, we'd walk into a meeting, and maybe we were in an interview, and the people that came from games were talking about exactly what was passionate to us, and we were all just on the same page. And that's when we knew we had the secret sauce, when we all believed in what we were doing.
[00:27:36.447] Kent Bye: So what are some of the biggest open questions that are kind of driving your work forward now?
[00:27:44.678] Pete Billington: I think social is a huge component. Trying to figure out how to have a character like the one that we've just created manage not one person but two or three, how do they allow those people in the experience to contribute to the story themselves and get that improv quality into the piece. One of the things that we constantly ask ourselves is why are we doing this in VR? That was another question that we got asked from Third Rail when they started to participate. They were very jealous of what we were able to do because they're always constrained by the physical world and they are masters at transforming that and manipulating it, but we really have no restrictions. So I think, how do we continue to sort of exploit the limitless potential of virtual reality to make people feel more connected? When I experience immersive theater, I do it not just by myself, I do it in the context of being with other people, and that somehow changes the experience, and I'm really excited about that.
[00:28:52.712] Jess Shamash: Yeah, I think the biggest open question for me and what I'm excited about is what we were alluding to earlier about your senses and how we can further push that with haptics or whatever to really tell stories that immerse you layers deep. I think there's so many possibilities just being at Sundance, seeing some of the other experiences. I think that's kind of what Sundance is about in some ways for this community is just collaborating, collaborating and seeing what everyone else is up to. And it's been super inspiring for us. So for me, kind of exploring going deeper into your senses within a narrative.
[00:29:27.434] Kent Bye: Yeah, I think a big trend is the mixed reality to sort of have the passive haptic feedback. And I know that the Void is starting to do that a lot more, both in the Ghostbusters experience as well as the Star Wars experience. And then there's an experience here at Sundance called Hero, which has a lot of mixed reality components. So being able to actually reach out with your hands and touch a scene and have it sort of represented both in a virtual world to have the visual feedback of another reality but sort of have a different physical reality that you're touching but, you know, your brain kind of fuses together. So I think it's difficult to do that at scale. So this is, I guess, more in the context of this location-based entertainment and so I don't know if something like Wolves in the Walls would be it seems like you're going for scale rather than sort of like there's this trade-off between going for that high fidelity of haptics but then you start to have to think about well now you have to have sort of a physical infrastructure to be able to replicate some of this stuff and then you sort of lose that ability to have the flexibility to sort of transport you through all these different worlds so quickly you start to have some of the physical constraints that you would have with sort of the actually immersive theater productions.
[00:30:29.362] Pete Billington: It's kind of a fascinating loop because what you do is you find these constraints and then you build a rule around those constraints and then you break that rule to actually create an effect. So a good example is we felt like with Henry you had to be in his home for 30 seconds minimum before you could parse any sort of narrative because everyone was just gonna look around and with Wolves we immediately wanted to break that rule and just throw you into the narrative like right away and so you make adjustments you know visually or you do theatrical tricks to like draw your attention where you want the audience to look But I love the fact that we're rapidly coming up with these rule sets that are sacred for VR, and then immediately someone looks and is like, no, that's absolutely wrong. I'm going to go break that. And that's where we're at right now in the process. It's an awesome time to be there. We were very stubbornly trying to break a lot of the rules that got established in 2015, 2016 on Wolves and see if we could establish some of our own rules that then someone can turn around and break as well.
[00:31:32.569] Jess Shamash: Yeah, and I think another example of how we did break our own rules and rules is early prototypes. We had everything where you can touch and interact with objects, and that completely distracted you from that intimacy and connection with Lucy. And so now we only have objects that you can interact with that advance the story or bring you closer with Lucy.
[00:31:53.355] Kent Bye: Yeah, it's interesting. And finally, what do you each think is kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:32:04.022] Pete Billington: It's such a hard question. I think we all talk about Westworld or the holodeck and I think what I really want is to connect with things the way that my son does. He's nine years old and every morning I wake up to him out in the yard like having a completely fabricated reality that he's totally immersed in and writing his own story on the fly. And I think once we unlock the storyteller in everyone, and allow them to go on their own personal journeys and tell their own stories and their own worlds and not have it sort of this client transactional relationship of storytellers to audience. You know, there will always be authors but the more people that can sort of go on their own adventures I think that might be the ultimate goal and the fact that we could all do that potentially together and that's how we interact on a certain level. Yeah, I'm just all about getting people together and having common experience and this week has been that, you know, with our peers. We finally are now talking to everyone about what we're doing and we're sharing ideas and so if there's a future where we can all occupy a space and tell each other stories like we would around the campfire a thousand years ago, we've probably gotten to a really good place.
[00:33:20.343] Jess Shamash: For me, I'm a believer in magic. And I think virtual reality is magic. I think where it can go can be magic. And if we can bring that uniqueness, that specialness, that feeling that anything is possible, the whimsy, I think VR can do that. And we'll do that. And we're excited to be a part of it.
[00:33:41.035] Kent Bye: Awesome. Well, thank you so much for joining me today. Thank you. Really appreciate it.
[00:33:44.396] Jess Shamash: Thank you so much.
[00:33:46.239] Kent Bye: So that was Pete Billington. He's the director of Wolves in the Walls, as well as Jesh Shamash. She's the head of creative production. So I have a number of different takeaways about this interview is that first of all, well, just to have this sense of embodied presence with the character and the way that it was interact with me, it was some of the most sophisticated character animations and interactions that I've had within virtual reality so far. And there was subtle things that I didn't necessarily notice in the moment. until they really broke it down to say all the different things that they did. I mean, to some degree, a lot of the things that you react to in these types of experiences, you're kind of reacting to subconsciously. And that's a bit of the magic of what they've been able to figure out with some of these body language cues is that they'll hand you a camera and you'll just grab it and you're not think too much about it but there's actually like a huge art of reaching out and turning the other way and you know just kind of leaving your hand standing there and they've got to actually create different ways of moving the story forward even if you decide not to take the intended action that is trying to be communicated through the body language of the immersive theater actor. And just the way that they were able to move through the space and highlight different parts of the space and being able to create something that felt very dynamic where you're basically in a single location but it feels like you're being transported into many different locations and the different types of interactions that you have within the piece. Now, I only got to go through the piece one time. I didn't have an opportunity to really stress test all the different branches, but there seemed to be what they called little twigs, which is like these small little branches and options where there might be little forks in the experience based upon the different levels of engagement that you have. And I don't think that these different branches are actually changing the outcome of the story, but eventually they might be able to, where you're able to just create this natural interaction with a character. So I thought this was a great translation and just in talking to Alberto Dennis in the previous interview, just to hear how much goes into what it means to be able to be an immersive theater actor, to be present in the moment in this kind of yogic state. And it's basically like years and years of experience of trying to both cultivate the process of communicating body language, but to also be able to read body language and I don't think we're at the point where we're actually taking in all these subtle body language cues, but eventually I think that's going to want to happen within the context of these virtual and immersive virtual reality experiences. For me personally, I think that this type of data should be ephemeral and maybe have some real time responses, but I don't necessarily think that our emotional data and what was happening radiated from our bodies should be captured and recorded forever and ever. And I think That's a deeper ethical issue that I'll be getting into some other companies that are looking at how to get some more interactive engagement that's happening within the context of a VR experience. And I think it's totally fine to do that with some real-time processing of whatever is emergent in the moment. But as soon as you start to capture that and record it and kind of put it into my permanent record, then that's where I get a little bit more hesitant. And so for me, that's my own sense of moral intuition for these different thresholds for how we should treat data that's being radiated from our body. So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast, and if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listeners-supported podcast, and so I do rely upon your donations in order to continue to bring you this coverage. So, you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
