Tender Claws’ The Under Presents is a landmark experience in the evolution of immersive storytelling. It packs in so many narrative innovations, novel game mechanics, experimental VR interactions, and even includes live immersive theater actors in a four-month run starting on November 19th.
Tender Claws is probably most well-known for their critically-acclaimed Virtual Virtual Reality, which seamless blended open world exploration with a very well-written and humorous narrative. They’ve been working on The Under Presents for over two years in collaboration with immersive theater troupe of Piehole. They’ve cultivated an innovative fusion that’s part VR game, part narrative, part music platform, and part experimental playground for live immersive theater researching the question of what exactly the magical “live” aspects of intimate one-on-one interactions in a social VR space.
The Under Presents experiments with time loops both a gameplay mechanic, but also as a narrative conceit in order to explore the story of eight characters who set off on a ship. The experience has a masterful on-boarding process that introduces you to this world and how it stretches across space and time, the unique locomotion system that warps space and time, as well as some of the methods for how to navigate the narrative portions of the experience. It’s probably best to just play through the experience before reading too much about it, and so feel free to pause and just check it out if you already know you’re interested in discovering everything for yourself. I played through the first third of the experience during a press review period and so there’s a lot about the narrative and story structure that I’m still actively exploring now that the finalized build is available.
What’s really quite unique about The Under Presents is that for the next four months, there will be live immersive theater actors who will be roaming around the multiplayer areas of the experience. Neal Stephenson’s Diamond Age sci-fi novel dreamed of reactive actors or “ractors” who would be paid to roam around virtual worlds acting out interactive stories, and so The Under Presents may be the first consumer VR experience that’s employing a set of actors for an initial four-month run of performances that run around the clock. Tender Claws producer Tanya Leal Soto said that they have the ability to monitor the number of users who are roaming around the multiplayer areas in order to help with the capacity planning for populating the world with these live actors.
The actors will be primarily colliding with the users of The Under Presents in what are known as “one-on-ones” within the immersive theater world. These one-on-one interactions with actors are usually randomly sprinkled through immersive theater shows like Sleep No More, but receiving one is rare and kind of like winning the lottery. There’s usually hundreds of audience members in an immersive theater show like Sleep No More, and so there’s only a tiny handful of intimate interactions that happen, and they’re usually up to the actor’s discretion to chose who will receive a highly-customized experience.
Other immersive theater shows like Then She Feel architected their show in order to optimize for these intimate one-on-one interactions. However, these shows only have a throughput of 15 people at a time for this two-hour experience. Because the The Under Presents doesn’t have any space constraints, then they’re able to spin off virtual instances where they’ll have immersive theater actors roaming around who are based initially in New York City and Los Angeles. Time will tell how they deal with scaling this out and how probable it will be that you will run into a live actor, but they’re trying to recreate these intimate interactions in a virtual space.
I was able to have four different one-on-one interactions with live theater actors in my two different press review periods at Sundance and this past month, and there’s definitely a unique quality of having a live interaction. As a user in The Under Presents, you can’t speak and so you’ll be forced to use body language and gestures in order to communicate. Director Samantha Gorman hopes that this will create different elements of emergent play with users, and she commented how it’s really quite amazing how much of someone’s temperamental energy and character can still be transmitted into the virtual space. There are a number of recorded interactions in the experience, and so the live interactions with actors stand out in how there is an emergent conversation that can unfold where there’s an asymmetry of information loss much like playing a game of charades.
Because the live actor can be very specific in how they react to muted users, then it creates a very special live moment where you get to be taken to a secret place, given more context about the world, and perhaps even taught a few tricks of ritual magic. It was immediately obvious to me when I was interacting with a live human and not artificial intelligence. We’re still quite a ways away from artificial general intelligence that could pass this type of Turing test of a live interaction, and so Gorman will be researching these interactions as part of her Ph.D. thesis that she’s in the process of actively researching writing with this project. There’s a lot more insights about emergent conversations and emergent play that she’s sure to find, patterns of user temperament, as well as the component parts for what exactly makes a live theatrical moment in VR so magical.
I had a chance to talk with Tender Claws producer Tanya Leal Soto as well as the co-director of Tender Claws Samantha Gorman, who also conceived of, directed, wrote, and directed the live immersive theater actors. We talk about the inspiration for The Under Presents, their collaborative process, and how she architected the interactive story.
Gorman said that she had surreal sci-fi / horror writer Brian Evenson develop the initial treatment, but she found that she needed to add an additional spatial treatment that determined how stories unfolded in parallel, but also how the story of the space unfolded. They developed their own motion capture solution that allowed them to do on-the-fly motion capture that was extremely efficient in being able to contain somewhere between 12-16 hours of motion capture data for entire experience. There are eight characters that each have their own storylines, and Gorman said that there are different stories and endings that are unlocked as you see a certain percentage of each of the character’s stories. Gorman also said that they developed their own set of customized tools in order to visualize and architect the parallel story lines that were unfolding across the ship as the coordination and timing was both very interdependent and multi-faceted.
Piehole was also very critical in helping to fill in the gaps for the characters in this experience. They would often improvise character traits, but also help to flesh out the characters by having different actors embody the avatars as they walked through the virtual sets. Stories are usually produced in a pretty linear fashion, but The Under Presents would take a highly iterative approach where acting would happen in virtual scenes with roughly sketched and temporary virtual props, and then the final art product was produced based upon those recorded motion captured interactions. So they were blending in the traditional waterfall approaches like the story and character treatments with a much more iterative design approach with improvised acting and bottom-up exploration. This would be fed back into the script, and then iterated on through multiple passes.
Soto & Gorman also said that Piehole was instrumental in recruiting many of the musical performances from the New York City artist scene who are featured on the stage, which serves as a centralized hub and multiplayer space. They were able to capture the essence of a live musical performance while being able to leverage the affordances of virtual reality to have a much more theatrical & surreal production than would be normally possible given the budgets of a indie musician. I found myself captivated in watching a number of performances while other users were roaming around performing ritual magic transmutations on interactable objects.
The Under Presents really does sit at the cross section of an indie game, experimental narrative storytelling, VR experience, and art piece that incorporates many influences from the theater world. Gorman has a background in theater and seemed happy to get back to her theater roots to explore how VR is able to create many site-specific theatrical interactions. Ordinarily there would be a lot of down time when following around characters who aren’t doing much in this type of immersive theater piece, and they actually have the characters blink in and out of existence in order to minimize dead time and make editing the narrative a lot easier as well. There’s a Matrix-glitch mechanic which actually helps you hone in on the narrative moments where something interesting is happening either through a monologue or dialogue with the actors.
Gorman also said that she was taking inspiration from other indie games like Braid for navigating space and time, the more passive narrative VR experience Invisible Hours, emergent play mechanics from Journey, the French theorist Guy Debord’s concept of dérive where “participants drop their everyday relations and ‘let themselves be drawn by the attractions of the terrain and the encounters they find there.'” They also took narrative inspiration from the sci-fi novel The Invention of Morel, which inspired the story that involves time loops. Gorman said that she is using virtual spaces to explore narrative structures that go way beyond simple branching narratives, and that virtual reality allows her to play with time in a narrative that goes beyond what any other medium can provide. There are many levels in which they’re exploring the bounds of time and space.
There are many puzzles and mysteries contained within the underlying structure of this experience that are a joy to discover. There’s also so many innovations when it comes to VR locomotion, motion capture, live immersive theater actors, an entire behind-the-scenes control schema for live actors to seamless navigate and perform actions in virtual spaces, scrubbing through a narrative collaborative and emergent play, and pushing the boundaries of what’s possible with narrative structures once you’re able to navigate through space and time. If you’re interested in tracking the evolution of storytelling and interactions within virtual reality, then The Under Presents is a must-watch experience that I suspect will continue to grow and evolve over time as a community forms and the many hidden secrets continue to be discovered. The Under Presents is is available for the Oculus Quest for $19.99, and was released on Tuesday November 19th, 2019.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So on Tuesday, November 19th, 2019, is the release of The Under Presents by Tanner Claus and Pi Hull. I had a chance to see The Under Presents at Sundance 2019, and it's this experiment using immersive theater actors, live immersive theater actors, while you're in this game world. So, people may be familiar with Tanner Claus's previous work, it's Virtual Virtual Reality, which is one of the best VR narratives that's out there. Actually, it's kind of a combination of a narrative game, open world exploration, really exploring this cross-section of the narrative game and immersive experience. And just the same, The Under Presents is exploring this cross-section, but bringing in even more theater components by having live immersive theater actors in certain sections of the game. Now, the entire game isn't all just immersive theater actors. There's an entire story world and a game that is also baked into there as well. So it's actually one of the experiences that I'm really, really excited to see what happens with it, because for the first four months of the release of this experience, they are gonna have these live immersive theater actors in the experience experimenting with what's it mean to have this live component of a VR experience. So I highly recommend people check this out. I think there's so many different narrative innovations and experiments and just great writing and storytelling and finding new ways of exploring narratives within virtual reality. We're talking to the co-director of Tender Claws, Samantha Gorman. She's also a writer and director on the piece, as well as Tanya Leal-Soto. She's a producer for Tender Claws and helped bring all of this project together. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Tanya and Samantha happened on Friday, November 1st, 2019, while they were in Los Angeles and I was in Portland, Oregon. So with that, let's go ahead and dive right in.
[00:02:07.238] Samantha Gorman: Hi, my name is Tania Leal Soto. I am currently the producer for Tender Claws. We have been working for the past two years in this experience called The Under Presents. It blends virtual reality and immersive theater. It's been kind of a crazy project. We've done motion capture and we have actors who can join the experience live.
[00:02:32.290] Tanya Leal Soto: Hi, my name is Samantha Gorman. I'm the co-director of Tundra Claws, which is a studio that specializes on the cusp of game art and experience, especially with a narrative element. So I am one of the people who design, create the main project, the idea for the project, lead the writing, directing, and various, in this particular case as well, we're working with the live actors who you'll see throughout the game.
[00:02:57.917] Kent Bye: Great. So I had a chance to do the preview at Sundance, and then I've had a chance to up to this point to get through probably about a third of it. And so I haven't been able to play through the entire experience, but maybe you could talk about the evolution of this project because you did the virtual virtual reality. You've been in this space for a while. So maybe you could talk about, about the Genesis and the catalyst of this specific experience.
[00:03:20.539] Tanya Leal Soto: Sure, it kind of came out, so Virtual Reality is part of our, I guess you could say almost trilogy of these sort of narrative, parodies, comedy type works where the game in itself or the product that is made is also sort of commenting on the whole industry and the technology that's being used, that's in play. Those games are some of the first I've really written dialogue for and my original background was in more Theater and drama so I wanted to return to that background to do our next work and like take a little bit of break in that series, but essentially there's an experiential theater company called piehole and We've known them for a while we've kind of operated in similar circles and we've admired their work and feel like there has been a lot of lip service and talking about the intersection of virtual reality and immersive theater and But for those of us who are from a theater background and have grown up in this awareness of either other live performance traditions or the awareness of traditions of like devised theater, and you know, like various other ways to play with the audience or location based or like immersive theaters where like, you know, audience can kind of travel through a narrative. we felt like we wanted to actually put a lot of that lip service into practice and create something where we could showcase these different types of theater in immersive entertainment in the way that like a lot of people had been making projections about or talking about the possibilities. And when I began working in this VR cave in 2002, which it's not a head mounted display. So it's a very different type of VR space where it's a room where the stereo graphics are kind of back projected and draw to one person's perspective. A lot of the things that were done in those spaces, thinking about when people think about VR at first, it's like, okay, if you're looking ahead and you like have an audience member, like, what are they clicking on? What are they looking on directly in front of you? But over time, for the next six years, I began to work with a dancer in that space and thought about VR as a more actually inherently theatrical space, especially when you're composing narrative or you're creating these different dangling aspects like weenies to attract attention to different audience members, where it's definitely closer to theater in the round. So we wanted to create a project that used narrative and storytelling in more of a theater in the round type of way. And we knew that Piehole was the company, you know, beyond the lip service that we could work with to actually really pull this off. So it was more of like our partnership than like the project emerged out of. the partnership and the desire to do the project. And we originally had wanted to play with the trope of a cruise ship and the idea of like the weird performances that happen on a cruise ship. And you know, what was it like for those characters backstage? And that story is definitely a departure from the story it ended up with, but it is definitely the origin of the desire to create this narrative.
[00:06:17.880] Kent Bye: Nice. Yeah. And maybe Tanya, you could give me a bit more context to see your background and your journey into this project.
[00:06:25.398] Samantha Gorman: I have a background in film, I'm a filmmaker, and I am also a puppeteer, which came in very handy for this project. I am the producer for the project, so basically I am the one running around making sure that everybody knows what's happening in every single area. This project in particular has been challenging because Pie Hall is in New York. And so a lot of the first stage of the project was having a team in New York and a team in Los Angeles. But it became really interesting, especially in the VR sense, I would like to say, because there was a point before the previous in Sundance that we were basically rehearsing in VR. So we would join this network space in the under and have coordinate rehearsals. And some of us didn't see each other till Sundance.
[00:07:16.485] Kent Bye: I had a chance to see the first preview at Sundance, and so the story is a very particular story world. So I'm wondering if you could just kind of introduce this world that we're walking into. If people have not tried it at all, like what would you want people to know about this world?
[00:07:33.371] Tanya Leal Soto: Yeah, the Sundance context is interesting because in the Sundance context, just for a little bit of background, is we did mostly the live performers and the stage acts. So we showcased the world as a platform for live actors and theater groups to inhabit, which is actually only a fraction of what the real project actually is. So, you know, a lot of people, I think, will be surprised when the project comes out to find that it is a fully fleshed out game where there's many hours of content and there is a narrative backbone to this game. And in the game, the narrative backbone is essentially this kind of cohesion of these two worlds. One is this sort of, I guess we kind of describe it as almost a metaphysical world, like a vaudeville stage outside time and space, where there's different acts that are kind of pulled from these different time periods that gather and perform for the audience. And at the nexus of this time mechanic and this time rip, it's all personified into this one character who is the omnipresent god of the space called the MC. And he's the one who selects the acts and is the proprietor of the establishment, so to speak. And that is kind of a framing or a hub world for the set for where a lot of these events take place. And one of the main events is at this point, there is a research vessel that is kind of an ill-fated research vessel that is doomed to become eventually trapped in Arctic ice and supplies dwindle and run out over time. And he has curated this act that has become stuck in a time loop as something that happens on the stage. So it's almost like a kind of offshoot of the theater where you need like a special ticket. It's like the VIP experience. You need a special ticket to be able to enter this world. and in the world it actually unfolds into multiple hours of content and there's eight characters involved and the various endings but there's elements of what the under is and this idea of time that trickles into the story and reflects back to what the main like stage in the hub world is and there's characters in both places that sort of can acknowledge each other there's a certain like ending that you only get to where you understand the world more if you save enough of the characters on the ship It's a very kind of intricately plotted narrative where in the way that Tender Claws often plots narrative where it's not necessarily like a linear distribution of content, but there's different seeds of kernels of these narrative monologues or moments that will happen with the MC that tells you more about the world. based on percentages or proportions of things you've unlocked or explored.
[00:10:06.811] Kent Bye: Interesting. Yeah, I know that I got through the first act and then into the second act. And so for me, what was really striking was the ability to locum out around the story through the space. It reminded me of something like Sleep No More, because Sleep No More is an immersive theater where it's unfolding over time. But probably even more so than Sleep No More, there's a piece that was by Speakeasy Society in San Francisco, which was like 39 different storylines unfolding over three hours. I think it was like 19 or 20 hours worth of content and dialogue. So something similar where thinking more about the story of individual actors and their relationship to each other. and how they're expanding across this space. So I was wondering if you could maybe talk a bit about how you architect something like this. How do you plan out so many of those different stories and take into account the character and character development? And then if you're using a traditional three-act structure to try to like have this unfold over time, or what different types of models and frameworks are used? I know you gave a talk with Tanya at the Immersive Design Summit, talking a little bit about your design process, but I'm just curious to hear a little bit more about How did you architect this story?
[00:11:16.796] Tanya Leal Soto: Yeah, so there's an interesting thing. I guess one of the things that like relates to the genesis of it is this idea of time. And I don't think I did say this, but that is actually important to that, not only the narrative, but also we often do these things with both the narrative and the mechanics where they complement each other or they kind of bleed over. So the idea of time does come into play as like a central mechanic for things you can do aboard the ship and ways that you as a player have to work with yourself. within these different iterations of time. So this idea of wanting to do something that was kind of how you move through space and playing with what is the nature of time and space in this VR component. And when I'm talking about space, I'm referencing some of the locomotion that Kent is addressing, that there'll be more video and I'm interested to see how people respond to it. And so those were sort of like the genesis of like a wider metaphysical idea. And then that drilled down into this concept of like, how do you set characters and dramatic scenes against each other in a way that's like simultaneous, where you can both see everything that's happening, but also cherry pick and look around at very specific things. To architect the actual story is a little bit of a Each story is different depending on the project and each approach necessitates a difference based on the technology you're exploring and the affordance and constraints of that technology. In this particular case, we had an idea of these characters that I worked with Brian Evanson who was a kind of a well-known like a surreal fiction horror writer and he created the first treatment of the script and we kind of worked on some of the you know character development together and you know create the initial characters and then wrote this initial treatment but he's super amazing. There's a difference I think for people who are trained to write in interactive media who are primed to think spatially about how like kind of narratives have to overlap So we we discovered and I discovered that like I had to go back and do a treatment over that and even add in scenes and fill in gaps and kind of do a lot of like the architecturing or the structure or to make sure a moment in like act one that might play out in the basement of a ship than has a result that plays out in Act 3 somewhere. So there was a lot of re-piecing, but in that re-piecing process, we also worked with one of our devs and we had this vision for a narrative tool where essentially, and most of this was actually part of the process where I would write a script in this text editing environment called Visual Studio Code and then be able to execute it. And when it executed it, I would get essential HTML interface of different timelines and different plots of the character. However, that wasn't in two dimensions. It wasn't enough. I had to keep a lot of the time code in my head to be able to lay it out. And just in those two dimensions, it wasn't enough to really fully fill in a lot of those gaps or to generate a lot of the content. Because you don't see the walking. You don't see the downtime. You don't see a lot of the flexibilities or the contours of the characters. So our next step was thinking about, and we actually had the ability to, at the beginning, trigger the script into a Unity animatic of characters moving through space. And we had just started to use that and work with that tool as a paradigm to be able to see all the scenes unfolding when we sat down to do the table read to record the VO. And that was the script was locked. So it was basically like only very little out of that tool, which I think, you know, if we had proceeded to continue to flesh that tool out with the implications for that power, of that tool are really, you know, significant. But a lot of it was fortunately, like, the experience of, like, designing and writing for space, mentally mapping things out. And this is where one of Peihol's role that's not immediately apparent came in is the table read. It wasn't just a table read, but it was also part of devised theater. So a lot of those gaps in characters were done partially through improv and partially through exercise. And, like, everybody in the room walking as one character and then talking. And seeing what emerged from that, and then those were recorded and partially bled back into the script. So I wove together this thing based on devised theatre, based on my own treatment, based on Brian's treatment. And then we stepped through and recorded it. And like, you know, there are missing beats. However, those missing beats were partially designed so that whenever you're watching people in real life, it's not like constant dramatic action. So you may see someone puttering around and like humming to themselves. And while that's from a video game point of view, you usually don't see NPCs like doing that for extended period of time. For this type of game, it's probably necessary to add texture to like the elements of the character and make them feel like they're inhabiting the space as like one does when they think they're alone. If that makes sense.
[00:16:03.030] Kent Bye: Yeah, it's super impressive, the clockwork nature of it. And I really liked the ways in which you're sort of indicating that there's like a glitch in the matrix or whatnot that directs you to a specific point that allows you to see a scene that's about to unfold, just how deliberate you were and be able to turn it into this puzzle element so that you could find these scenes and then you are rewarded by this narrative interaction, either by a character exposition or an interaction that happens. And I was curious if Tanya, you could talk a little bit about from your perspective of a producer, because this is obviously a lot of moving parts here in terms of trying to get all of this together and coordinated. And Samantha just walked through the degree to which she had to do this high level of architecture, very complicated. So how to actually like translate that into the actual pragmatic actors, motion capture and everything else to actually make it happen.
[00:16:57.594] Samantha Gorman: Pragmatically, it was magical. I guess we should say that the motion capture that we used wasn't like a full-on expensive motion capture suit in a studio. It was a rig that we also created for this project. So what we ended up doing was we selected by acts the scenes that we were going to motion capture. And we went to New York and we did those scenes. However, as Samantha was saying, some of the walking and some of the more idle scenes was things that we sort of waited until we started assembling everything more in a, let's call it editing, right? so that we could time right, how long does each walk take depending on the boat. To me, it was really interesting the way this project has developed because it's working hand by hand with what the actors are doing, how they're developing their characters. And I say this because everything they do, the walks, how fast or how slow they talk, changes the timing. And that translates into all the artists, the 3D artists that are building the ship and building the sceneries and building even the props, you know, like it's a back and forth of like creating what the actors are doing or the actors responding to something new that got created by the artists. And I think we we managed pretty well to be able to go back and forth and respond to what each team was doing. To me, that was very, very interesting.
[00:18:28.526] Kent Bye: And so just to clarify, it sounds like you're using all of the actors that are in these more cinematic scenes. Are all those actors sourced from Piehole as their immersive theater troupe, or did you have to bring in other actors as well?
[00:18:41.874] Tanya Leal Soto: Yeah, so the majority of them are Piehole as well. Some of them who were actually devising some of their own, you know, dialogue in the Vice Theater, others, you know, who weren't. But they also brought in people that they knew. They knew some mimes. They also helped to cast the VO. So in many cases, the people you see acting are also their own voices, but in other cases, it's separate. Like the chef, the person who was the physical embodiment of the chef was not the VO for the chef. So there's that component to it. However, what I think was interesting about the tools, because there's a whole side of this project we could talk about forever, which was the tools, but we actually, Tender Claws made our own timeline and animation abilities to edit and create those animations live. along with like the basic rig that we did so it was both difficult because you know it was uh you know running on our own tools and also both really generative and that it led a lot of flexibility so we could set it up almost anywhere and you know we have hours of mocap that we wouldn't be able to do in it as an indie studio unless we created our own like system and we're like creative about it and what Tanya said is actually really true where we have actors come in they'll be like, okay, well, we need would call up the art team and we're like, okay, well, we need a sofa over here, you know, can we send out a sofa and then we'd like have to like, create an outline and create essentially a physical representation of that in boxes. And then like the actors would like engage with the box sofa and then the art team would model that based on the actor engagement or vice versa. Like sometimes like the art team would be like okay and now there's these test tubes over here and then I would be there to like write con okay this character has downtime the test tubes over here mean x and like these are the set pieces how does the actor engage or improv with these set pieces and then we would record that and edit it live.
[00:20:33.212] Kent Bye: Yeah, the thing I was really impressed with was just how much content is in this experience, not only from like a performance on the stage, but also just like the amount of acting. Do you have a sense if you were to like lay back to back on end, and if you were to watch it as a linear piece, how many hours of content is, is included in the under presents or what you're aiming for?
[00:20:56.162] Tanya Leal Soto: People ask, yeah, it's hard because it's like the depth, it's more of like drilling down in the layers, but like, yeah, Tonya and I are laughing. So the ship itself, I think, has eight characters, each has roughly an hour and a half of constant mocap. And then there's pre-recorded acts, and those are at least an hour of constant content. Tonya, I know I'm forgetting other things.
[00:21:16.888] Samantha Gorman: There is this thing that I don't want to spoil for Kent if he hasn't played it, but each character has what we're calling memory labyrinths. So the way to save them is finding their memory labyrinth and you'll get sort of pulled into this other world which is a memory of the character. And they're puzzles and they also work in a loop, but that is another probably like 30 minutes per character. Oh my gosh. Uh, but you can spend there just going into, yeah.
[00:21:44.421] Kent Bye: Wow. So yeah, that's probably around 12 to 16 hours at least somewhere in that range. So, and I looked at the size of the download, you know, if you were to do volumetric capture of that, that would be at least 20 times larger. So you've found a way to do an art style on top of the motion capture to be able to get a lot of content in there. As well as the live interactive immersive theater elements, which is a whole other element, which is like Piehole going in there at certain times and then having these immersive theater interactions. But 12 to 16 hours, that's a lot of content to put into a single experience.
[00:22:19.241] Samantha Gorman: Yeah. Yeah. And even like right now, something that we're working right now is we're training some actors because we're having a group of actors from LA, which it's just started and it's going really well. But Samantha's working with them to like develop their own characters. We're going to be doing the one on ones or other shows on stage. And yeah, that probably should also count as content, right?
[00:22:43.295] Tanya Leal Soto: There's also a hidden multiplayer minigame as well, that there's a whole layer that can be discovered in the under just between players of ways to engage with the space and create something we're calling ritual magic, which enables opportunities for emergent play.
[00:22:59.342] Kent Bye: So I did this experience at Sundance and then I did it again here. So I think I had three or four immersive theater collisions at Sundance. I was like just trying to get as many as I could. I went through twice and I had just one just today. Shannon had given me some hours that said, here's some times when the actors are going to be in these spaces. And so I targeted it to be able to be sure that I played through the introduction stuff so I could get to the part where I could actually interact with the different theater actors. And so. Well, how are you planning on doing this? Cause obviously you can't have the actors in there forever. And so is this going to be a theater run where you're saying, okay, from this launch date until this time, we're going to have enough resources to be able to pay these actors to be there live. And then are there going to be times that people that you're giving to say they're going to be more likely, or is it going to be more of like, you just happened to run into them without having any more additional context or information? Just curious how you start to handle that element of the logistics and informing the audience of this. And then if people want to sort of increase their chances or try to specifically have some of these one-on-one experiences.
[00:24:09.735] Samantha Gorman: So right now, we are going to have actors and staff for the first four months. They're going to be there almost throughout the day. We do have a system where we can monitor how many people are playing the game and how many people are in the multiplayer areas so that we can send an actor at that time. And also we can say, oh, right now, the first four months, and eventually, I don't know, in six, seven months, be like, oh, we're going to have a new run with different actors. That's our ideal scenario.
[00:24:34.933] Kent Bye: Yeah, the thing that was striking to me was all of these secret spaces that you get introduced to. There's a big part of the architecture of the space where you may discover on your own, but there's probably some stuff that you could only get to if you run into some of these different interactions. It gave me this feeling of getting introduced to like these secret rooms that I would have no idea how to access myself. But maybe you could just talk a bit about your inspiration for that, being at the right place and having those one-on-one interactions and then being taken to this special place.
[00:25:06.928] Samantha Gorman: I think, and I'll let Samantha after this talk about it from more of the story area, but part of something we're trying to do too is let the live actors teach the players some of how to get to some of these areas. But the goal is also to let player to player also start teaching this idea of the spells, because some of the spells will help you go to one of these secret locations and you need for some of them, you need a certain amount of people. But the general idea is, yes, like the actors will teach some of the players how to access these places or create new spells. And then we are hoping that player to player, this is going to continue to be passed on. And Samantha, we want to talk on the story side.
[00:25:50.142] Tanya Leal Soto: Yeah, I think like the general feeling we're going for is like, you know, situationists, right, like kind of serendipitous encounters, like the, I'm blanking out on the theorist, the actual term, but like the idea of the art of wandering is, you know, part of it. However, there are a few spaces and like, this is something that we're doing in particular as we train. One of the things I want actors to do in a way that's not necessarily like over-the-top or really heavy-handed, but is to layer on exposition about the world and be one of the main things that bridges the gap of what those spaces are with the rest of the under. And there's elements in those spaces that can only be accessed with certain characters because it is part of the narrative or the fabric of what the under is. you know there are some things where we're kind of adding in but basically previous acts so like it gives the ship context in terms of things that have happened there in the past and set pieces that have happened there in the past and those are probably married to specific types of characters which are one of the things that we are working with the actors in LA this week are to like inhabit these spaces and think about what they are and what their relation to them is as an actor
[00:26:59.673] Kent Bye: You had said, were you trying to think of the term actual happenings? I don't know if that was it.
[00:27:04.096] Tanya Leal Soto: Uh, happenings, but it's a term that's like the wandering, the city.
[00:27:07.939] Kent Bye: Like collisions or serendipitous collisions.
[00:27:10.441] Tanya Leal Soto: Yeah. And there's a French dude, like I totally should know this, but there is a, like a French theorist who talks about like the art of wandering and like, there's a certain like term for it. But anyway, so that was, you know, like a main inspiration that I'm, you know, blanking on at the moment.
[00:27:26.852] Kent Bye: Yeah, I think of serendipitous collisions or synchronicities or you fall into something. And I really like the ways that you've set that up. And I had a collision with one of the actors and had one of these one-on-one experiences. And what was really striking to me, because I had a number of different collisions at Sundance, and the first time I had it, I was like play acting and speaking all these lines and of course discovered at some point that they can't hear me at all. And so then this time I was much more aware of trying to use my entire body to communicate, which was fun because there's somebody who's talking to me and then I have to be able to communicate back. And there was one point when this actor had asked me a very specific detailed question. I had an idea. And then I was like, Oh no, how do I translate this really complicated idea into like charades? And I was kind of unprepared to be able to do that, but found myself wanting to develop my own language that. would be contextual with whatever objects were around me to be able to then point and start to do different interactions to be able to communicate with other people in this experience.
[00:28:32.842] Tanya Leal Soto: Yeah, no, exactly. Those are the types of things we refer to as like emergent play and it's sort of like it's asking you to perform as well as a character and the ways that we try to like inhabit the world also with those props so that there is a part of the performance which is using the environment and using your gestural, your body language. It's interesting because you can see, even though the characters are simple, there are energies and feelings of people that get across. Like we knew actually, Kent, we knew it was you. We're just like, oh, that's Kent. And, like, it's hard to explain how and why that happens but there's just so much with, like, the gestural interface, the way that people decide to respond. For instance, you were- you were very different than we had someone else come later, completely different energies. You were definitely- I think you were in the morning and you were very interested in, like, you encounter the cat but you were interested in, like, wandering and looking at the spaces she brought you which was a very different interaction that the other person had which was really just full attention on her continuously. So it's like interesting even in that limited gestural vocabulary that there's a lot of information you can still glean and like actors are surprised too when they get into that space. But the thing we purposely limited the audio and people are confused about why and there's like many hosts of reasons but it was like actually primarily a creative decision in order to like foster these methods of communication and emergent play as well.
[00:29:59.813] Samantha Gorman: You're right in saying, Samantha, that it's really interesting how you can recognize people during Sundance. A lot of the people who were coming out, and that was like three at a time, they were like, oh, yeah, I know who you were. You were this guy who kept on doing this on the stairs. And it was all based on the body movement, even though it sounds strange because it feels limited. But once you see somebody as one of the players moving, you are able to figure out if you know that person.
[00:30:28.593] Kent Bye: Yeah, and the other thing that brings to mind, because I know you were, we were talking at Sundance where you had developed a whole system for the actors to be able to do specific actions, but you didn't want to like, you know, in VR chat, you can tell when people like pull up a menu and they start to point with their fingers, you know, and like you wanted to make it sort of more embodied or more either gesture based or allow the actor to still be in the motion in their acting space, but had to create a whole layer of controls For those live immersive theater actors to be able to navigate and pull up different objects to teleport. And if they need to get away from somebody or put them in a cage, like maybe you talk about that iterative process of how you created the actual controls for the immersive theater actors and all the different considerations you had to do in order to optimize it, to create this kind of seamless gestural interfaces for them to be able to do all these extra things as an actor.
[00:31:23.524] Tanya Leal Soto: I think that's something even in actor training, there are advanced skills in terms of like layering people up to be comfortable with the control system. It's interesting because we're getting actors from a variety of backgrounds and there are some people who are gamers. And of course it will come more natural to them, but that doesn't necessarily mean that people who it will take a little bit longer. For instance, we had one actress who was really strong and it took her a little bit longer, but once she got the controls, like she had a fluidity in the space, you know, that was a little bit like different. So to start, we did use some actors who were a little bit more, had some sort of joystick, you know, or like controller experience. And we sent them an iteration of the UI, and then they would give us notes, and we'd go back and forth. And when we first started out, You could, it's partially like how the actor became used to the controls. They weren't as used to it, but because the way the UI was also shaped, you could see a little bit like the hand gesture, like they were pointing at menus a bit. And we wanted to obviously like kind of eliminate that friction and smooth it out more. And part of it is in the UI and the way it appears. And these are just Oculus controls, but part of it is also actors knowing there's a dressing room, the full dressing room they go into with all these options where they can actually rehearse. And there's a mirror there. in VR so they know a little bit about the avatars in the body and what types of actions cover for that.
[00:32:48.402] Kent Bye: Nice. And I was wondering if you could talk a bit about the process of casting all these extra performers, because you said that the original genesis of this project was to think about the different theater shows that happened and a cruise ship. And so you've kind of orchestrated this. So there's like this central hub where people can go back to, and then there's like these different acts and different performers. And there's a number of different performances that I just watched from beginning to end, just because it was such a compelling design and artistry and very catchy songs. And so maybe you can talk a bit about the process of actually like curating and casting and producing all of these performances for The Under Presents.
[00:33:28.930] Samantha Gorman: So Paiho actually helped us curate the acts that you see in The Under. And a lot of the acts are real performers from New York. Like you can Google them. You can Google Ronald P, Erin Markey. And even Helvetica, there are shows that you can go and see in New York. We just created their avatars. But it is pretty cool that you can actually see them. There was, I think it was Ronald Pete that showed up for the day we were motion capturing. And like, he didn't quite know what to expect and realized that we had an avatar of himself that he was going to inhabit on this motion capture rig. But a lot of the idea, again, is to have these sort of wandering acts that to a very first degree, they seem like they don't have a cohesion between each other. But in reality, they all fit into this larger world.
[00:34:21.103] Tanya Leal Soto: So basically, as Tanya was saying, one of PIHOLE's things they were working on was curating various performers from the downtown New York theater scene. So they were not only people they were familiar with because they had been working in that community for some time, but at different acts they went to. Like the one you were talking about was actually Sean Spada, who Jeff, who's one of the main actors, went and saw and then approached him after he had finished playing the piano and was like, hey, do you want to be in VR? And he didn't really know what to expect. And then he's the one who showed up at like where we were mo-capping. And suddenly like our artists had done a bunch of renditions of like different performers we thought, you know, that he would be and like, you know, his piano. And he was just like, it must have been very surreal for him. But the acts themselves, one of the main things that the actors are doing. is a lot of the actors and the performers we will have as live people are examples of acts that are offstage, or they can also interrupt the stage to create live acts. But their existence in the under, and one of the jobs they do is to narratively create the exposition that ties why there's all these variety of acts around to what the under is as a place. So they're almost living human story archives. that go around interacting with the player and filling in some of the gaps in the context.
[00:35:42.530] Kent Bye: Nice. Yeah. And so just as we're starting to wrap up here, I'm just curious if you could talk a bit about some of the either biggest open questions you're trying to answer or open problems that you're still trying to solve as you do a production like this.
[00:35:57.115] Tanya Leal Soto: Yeah I mean it's super for me I think it's hard because you know when you launch a game people are like oh it's launched it's out okay but like for me and this is a lot of you know my private work I do too is like a research and you know and my doctorate is it's more like a research lab right it's like an ongoing experiment. And I think as we do our training this week, and then we're going to be doing rehearsals, and I'll probably be in there invisible, also taking video. And like, I'm particularly interested in how, what is the edge of expectation in terms of when people encounter these actors, if they're thinking like they're AI, or they're live, what is that seed of liveness, right, that can be brought to these environment that like kind of triggers this intimacy or this exchange of energy? And like, how do you like qualify that? what are some like interesting strategies for performing not only like you know in these specific bodies but in this sort of context of this open world. So there's a lot of open questions for what will come out when we have a larger audience to test with. Yeah I think you know I'd like to write about that and you know hopefully talked about that more in the future.
[00:37:02.219] Samantha Gorman: I think for me, it's going to be really interesting to see the audience, the reaction of the audience, because this experience in particular is, to me, part game, part experience, part cinematic. For what we've seen with the places, like different people approach it very differently. gamers, you see them going in and like trying to touch everything and trying to play with everything, you know, and other sort of still more passive VR spectators, like go lightly and like are trying to find like the path of what is that they have to do. That is going to be like super, super interesting. And I think it has to do with like these new worlds, you know, these new immersive world, how are people accessing them? And like, how do we best create these worlds or these narratives?
[00:37:48.415] Tanya Leal Soto: It's interesting because it's right on the cusp and like, you know, Quest is definitely the audience is trending and the curation trending more towards games games. So, you know, we'll have a lot of people entering who are more from that mindset and want to test the bounds and the limits of the world. But when, for instance, we were showing you in Sundance, a lot of people's questions were like, more like, what is the trajectory of what the characters do to you and what you witness? You know, like, where do you have to stand or be for the story to happen to you? And that's a very different parameter than like game players, you know, think about like activating story.
[00:38:20.444] Kent Bye: Right. Yeah. Quick question. I'm just curious if there's any specific inspirations of other things, whether it's like sleep no more or a speakeasy society, or I know 1111 came out with a lot of ways of navigating time. If there's any specific inspirations that you took as you're putting together all these different mechanics.
[00:38:39.560] Tanya Leal Soto: Yeah, so in the game space, the idea of time loops, navigating time and working with yourself, that's a common game approach. Grade. I think when we were making this, the game Invisible Hours also came out, which is, it's definitely watching. It's a different kind of paradigm. Journey was obviously one of our main inspirations. as you know is for probably most indie games, but you can tell that you know easily in so many different aspects and in terms of like the multiplayer as well. Morale. Morale. Oh the island of the invention of morale was actually, that's true, I totally forgot that was like the basis for a lot of the story. That is this sort of a man who's stranded and he's watching a recording that he thinks is real and it's on this time loop and like how he engages with those characters in that recording. It's a book and I'm Who is it by again? Both of us. Yeah.
[00:39:34.853] Kent Bye: Yeah. Well, just the way that you're navigating time and bending space and time as you're locomoting. I think there's a lot of really powerful aspects there of the relationship between space and time and this experience that I am really taking away. I'm really enjoying it. And, uh, and finally, just curious if you could tell me what you each think is the ultimate potential of immersive technologies, immersive storytelling, and what they might be able to enable.
[00:40:00.336] Tanya Leal Soto: Um, yeah, I mean, that's so It's like, you know, like a good soundbite that like summarizes the universe as we know it.
[00:40:09.967] Samantha Gorman: 42. The answer is 42, Ken.
[00:40:12.609] Tanya Leal Soto: Like, I mean, I'm interested in working. The reason I work in this space is because I am interested in pushing the notion of from someone who comes from writing off the page and is interested in seeing how people want to engage with narrative and are just getting into understanding that this idea of branching out of digital storytelling can exist. But we've really, in reality, we've gone past beyond branching, like, you know, a while back. And I think it's less like nodes of exact, like, discrete options, you know, and we could, like, look at paradigms which are more, like, spatial. This idea of, like, storytelling through, like, manipulation of time and space. And that's one of the things that I think, like, immersive media allows easier than, you know, like, certainly print. Even, you know, some of the iPad work we had done, adding the third dimension is actually a powerful, extra tool for content that I think is really essential.
[00:41:03.297] Samantha Gorman: I think for me, it is what we were talking about earlier, this idea of wonder, you know, and to be able to leave the story. We all love stories that is never going to change regardless of what medium they are. But this particular medium lets you actually be part of the story and that makes it more real.
[00:41:20.092] Kent Bye: Nice. And is there anything else that's left unsaid that you'd like to say to the immersive community?
[00:41:27.218] Tanya Leal Soto: I don't know, I'm just excited to see how this shakes down and hope that people find wonder in it. And thanks for supporting us and trying it out.
[00:41:36.912] Samantha Gorman: Yeah, thank you, Ken, for playing the game and jumping into this crazy experience. Yeah, thank you so much. And we hope people like it.
[00:41:45.859] Kent Bye: Yeah, I think there's certainly a lot of really amazing narrative innovations. I love the platform you've built just to have those live immersive theater actors. I'm so excited that there's an experience that's starting to make this blending and blurring of the live element. A lot of really fascinating questions that you're Asking and I think really pushing forward a lot of innovations with this piece. So I look forward for it to get out there and for people to try it out. So yeah, just thank you both Samantha and Tanya for joining me today on the podcast. So thank you.
[00:42:13.080] Tanya Leal Soto: Awesome. Thanks so much. Thank you.
[00:42:16.289] Kent Bye: So that was Samantha Gorman. She's the co-director of Tender Claws, as well as the writer and director of The Under Presents, as well as Tanya Leal Soto. She's a producer for Tender Claws. So I have a number of different takeaways about this interview is that, first of all, there are a lot of narrative innovations that are coming through in this piece, especially with how you navigate around the story. Just the way that you're locomoting around, you're kind of like reaching out and grabbing the middle of spacetime and like pulling it towards you. And then the rest of the surrounding area comes back and kind of wraps around. And I actually found that to be a really quick and efficient way to navigate around a space and to maintain a sense of integrity when it comes to your spatial awareness. It's kind of a blend of just pure teleportation where it's comfortable for people blinking in and out of existence and teleporting, but you lose that sense of cohesion of the space that you're in. And I feel like the innovation that they've come through with the locomotion is that it preserves that spatial orientation while you're able to quickly locomote around. I actually really like this as a way of moving around the space. Even though there's an option to turn it off, I turned it off and felt like I was losing more of that cohesion of being in the space. So I really like that. But they also came up with a lot of new ways of being able to do motion capture on the budget. and to what sounded like created all sorts of really sophisticated tools. I didn't get a chance to see exactly what Samantha was talking about in terms of like being able to write visual studio code and then to be able to like export that out into a series of a timeline of like interactive HTML documents. And it sounds like they're also able to compile the script down into like a Unity animatic as it's moving around space. So they had to develop a lot of their tools in order to do a lot of the motion capture that they did, but is extremely efficient because if they said there's like 12 to 16 hours worth of motion capture content and dialogue, really efficient to be able to pack that into a relatively small download size. The other interesting thing about how they developed the story, Samantha said that they collaborated with Brian Eviston. He's a surreal sci-fi horror writer. He wrote the initial treatment with the characters, but Samantha found that she needed to actually do another treatment, which was like a little bit more of a spatial treatment. So thinking about how the stories were happening at the same time across the space, across this ship that you're on. And so she needed to do another spatial treatment on top of the existing treatment. And then there seemed to be a bit of a combination of writing down like the story, like getting a sense of what the arc of the story is, but there was a lot of gaps that the characters had to be fleshed out. And so it sounded like working with Piehole, they were able to allow the actors to be able to flesh out each of these characters and to be able to improvise a lot of the specific details of who these characters were. especially how they move and having the theater troupe come in and have each of them embody the character as they walk through a space and then to see what emerged and then able to then feed that back into the script and then be able to tightly orchestrate all these different components. Now, the one thing that they did do that made it easier on themselves is that they don't have the characters moving through the space nonstop. That would have meant that they had to do editing of the motion capture between this huge space and by having them blink in and out of existence, allows you to stop following them because they're not going to have anything interesting happen with those characters and They have this mechanic that allows you to see how the memories are fading away and you have to go Restore the memory and then from that point then there's something interesting that happens after that that allows you to focus in on the actual interactions or interesting exposition of the characters that are happening throughout the story and But it reminds me of this combination of trying to take this iterative game design approach with something that tends to be a little bit more of a waterfall approach with developing stories where you kind of write it and you produce it and then you edit it. There's some changes that happen throughout that process, but it's more of this linear process of storytelling rather than this more game design iterative approach. They seem to have this combination of those two approaches of allowing the actors to improvise different elements of the character and interactions, and to still have the overall character arc, but to have this top-down planning from the treatment and the story, more the waterfall approach and combination, and more the bottom-up or iterative approach that they were able to piece the story together as well. So I made it through about a third of the story so far and I'm kind of waiting for everything to be finalized and polished and released and especially as I go in there and have the live immersive theater actors there just the whole time before I play through the rest of the experience. But from what I saw so far, there's just a lot of really interesting characters and story and also just innovative ways of navigating around the story. So if anybody's into narrative experiments and the future of blending together what an experience is and what a story is within virtual reality, this is certainly, I think, one of the best examples that I've seen so far. Certainly a lot of really innovative innovations that they've done. So I'm super excited to see where this is going to go in the future, to have these live immersive theater actors. This is what Neil Stevenson in the Diamond Age talked about in terms of these reactive actors or Ractors, where he imagined a future where people would be living into these immersive theater type of experiences and then paying people to be like these actors. And this is essentially what we have, is like these live immersive theater actors that are able to have these one-on-one interactions with you. They take you to these secret places in the under, and then potentially teach you how to do specific things. Because as Samantha alluded to, there seems to be different aspects of The Under Presents where there's these secret areas that you can only get to if you get enough people that are also going through this experience. These silent mimes that you can see moving around, you're able to do these acts of ritual magic together to be able to unlock different aspects of The Ender Presents. So there's many different layers to this experience and I was able to start to peel the first layer off the onion but I'm excited to dive in and to really explore around this experience. And the last thing I'd say is that they're experimenting with different aspects of emergent play. So seeing how these groups can act together and see what emerges because there's going to have all these people together with these different objects. And I think there's just going to be things that they didn't even imagine people are going to start doing. And also there's going to be a part of figuring out how to communicate with each other, with your bodies and gestures in a way that is somewhat similar to Journey, where you're able to have these different types of behavioral interactions without being able to speak. So you can't speak as someone who's in the under, but you're able to do these different gestures just to see what kind of emergent behaviors come out of that. And also just the art of wondering and having these different aspects of serendipitously colliding with these different narrative moments and to be able to really allow you to explore around a space and to discover these different secret areas or to run into an immersive theater actor and to have them take you on this whole adventure. So really drawing upon a lot of these different aspects of these one-on-one interactions that we've seen in different immersive theater plays like Sleep No More or Then She Fell from Third Rail Productions, which was really kind of orchestrated around those one-on-one intimate connections. And so Samantha is actually actively writing her PhD and so she's actively researching what does it mean to have these live interactions within VR. So, that's all that I have for today, and I just wanted to thank you for joining me today on the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a list-supported podcast, and so I do rely upon donations from listeners like yourself in order to continue to bring you this coverage. So, you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
