Kiira Benzing‘s Loveseat was an ambitious fusion of live theater performance with VR technologies that premiered at the Venice Film Festival in 2019. They performed live for the audience in Venice while wearing VR headets as their performances were simultaneously broadcast within the virtual reality environment of High Fidelity (back when they still had VR components before their pivot to spatial audio).
There have been a lot of other fusions of live theater with VR technologies recently, including Benzing’s follow-up live theater in VR piece a year later called Finding Pandora X, which won the Best VR Immersive User Experience at Venice 2020. She took a lot of the lessons learned from Loveseat and applied them to Finding Pandora X, especially the fact that when you do a live performance simultaneously in VR and IRL, then you end up doubling the production needs and staff that need to attend to both performance contexts.
I had a chance to unpack some of the other lessons learned from Benzing at Venice 2019 after one of their performances, including some of the specific acting insights from all three of the actors involved in the production: Jenn Harris, Jonathan David Martin, and Sam Kebede.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So there are a lot of different live immersive theater mashed up with virtual reality experiences that are happening this past week. And I wanted to dig into my archives to go into some of the roots of some of the technologies from previous film festivals. But just to set a little bit of a context, the Royal Shakespeare Company, in collaboration with Marshmallow Leisure Feast, as well as with the Manchester International Festival, as well as the Philharmonia Orchestra, they produced a piece called Dream, which involves all sorts of live performance that's broadcast within this virtual world. And a lot of those technologies and those ideas, I think, have seeds into previous projects that have been happening over the past couple of years. I know that there was a a real time live winner from Epic Unreal Engine, where they're doing real time motion capture within the Unreal Engine, which is a basis of what Dreams is using, but also just generally using the volumetric affordances of virtual reality to be able to broadcast live. acting performances. And I'm going to go back to a piece that showed at Venice 2019 called Love Seat by Kyra Benzing. She had three different actors there. They had a piece that was broadcast live in high fidelity that they were acting and doing all sorts of stuff like flying around and changing scale and doing things you could only do in VR. But they also had a co-located performance that they were giving there within Venice with three actors, including Jen Harris, John David Martin, as well as Sankebete. So I had a chance to talk to all of them after one of their performances in Venice back in 2019 and wanted to air it now just because after this piece of Love Seat the following year, Kira Benzing came back with Pandora X, which actually won an award at Venice. And then that's actually showing here at South by Southwest, which I saw the second time last night. So there's a kind of an evolution of some of these live performances, but I want to go back to the Love Seat piece just because there's a lot of insights that came out of that conversation that feel very relevant to all the other things that are happening. I mean, a couple of other things just within this past week and coming up here is that the Fifth Wall Forum was a collaboration of a lot of immersive XR technologists, as well as immersive theater folks. And they put together 11 different projects across all these different teams, and they were presenting their initial glimpses this past Saturday. And also just coming up, there's a onboard, which has a number of different immersive theater experiments. And that's by a number of different folks like the jigsaw ensemble, active replica and agile lens. So a number of different prototypes that are happening in platforms like Mozilla hubs. So there's a lot of different activity when it comes to live immersive performance. That's coming up here that I'll be talking to some of the folks from marshmallow laser feast, and perhaps some other representatives from the world Shakespeare company to kind of dive into that as well. But I wanted to just, uh, put this out before the South by Southwest gets properly started here later today. And I'll be digging into some of those content and then talking to other folks around this topic later this week. So that's what we're coming on today's episode of the voice of VR podcast. So this interview with Kira, Jen, Jonathan, and Sam happened on Saturday, August 31st, 2019. So with that, let's go ahead and dive right in.
[00:03:15.532] Kiira Benzing: I'm Kira Benzing and I am the director of Love Seat and Love Seat Soundstage and this is a VR theater production.
[00:03:24.356] Jonathan David Martin: I'm Jonathan David Martin. I'm an actor in Love Seat and I'm a creator of theater and mixed media.
[00:03:30.318] Jenn Harris: I'm Jen Harris and I'm an actor in Love Seat and I'm also a producer and writer.
[00:03:36.655] Sam Kebede: My name is Sam Cabetta. I play the host on Love Seat.
[00:03:41.700] Kent Bye: So maybe you could give me a bit more context for each of your backgrounds and how you got into this immersive space.
[00:03:48.218] Kiira Benzing: I've been working in, I want to say, transmedia and immersive storytelling for several years now, coming from a theater background originally, then moving into documentary filmmaking, and then into these kind of larger installation-based, totally nonlinear productions. And this is for sure one of the most complicated ones I've put together, because there are so many moving parts, and it's all live.
[00:04:13.115] Jonathan David Martin: Like Kira, I also come from a theatre background. I've been doing a lot of work in transmedia, and in particular working with collaborators who are from a design and technology background. Unlike Loveseat, where we take theatre and put it into VR settings, I've been actually doing things of taking VR and putting it into a theatre setting, so sort of complementary backgrounds.
[00:04:33.397] Jenn Harris: I come from a theater background, as both of these two, and I actually met, I'm also a creator of shorts and web content, and I met Kira at the Tribeca Film Festival two years ago, when I had a web series premiere there, and we just struck up a conversation, and she taught me a little bit about the VR world, and it was, yeah, I was intrigued.
[00:04:53.082] Sam Kebede: My background, I am a stage performer and a comedian. I do MC work a lot, which is very useful in a production like this. And that's me.
[00:05:03.770] Kent Bye: And so maybe you could give me a little bit more context as how Love Seat came about and what was the evolution of this project and how it ended up here at Venice.
[00:05:12.127] Kiira Benzing: Loveseat came about because I've been trying to do something narrative and scripted in the space where we could build a show and integrate the magic of what you can do in VR but also expose the widest audience possible to it without putting an entire audience in headsets. So basically we're able to have our actors fly and change costumes on the spot and smoke come from anywhere in the space. And that can happen in an instant. Those things that are a little bit more complicated in the real world, we've got this wide world of possibilities and unique physics that we can play with in a game engine. And we're working uniquely in a social VR platform, also very passionately curious about exploring a social communal audience in a virtual world, and then how we're used to that tradition here in the physical real world. And how do I bring those two types of communities together? So we're beginning to explore that in these pre-show activities and introduce the two audiences together. And I think there's a lot of room for that to grow in the future. And that's been an exciting first step. And then I brought on Mac Rogers, who's a New York City playwright, to start watching some research experiments that I was working on with this social VR platform called High Fidelity. And Mac started ducking his head in and understanding that we could play with flying and all of these things that are much harder to do in the real world again because putting an actor in a harness and having them physically train for that is a lot harder. Here it's more like a mental training that they're doing. I mean Jen does a lot of flying in this show and I'm sure she can talk more about that later. And so that was the dream really was to do something scripted and Venice gave me the opportunity to be here which felt like well now we write the show and we just we do it.
[00:06:56.870] Kent Bye: Yeah, and as I watch the show, there's the three actors that are in and out of virtual reality, and there's screens showing what they're seeing, or maybe like a third-person perspective of what is happening. I chose to dive into VR, but I was coming in and out of VR, so I could actually get like three or four lenses of what was happening, because I could go into VR, be in the audience in a VR stage that had your virtual characters up on stage. I could Lift up my headset and look at your first-person perspective point of view to see what you were seeing Because I was right next to your monitor. I could look up on the screen Yeah, I was next to your monitor watching what you were seeing and then I could look up and see What the third-person camera that was shooting the virtual camera was seeing and I'm more of an omniscient point of view and then I could just look at you as actors to see what was happening and so for me I was kind of like modulating and editing my experience by flipping in and out because there were some things that were, I was in some ways testing to see what was different to be able to see both at the same time and context switch back and forth. But I'm just curious what the experience has been for you as actors where you're actually in front of a live audience, but your face is completely occluded. You can't actually see them in that you are acting in these virtual spaces and like what that has been like to sort of blend these two worlds together, the virtual and the real.
[00:08:17.992] Jonathan David Martin: It's a totally new experience as an actor. There are things that I've done in particular with puppetry that are good starting place to understand the mental framework and the attention that one has to have on a stage that is at once live and virtual. The puppetry is useful because really the avatar is a puppet and so a lot of our early rehearsals were just figuring out how does this puppet work and what is the audience seeing and how are they the most expressive and how do they most come alive. So there's that just sort of part of it which is familiar but then where VR is really unique is that you have the opportunity as a performer to perform two shows at once. You're performing on a virtual stage for a virtual audience via a virtual avatar character and and also in this context live for an audience that's there in the room with you and with actors who are live in the room with you. And so it's this great rub your belly, pat your head kind of activity and that's a fun challenge to really give a full performance for both of them.
[00:09:20.990] Jenn Harris: Yeah, I would say it's interesting. I found the technical part of it intimidating at first because I don't even identify as a gamer. And that was what I came in sort of thinking that I sort of had to do or understand. And so it was interesting to just get used to the controls, just basic use of using VR. And then It's one of those things like in theater when you do like stage fighting or combat and dance and anything like that, you have to, if you're still doing the scene, you still have to think technically what you're doing. But there's technically stuff that I have to keep an eye on that normally you wouldn't have to. And it's interesting, at first when I started doing it, I did feel like I wanted to take off my VR mask and just do the scene with Jonathan. I was like, I just want to do the scene with Jonathan. But now I feel so comfortable looking at his avatar. And on that stage, because we've spent so much time doing it, that I feel like that's where the scene is, and that's where we're acting. It feels correct to be in there and look at his avatar and do the scene. So it was a learning curve, but it still has the same map as doing a play.
[00:10:30.220] Kent Bye: Cool. And you said you wanted to ask a question.
[00:10:31.861] Kiira Benzing: Yeah, I've got a question for the actors. Does it feel like a real stage to you when you're in VR? Does it feel the same to when you're on a theatrical stage?
[00:10:42.790] Jenn Harris: I do feel like it. It didn't at first. But it does now, in a way. It feels like it's its own stage. But it does, to me, feel like a stage. And it feels that way because we've set blocking, like you would if you are on a real stage. So I actually do feel like it is a real stage. I do.
[00:11:05.110] Sam Kebede: I would almost like it's doubly so in the sense that a lot of times when we're doing stage performances, there are lots of surreal elements. There's flying, there's like cherry rigs and everything that kind of like lets you do anything. And with this, it's just a simpler shortcut to that, you know, like, oh, if we needed to like have me rappel in for a show, which I've done recently, it's so much easier because now it's just right thumb down and then it's there. So it's like I constantly feel like I'm just on a regular stage where physics is different.
[00:11:35.612] Jonathan David Martin: Yes. Yeah, yeah, it really is. I think actually the thing that makes it feel like a real stage for me is that the audience, the virtual audience for this show is able to sign in from anywhere in the world, that they are also physically around that virtual stage. And so that helps me feel like I'm really playing in the round, as it were, in an arena where the audience is on all four sides. And that's very true to how it is in real life.
[00:12:02.862] Kent Bye: Yeah, because I was watching it and having all these different perspectives of being able to be in VR, to come up and look at some of the monitors on the side, to be able to look up on the screen and look at what was happening between the two actors. Well, first of all, the audio was weird because mostly when I'm in VR, I'm hearing like a virtual audio. And so it was weird to be in VR, but hear the acoustics. That was actually like I was on the right side to have the actors both spatially correct. Sometimes it was sort of like I could tell there was like somewhat off because the way that had moved around a little bit but the audio I think was the most striking because it was like the highest fidelity audio I ever had because it was like the natural reflections off of the room and it felt like More real but at the same time there is the fidelity of the emotional expressions that what I would lift up I would see like the fidelity of the facial expressions, but I think is probably the biggest lack of you know what you get from a live actor and on stage is to see the sort of emotional expression that comes to the face. And I think without that, it feels like it's a little bit more muted emotionally and you might have to, like, compensate a little bit more. So I'm just curious, as actors, as you rely so much on body language cues, and I'm wondering if you've found that there was a new body language cues that you maybe had to notice or if there was certain aspects of the facial expressions that you felt made it harder for you to make connections with each other in different ways.
[00:13:30.013] Sam Kebede: You know, I found that a lot of times, because the face would be far more fixed at times, I had to brighten my voice a lot more if I wanted to show a smile. Like, almost as if, like, oh, if I sound like this, you're going to think that maybe, like, you can feel and hear the smile on my face. And there were lots of moments of that I found. and like finding the physical vocabulary because I mean I play a lot of big characters and if I'm I can normally just like walk 10 feet or go left or right but I'm in this box so it's like cool let's compress all that and if I want to have any lateral movement it's just my thumbs but I have all the vertical space I want.
[00:14:09.714] Jonathan David Martin: Yeah, I would say I'm constantly thinking about the performance differently than I would in a normal show in terms of how I'm expressing the life of the character at any given moment. And for me, it's very similar to puppetry where the puppets have ways that they express themselves that may be more specific actually than people can, in part because you don't have all the different ways that people can express themselves. In this show, the avatars don't really have a lot of specific facial expressions, but it does mean that if I tilt my head a little bit, and that's something that the avatars do very well, those little subtle gestures of head movement, or of hand movement, or of body posture, that that then substitutes for a lot of things you would do on camera with your face.
[00:14:55.335] Jenn Harris: Yeah, and I would add to all of that that I'm still learning. I still feel very much in a process of learning what my avatar, what is successful communication and what is overkill or movement that could look like a glitch or that could just not look as natural as I would like to. I still feel like that's something I'm very much learning.
[00:15:21.727] Kent Bye: Well, it seems like there is a lot of things coming together here maybe for the first time to have like a whole stage built out in a virtual world with high fidelity, a social VR gathering, and then you're having live actors performing and then streaming that live into the virtual space while also streaming different elements of the camera shot and so maybe could talk about like and in some ways you have two audiences you're serving and it's like you split who you're serving then do you preference one or the other I only saw it here so I don't know like when people took off the headsets if you know the audio was still there what that was like to then have the avatar that was there kind of like freeze or disappear or you know there were certain ways in which the audience that's here in Venice got the better they were prioritized in some way so there was a bit of a hierarchy in that sense of like preferencing the people that were here rather than leaving the audience here behind or in the dark. So just curious to hear a little bit about this fusion, but also this balance of you have these two contexts that you're serving at the same time.
[00:16:24.817] Kiira Benzing: There's two sets of preferences going on because also the virtual world can see certain things happening with blocking that the real world can't. Like we're giving them a two-dimensional perspective of those things and changing those camera angles constantly across three projection screens. So we're doing our best here in the physical world to represent that. But also the virtual audience, and there's not a lot of onboarding for this yet. That'll be in a future version. But the virtual audience can change their view at any time. And eventually they will be able to fly up and join Abby in the clouds. So there are these other possibilities there that they have views into things that the real world audience doesn't get. And so in a sense there's kind of two sets of priorities going on. It's also interesting as I've learned building this production on this kind of rapid prototyping sense that we basically have double sets of crew. If anything, we almost need more people. We have two sound engineers mixing and cueing music, mixing for the show in VR and mixing for the live show here. Sarah, our stage manager, Sarah Zarod, who's just doing a phenomenal job, is running two sets of lighting cues and prop is set dressing, you know, the chair appearing, all these cues. She's running in the game engine, in high fidelity, and also running physical real-world cues. In a way, we've had two sets of costume designers. Our character modeler and our rigging is one set of costumes, and all of those costume changes are rebuilds of those avatars for them to change into on the fly, plus their physical costumes that they're wearing here. So, it's been an interesting production for me to see, like, we're basically running two shows at the same time. And sure, right now, some things might feel short-changed for one audience or the other, but that's only because we're early in this process right now, and we need double the amount of people.
[00:18:16.413] Kent Bye: And I'm curious to hear from actors in terms of there's different moments that I feel like as I was watching in VR, I was like, okay, these are moments that are better to be in the virtual audience. It's like the scale shift or to be able to change costumes or to like, you know, fly around or to see the virtual sets in a more spatial sense rather than a projected 2D sense. But just curious from your acting perspective, like the moments that you have in VR that are like totally unique to VR that you can never really have. in acting and sort of what those are like, whether it's a scale shift or the costume change or the sets that are appearing or the flying around or whatever it is for you, like kind of the most striking aspect of acting in VR.
[00:18:57.706] Sam Kebede: Yeah, I'd say that the big thing for me would be the teleporting, because I feel like that adds this new physical vocabulary that I will never be able to do. I was actually recently doing a production of Midsummer Night's Dream, where I only jumped from place to place. That was a big part of my physical vocabulary, because I wanted to make it seem like I was randomly appearing. This was what I was trying to do, and I would never be able to do that in a real reality play.
[00:19:25.312] Jenn Harris: I think flying is pretty incredible. And I think when you're using descriptive dialogue that is poetic or might seem a little maybe possibly on a blanks over poetics in some spots, it's very helpful to have like magical realism help you with a dialogue that is sort of beyond what like a human capacity on a stage is. So flying and transforming into something helps facilitate a dialogue that is poetic, I find.
[00:19:56.873] Jonathan David Martin: Yeah, I would echo both of those things. I would actually say one interesting challenge is the fact that you don't have your sense of touch in the virtual world like you do on an actual stage, which is funny because I'm using my sense of touch quite a lot on the soundstage part and sort of the live audience part, you know, to understand where I'm standing in space so that I don't leave the sort of area that's been taped out for me. But in the virtual stage, you know, we have sets that are moving in and out and I'm always having to sort of be aware spatially where scenic pieces are so that I don't actually walk into them. Because if you walk into them, they don't let you know that you've walked into them. You just walk into them. So, you know, there are moments like where I've come out of the headset for a monologue and then come back in and I find I'm like standing in the middle of a set piece and it's a little awkward. It's a little embarrassing. You're like, I'm so sorry. Pardon me. Let me just move out of that. So that's an interesting challenge, or wanting to make it look like you're leaning against a set piece. And of course, you can't actually lean against that thing. So you're having to gauge without having to look at it too closely to know, I think it's about this height. Or actually, the gestures of your head, like, oh, I want a moment where I'm nervous, so I try to fix my avatar's hair. And so I spend a lot of time with a virtual mirror to know my hair as the avatar isn't right on my actual head. It's about four inches above. But I kind of now know. spatially where it is so that I know I'm getting a pretty good approximation and can create those things so it looks like my avatar actually is printing himself or printing himself.
[00:21:29.878] Kent Bye: Yeah, there's an interesting paradox in the sense that you're doing this live, and I'm here and I can tell that it's live, but I would expect that there's more ways in which that you could capture what you were doing here on stage and replay it back in the virtual world, and some people may not be able to tell the difference at all because While I'm live, you are projecting your audio, and as you are turning your head around, the audio reflections are actually changing very dynamically, and that level of precision in virtual worlds aren't there yet. And even if they were there in VR, you could still simulate it in a way that it'd be very difficult to tell whether or not this was live or recorded. And so, for me, it starts to beg the question of, like, what is the liveness element for the virtual audience? to allow them to know that this is happening right now, rather than something that could have been recorded. And what that means in terms of, like, when you think of theater, there's no question that it's live, but in a virtual theater, there is this sort of question in my mind.
[00:22:30.352] Kiira Benzing: There's a lobby scene that we didn't quite have time to fully flesh out where that because I think in social VR the element of the social community coming together and conversing with each other and this kind of beautiful complimentary action that happens about like them noticing each other's incredibly customized, unique avatars and what part of the world they're joining in from. I think that's an element that we haven't had time to explore. It was put in the script and that will come back. But that lobby area, which is getting to happen here in Venice, and then this kind of greeting between these two virtual and live worlds, I think that will help us remember this element of live, that this is live. It's happening as close to now as we can. Of course, there's always going to be a little bit of a delay, but we're pretty much in the moment, which I think is what's special about performance in general, be it dance, be it theater, be it live music, all of those things happening in that breath of time that we share together by breathing in a space together. And what that means for a social world, I'm not exactly sure, but I do believe that social VR will grow and blossom. And the more that we can find meaningful content and stories to tell in those worlds and bring people together around those types of events, I think that will start to develop a different kind of culture than we've maybe even seen so far. And that will lead us to having more enriched, meaningful experiences in those worlds and perhaps even developing memories that we share with each other from being in those worlds together. So it's my dream to build a kind of community lobby space that happens pre-show and post-show where people can get together and meet each other, have a beautiful experience, journey together through things, jump to the clouds with Abby, change their perspective, It's interesting to me, Kent, that you lifted out of headset and you watched through all these different screens. You had these many different types of perspectives and I think that's becoming a recurring theme in my work because it was also in Run-In where you could choose to jump to these different walls and experience from different perspectives. So I guess that's coming back. I'm noticing that now. What I think will be special is if we can find these types of shared narratives together and then have that post-show conversation as well. Engage in those things where you might go get a drink after in the theater, but what does that mean for the virtual social world? Maybe you jump to a new kind of hangout area where you get to have a meaningful discussion about the meaningful experience that you've had.
[00:25:11.897] Kent Bye: The thing that comes to mind is improv and how like with improv there's like more of a conversation in real time with the audience and that this is obviously a scripted narrative where the building and releasing of tension in a very fixed timing way that makes it difficult to have that user participatory interaction but does feel like that there's this well there's an element of having people do like kind of mad libs but the virtual audience isn't necessarily able to participate in that so this sort of like a more of a logistical issue to sort of be totally inclusive there but i'm just curious from acting perspective if you've experimenting with doing, like, live improv, both on, like, real-life stage, but also in VR, and kind of seeing the difference of what it's like to have that, like, live element while you're in these virtual spaces, sort of interacting with each other, especially because there is so much about body language and facial cues that you may pick up on as well that you're also missing out on. And so, if you've had an opportunity to experience any improv in VR,
[00:26:14.147] Sam Kebede: Yes, I have. I, you know, I find when I'm in the, because I've experimented a lot, especially through our previews and with the audiences doing that more and more. There were lots of times, especially earlier on, where I would wait for them to emote and I would like encourage them to do it more. We're still trying to find new, more nuanced ways to encourage that. I know today we were on a bit of a rush schedule because usually I get a chance to be in there and be like, hey, you're there, you're doing this, give me an emote. But that's the hardest thing is that I can't vocally hear anything. A lot of times, you know, when doing real life improv, you know, the house lights are off and you're on stage and everything is vocally. That's why they say, can I hear a this? Can I hear a that? So you can't see anything. So you kind of are trained over the years to fully go off whatever you hear, which I can still do in the room when I have my headset on. Like if I hear someone, cackle or if I hear someone do something. The other day, one of our spotters, Mark, I think he was trying to tell me there was someone behind me raising their hand. And I was able to have this great rapport with the whole crowd, even though I can't see anything. So I'm still trying to interact and figure out what that is without sound, without these lag times, without letting it drag the show.
[00:27:31.560] Kiira Benzing: I did an improv show. So some of my early research with High Fidelity, we did a case study with some of my colleagues in New York. We've been writing a blog called Alive in Plastic Land and that initially started with researching what types of avatars performers need to be able to do as realistic and compelling a performance as possible. and then that turned into a study that we published and released at IEEE GEM this year at Yale. On the way to that publication, we did an improv show where we did short form improv and we taught an improv class, all in high fidelity. And I like the idea of improv because I think that there's a sense of play that happens, but at the same time, it's really hard to do things like bring in props and cue, that stuff. There's just so much coding and 3D modeling that goes into that. So by having a scripted show, we're able to prepare for those beats in a more magical way. And I think that there will be time and room to explore again with improv. But for now, I'm feeling really happy about us exploring this beautiful world that Mac Rogers has written for us.
[00:28:39.212] Kent Bye: Yeah, at Sundance this past year was Tender Claws' The Under, which is, I think, one of the first experiences I've seen, at least, where there's a plan to release a virtual reality experience with paid actors to be able to do, like, kind of live immersive theater in VR, kind of the Neal Stephenson dream of Diamond Age, where people are actually getting paid to be Ractors and go into these virtual spaces and provide entertainment. But, you know, there's things like Sleep No More, which is obviously very much focused on moving your body through this physical space, and I think locomotion VR is problematic to be able to fully simulate what that feels like but it feels like there's some opportunity here to Get to the point where it's a little bit more like the people that are there are being able to either be ghosts or maybe even Cultivate the skills to be live-action roleplay people who actually embody their own characters and engage with each other And so it feels like this iteration is sort of like whenever there's a new medium you kind of replicate the previous medium so radio would have like opera plays and then like TV would have like radio plays and then the medium develops on itself. And so I feel like this is a good porting of a stage play in VR but it doesn't feel like to me like the final form of where this is all going which is like to really engage and allow the audience to fully participate either in the story or in some ways moving around in a way that doesn't have a proscenium and you can actually Go around and explore these spaces So I'm just curious with that in mind what you see is kind of like the next like what you learn from this Process and then what you see where this is going next
[00:30:08.097] Kiira Benzing: That's exactly right. So I have more complicated scripts that I've been developing with one of my co-writers, Alyssa Landry, and I'm very much into world building and creating these other realities that people can very much have different types of non-linear perspectives and participate. And I would love for our audience to jump through portals and follow actors around and split off and have branching narratives and come back to story points. It's just a lot to build. So we'll get there. That's certainly in the future. I'm sort of sitting on those ideas right now while I learn from this experience. And then for me, every experience that we do is a building block for what we can get to.
[00:30:41.986] Kent Bye: So what did you learn from the experience?
[00:30:45.137] Kiira Benzing: Though we're running two productions and it's pretty hard. We're networking so many computers here on a tiny island in Venice. So, you know, we're pulling off a lot of magic tricks right now. I mean, it's happening. Right now it's like one day at a time, but I can see the future. You know, my brain is already there. The minute I can have an audience jump through a portal and follow her on the cloud and they get different monologues in different spaces. I totally want that to happen. And Mac and I have actually been developing concepts for that, and we just pulled back for this to execute something that we could handle. And even here, I would say most people might categorize this as ambitious. That's a weird word for me, because it just feels like this is just reality, just how my brain works. But we are pulling off a lot. And it's a testament to the enormous crew and talent that have jumped in and learned these skill sets that we've developed really fast.
[00:31:41.634] Kent Bye: I'm curious to hear from the actors here where you would like to go, what you would like to experience either as an actor in a virtual space or what the lessons you're taking away from this experience.
[00:31:53.038] Jonathan David Martin: I think I would love to continue to have experiences that blend our online personas and offline personas. I think the one thing that I would love to continue to import from things like traditional theater in particular is the empathy that watching a story in that way can create. And I think VR can actually be a really powerful tool to create empathy, especially around stories that the audience has very little frame of reference for. I think there's some really powerful stories to be told in that way. And I think there's a way to combine the agency that you have in things like video games that are really native to online spaces with the empathy that you have from analog performance styles. And I'm really excited to see where that kind of hybridity goes in terms of the way that we then tell whatever we define as stories, whether they're more interactive or less. and that there be more variety in terms of those kind of experiences and stories. So that's really exciting for me, both as a performer, a content creator, and as an audience member, because I think there are different types of stories that want to be told different ways, and I think, especially looking around at the stories that are being told here at Venice, they're telling stories that I watch them and I'm really moved and engaged, and I can't imagine them being told as compellingly in another format.
[00:33:12.908] Jenn Harris: Yeah, I would agree with that. There's something to this medium of telling a story that just hasn't been tapped yet. And there are some stories that just need to be told in a way that just is deeper or more expansive visually and audio together. I think what's interesting learning about VR in the future is sometimes people don't like participation live. People don't want to participate. They don't feel comfortable. they're more introverted. And there's something about virtual reality where it seems like someone can give of themselves a little bit more if they're in an avatar body than a physical space and they're in their own body. And there's something that is very important that people are able to connect in some way if they don't feel comfortable doing it face to face or in front of someone. So there's something very interesting about the social aspect to what we're trying to do here that is inclusive. for people who want to interact but want to do it in the way that feels comfortable for them. For some people, live just isn't as comfortable. And from a writing perspective, it kind of is like Britain for a little bit of animation. And there's something that is really fun when you get to write without any limit of having to think of how to film it or how you're going to stage that. Because you can write something and go, well, then they just fly there. There's something very freeing about a writing process that's in a medium where anything is possible. And I think that that's really exciting. And as a performer, it's really fun to think that you're going to get to participate in an anything-is-possible kind of way. And there's so many stories that can get fleshed out in an anything-is-possible kind of way. And I just think that's pretty cool.
[00:34:59.736] Sam Kebede: I suppose the biggest thing I'm kind of looking forward to in terms of VR is the increase in availability for headsets both for the opportunity for all audiences members to have them but also so that the socio-economic barriers that VR brings can kind of be brought down so we can see more works from communities that may be disenfranchised or may not have the means to express themselves through this new and burgeoning art form.
[00:35:30.209] Kiira Benzing: One of my hopes is to try and make just theater more democratic, make it more accessible for people. So for people that can't afford to get to Broadway, people that can't afford to get to the West End, they have an opportunity to sit in a headset and get there. Now that is also expensive and hard to do too. But now through a platform like High Fidelity, they can join in through a desktop monitor and several friends could watch together. They could even project that on a wall themselves. So I think there's, I'm trying to break down the layers of the inability to access some of these beautiful art forms.
[00:36:05.163] Kent Bye: And as I talked to different creators here at Venice, people are coming from a lot of people from the film background, some people from gaming, but it's mostly film and a little splattering of some people from a theater background. When I talk to people, they have gone back and forth, some of them from film and gaming, and then some have the theater, but it feels like for a lot of people, the theater is a big gap in terms of it wasn't necessarily like the intuitive thing to have on your resume to sort of be where we're at right now with virtual reality. But it seems like that there's something really important about both spatial storytelling as well as the way that you move your body to really get the full context of a scene, which I think in the 2D frame in a film, you sort of get cut off from a lot of that context. But I'm just curious from each of your perspectives of what you personally think that theater has to teach the medium of virtual reality.
[00:36:55.520] Kiira Benzing: I think theater can teach us a lot. I mean, first of all, amphitheaters were built in this incredible way that enable us to watch a performance and gather as a community. And all of these stages that were built in the round so many years ago, that enables us to live in a more three-dimensional, experience something in a more three-dimensional way, which we already do in the real world, but when you transition to something like film and cinema, you're thinking in a very different type of format. You're thinking in a rectangle. You're thinking it's flat. Distance changes. Spatial audio changes. Like the way that things come and appear in the world changes. So I think that's why I've transitioned very quickly into creating VR because of that theater background. I get that question a lot. Like, how do you figure out how to do this? and I'm not sure. I think it's only because of my years spent in theater.
[00:37:50.848] Jonathan David Martin: Yeah, I think that's really, really true. And I think that theater, as opposed to film, is actually much more democratic in a way with the audience. Because with film, you can really control what the audience sees, what they hear, the rhythm of it. And theater necessarily gives a lot of that control both over to the performers and also to the audience. That, yes, you want to guide their eye, but they have a lot of agency about where they look and what they're paying attention to. And I think that that understanding of how to guide an audience's eye in a theatrical way is incredibly useful when you're applying that to telling VR stories. And I think the kinds of stories that can be told in VR can learn a lot from how theatrical stories are told. in the sense that theater can borrow from all art forms, it can borrow from dance, it can borrow from music, and so while a lot of theater is dialogue driven, sometimes it's very visually driven, and being able to toggle between those different modes of storytelling is also something that you can do in VR. Like with Love Seat where it's not photorealistic and it doesn't need to be, we're very satisfied as an audience to kind of fill in some of those details like around facial expressions and that kind of stuff. That's something that I think is harder to do like with film but is very much a part of the world of VR.
[00:39:15.197] Jenn Harris: Yeah, echoing as to what Jonathan was saying, I just think it's more vulnerable. Theater to me has always felt more vulnerable than film and television, and I think that that's vulnerable whether it's emotionally or just physically. You're having to share space with people in some way, shape, or form and see something live. And to me, vulnerability is like dangerous and exciting. Also vulnerable, like something could happen. Someone could fall. You know, it's not perfect. It hasn't been edited perfectly. And it leaves room for vulnerability and for space for you to just come up with your own thoughts and ideas and not be told how to think and feel. And I actually think that's a very generous art form.
[00:40:00.510] Sam Kebede: I think that one of the many things as we've been listening that theater can teach VR is trust and simplicity. I think that a lot of the greatest moments in a performance come from very easy means and the trust and the togetherness I find when we do this show. To heighten the show, I do something as easy as saying, I would like you all to applaud. And that's something I've had to learn through the theater of just asking for something from them and trusting that they want the show to go as well as anyone else would want to. And I think if you come from a film or TV background, you may not know how easy it is to ask the audience to help you or your viewer to help you.
[00:40:41.154] Kent Bye: And finally, just curious what each of you think the ultimate potential of immersive storytelling and what that might be able to enable.
[00:40:50.420] Kiira Benzing: Can somebody else start?
[00:40:51.841] Sam Kebede: I'm always last. I'd say the big thing that I find lacking in any sort of storytelling is the fact that you can so easily be separated via space and time. Like sure, I can do a piece about the Amazon and what's going on there and the destruction of this beautiful forest and species. But if you could put someone there and let them see it, let them be away from wherever they currently are and be forced to be part of this world, that's what I think it is. I think the biggest thing that separates us is space in terms of understanding what people are going through. And that's the final piece that I think a lot of performance, whether it be film or theater, is lacking. And I think that that is what immersive virtual reality offers, that final piece so people can truly fully empathize and realize what's going on in the world.
[00:41:48.867] Jenn Harris: I don't know yet. I'm still thinking.
[00:41:53.566] Jonathan David Martin: I don't know if there is one thing that I would say is like the thing to answer your question. Because I think what's exciting is that it could be so many things, but one that does jump to mind is about the ability for immersive experiences in general to bring people together who wouldn't otherwise be able to come together and to see each other in a context that might short-circuit some of the assumptions we have about other people that I think leads to a lot of the Tribal divides that we have at the moment I think like any technology it also has the possibility to exacerbate those but I really do think that the possibility for people to come together and around stories or around conversations and in the new kind of ways that immersive experiences will allow. It gives me hope to be able to talk about things in ways that we usually trip over ourselves or don't get out of our own sort of trenches about.
[00:42:55.683] Jenn Harris: Yeah, mine's short. You know, I would also like to say just delight. Just there's something about technology that just delights me to see something new or to see a way someone has created a VR to help tell a story or say something. Sometimes it isn't deep or doesn't have any vulnerability like I was talking about that tells a story or is expansive. It's just straight up delightful to sort of be able to fly or see things and just be able to imagine and then actually do it is really fun.
[00:43:26.601] Kiira Benzing: So to add to what Jonathan was saying about bringing communities together, I definitely think that there's new potential for that. The other thing that I think immersive storytelling can give us is a different way for us to understand space. So we don't need a proscenium stage built to experience an immersive story. It could happen in a house, like a piece that Jonathan has did. It could happen in a black box, like work that Jen has done. It could happen at a comedy theater, like Sam has done. And it could also happen in an art gallery, in a museum, in a school. There's so many possibilities. by using a virtual medium that gives us a three-dimensional world to enter into, and then what we can mix there. So here, we're playing so much with spaces. We have a physical installation here in Venice, but we've also built this virtual space. And then our audience is interacting with each other. There's also like a third space that's happening in between those things. And when we get that really right, we're just exploring that here as we stream between those things and work those screens and then that streamcast is like, a bunch of different things that we're juggling right now. As we continue to explore those things and harness the potential of that, I think we will hit into this other kind of space that we don't even fully understand yet. That's where I'm really excited. I think that goes beyond screens. It goes beyond the two-dimensional aspect of how we've been consuming content. So I think that's a huge potential.
[00:44:55.695] Kent Bye: Great. Is there anything else that's left unsaid that you'd like to say to the immersive community?
[00:45:02.103] Kiira Benzing: Awesome.
[00:45:07.227] Kent Bye: Great. Well, thank you so much for joining me today on the podcast. So thank you.
[00:45:09.870] Kiira Benzing: Thanks, Ken. Thank you. Thank you, virtual audience. Thank you, Mercer. Thank you, attendees.
[00:45:14.654] Sam Kebede: Yeah, thanks for listening. Yeah, thanks.
[00:45:18.988] Kent Bye: So that was Kara Bensing. She's the director of Love Seat, Jen Harris, an actor in Love Seat, also a producer and writer, John David Martin, an actor in Love Seat as well as a creator in Mixed Media, as well as Sam Kibete. He's the actor and the master of ceremonies within Love Seat. So I've had a number of different takeaways about this interview is that first of all, well, the thing that really stuck with me was Kira was saying that they actually had to do two productions at the same time. So actually have two different sound mixers mixing the sound for live performance, as well as mixing it within the immersive performance. And I think the takeaway from her after that was that it's a lot harder to do like a mixed media performance than it is to choose one medium over the other. Otherwise, you end up having to do two performances and they're all stacked on top of each other. And so the following year with Pandora X, it was just a VR chat performance. There was no in real life component. In some ways, that was because of the pandemic, but also it was just logistically easier for her. And so there's just kind of streamlining this, but also a lot of the narrative experiments that she was talking about here in this interview, she ended up actually performing and pulling off within Pandora X in terms of, you know, having different choices where you go off and. I actually have an interview with her about Pandora X that I did back in September, right after it premiered at Venice and the first time I saw it, and hopefully we'll be getting to that soon. But I think the whole idea of doing these live performances within these immersive technologies and be able to broadcast it out live is something that we're seeing come to a next phase with something like the Royal Shakespeare Company, because they have something that is a performance that also has elements of people just kind of passively being able to watch it through a 2D screen, but also different interactive elements. And yeah, I guess, you know, a takeaway that I have from watching that and these two different experiences is that the more that as an audience, you're able to have direct engagement and have some sort of improv aspects where you're able to engage with the actors in some way that gives it that live element that they were talking about this kind of vulnerability of the theater. And they ended up, implementing a lot more of that within the Pandora X, but the loveseat was just a passive theater performance where you're just watching it. And so whether you're watching it live or watching it within VR, there was no real interactivity or engagement from the audience. And so as we look at something like Dream from the Royal Shakespeare Company, they're starting to sprinkle in the little aspects of the interactivity, but. None of that interactivity is necessarily changing the narrative direction. So there's not a lot of narrative agency that happens. And within the Pandora X, there's actually a lot more choices that you make that you're able to actually choose what branch of the narrative you're going down. You're not actually in some sense changing how things are turning out, but there's a lot more collaborative interactivity that I think is really quite interesting. And I'll be diving into that more when I publish the Pandora X podcast. But what's also interesting is just, you know, I saw this piece a little over a year and a half ago now, and some of the stuff that I forgot. So there's a bit of my own archiving process by seeing an experience and then being able to talk about all the nuance of my experience within the moment. And then I'm able to revisit it like a year and a half later and kind of recall some of those different insights. Like, I don't remember being able to see all those different screens, but I was in VR. I was able to lift up my VR headset, see a 2D screen of the first person perspective of the different actors, and then also to look at their spatial relationships to each other in real life on the stage. And so I had like three different levels of this performance that I was kind of switching back and forth between, which I think was quite interesting. And it's also something that if we look at what Dreams is doing, it's able to take a 2D virtual camera and be able to capture something that's happening within the performance in order to broadcast it out to like five to 10,000 people. And there's a certain amount of limited number of interactive tickets, but there's these different tiers that you have people just watching a passive 2d video. Maybe you're able to have some sort of interactivity and then you have something like meta movie where your protagonist, you have the most amount of narrative agency, but you also have these fly bots are kind of flying around that are able to have more embodied presence within these virtual worlds. And then the next level up would be if you're able to kind of capture this in a 2D screen and allow people to watch as it unfolds. So I think that's kind of an interesting model in terms of having these different tiers. And we're starting to see that kind of get fleshed out a little bit more, especially when you have something like Dream. So look forward to talking to the team from the Dream project from Marshmello, Lois, and Feastin, Royal Shakespeare Company, to be able to dig into that a little bit more. But I think the question that comes up again and again and again with these different types of live theater performances is, what is the liveness of the live? And how do you interrogate that in certain ways? And whenever you start to scale up to be tens of thousands of people, then there's less and less opportunities for you to really interrogate the liveness of the live. You can see that it's perhaps a live performance, but you really don't know whether or not it's prerecorded. How do you ensure that it's a live moment? I think the best way to ensure that is that you have some degree in which that you're able to interact dynamically. So that's just some of my early take in terms of, as I've been seeing a lot of these different performances, that's a question that comes up again and again is like, how do I know that this is not just prerecorded? And how do I really interrogate the liveness of the live? And if you are convinced that it is live, then what is that element that makes it feel like it's a live performance that's distinct and different than all the other ones as well? And also Kira had said that, you know, it'd be nice to have some sort of place to be able to hang out and debrief. And she actually implemented that within Pandora X to be able to have kind of like an after party area to be able to just talk to the people as they went through it and able to share different aspects of their experience. And so that's, I think actually a really nice thing that something like sleep no more, where after you go and see it, there's like a bar to hang out in. And so there. a chance to kind of like share your own experiences of what you went through. So I think that's a really nice addition as well. And I hope to see a lot more of that kind of more casual offboarding that allows a social space. You're able to set a certain context and shared experience for everybody. And then you're able to then perhaps dive into that a little bit more afterwards. And the final point is that there was someone who said you can't lean on virtual objects. Actually, you can have passive object objects that allow you to stand up and then you're able to have that in real life, but also have the illusion as if you are moving around some of these virtual objects, which is something that Dream did quite well, actually, to have these the stage where they have different ways that you change different heights and just gives you the illusion that they're actually embedded within that virtual world. So that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening. It just gives you the illusion that they're actually embedded within that virtual... that they're actually embedded within that virtual world. That they're actually, oh my God. I just cracked this, this thing dropped and it cracked. My God. That's really annoying.
