Punchdrunk is a 20 year-old immersive theater company who likes to blur the line between what’s real and what’s an imaginal dream-like storyworld. They collaborated with Samsung in 2016 to create a blend of immersive theater and virtual reality in an experience called Believe Your Eyes. The Phi Centre sponsored a VR Gallery at Venice, which included VR experiences by a number of contemporary artists as well as Punchdrunk’s Believe Your Eyes.
I was able to catch up with one of the directors and co-creators of Believe Your Eyes Kath Duggan in Venice to talk about their experiential design process, some of the cinematic and embodied language they use to communicate with each other, and how they’re committed to putting the audience at the center of the experience while continuing to blur the lines between what’s real and what’s a dream-like fiction. She says that dreams are an associative, non-linear structure of experience that people are comfortable with the fragmented and cut-up nature of, and a lot of the work of Punchdrunk aims to try to replicate that dream-like quality of presence.
This conversation is spoiler-free for the experience of Believe Your Eyes as we try to abstract out the deeper experiential design principles that Punchdrunk uses.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So this is the last of a brief series looking at dreams, dream logic, and symbolic storytelling language of virtual reality. Today's interview is with Punch Drunks, Cath Duggan. She's a art director there, and there was a VR experience that Punchdrunk did in collaboration with Samsung. So for people who don't know, Punchdrunk does the immersive theater show of Sleep No More, which opened up in 2011. Punchdrunk's been around for about 20 years now. So they've been exploring immersive theater, putting the audience in the center of the experience. And so they collaborated with Samsung to create this VR experience. Samsung originally had it at Cannes back in 2016. It was in Montreal for a while at the Phi Gallery. They had it in Venice. It was in a gallery that had a lot of different other stuff that was curated by the Phi Gallery. So we'll be not really talking too specifically about the experience, but more about the broader experiential design principles. But if it does come through your town in some way, keep an eye out for Believe in Your Eyes by Punished Drunk. So we'll be exploring the experiential design process, this evolution of immersive storytelling and virtual reality with Punchdrunk, and exploring different aspects of dream, dream logic, and the spatial storytelling language of immersive experiences. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Cath happened during the Venice International Film Festival. It was not at the Venice Festival. The Venice Festival was happening mostly on the VR Island off of Lido. This was actually in Venice. The Phi Gallery had a whole location where they had a number of different experiences by contemporary artists that are pretty well known, as well as this piece by Punchdrunk called Believe in Your Eyes. And I did this interview with Cath on Saturday, August 31st, 2019. So with that, let's go ahead and dive right in.
[00:02:06.860] Kath Duggan: My name's Kath Duggan, and I am an Associate Director at Punch Drunk, which is a theatre company based in London. And maybe it's good to tell you about that theatre company, actually, then we can put the VR in context to that. So we have been going about 20 years now, so it's a while, and it was driven by the desire to put the audience's experience at the centre. of a theatrical experience and rather than have them sit back passively observing something to be up on their feet close to the action and to have a more visceral experience rather than sitting back and letting the brain process what they're seeing. The kind of work that's been happening over the last 20 years, and it's starting to shift slightly now, but a lot of it has been what we call our mask shows. So we take over a building, an empty building, and transform the inside with a cinematic level of detail, and transform it to another world, but a realistic one, so touch real. So if there's a table in there, you can touch it. If we're transforming it to 1950s America, you really feel like you're in 1950s America. and brilliant performers inhabit that world and the audience wear masks and are free to roam and free to feel. So what it's like if a performer, a character kind of whisks past you suddenly, how does that feel? How does it feel to sit in the corner and observe this world pass you by? How does it feel to rifle through? a cabinet that you're looking at the diaries of one of these characters. So yeah they've been the mask joes and then more recently we've been looking at ways to blend the reality and fiction and bleed them together. So if you were putting that world out into the real world and those rooms were scattered across a city, how could you not know the difference between a performer and a member of the public? So how suddenly can your perception of the world around you shift? And you not know what's fiction. So that's been the way it's been driving a little recently. And with the use of new technologies, experimenting with new technologies to try and find that point where fiction and reality blur.
[00:04:30.797] Kent Bye: Yeah, and I first saw a slip no more October 14th 2011 and so happened to be in New York City and it really Completely blew my mind. In fact, I saw it and then I like went home and stayed up all night Writing about my experience because I had to find the language to try to describe what I just experienced and I had noticed that people were Going through it and they were like, oh, this is impossible to describe just have to go see it so I feel like there's a lot of word-of-mouth and people really saying well, I can't describe what I experienced, but you have to go experience yourself. Then we can have this similar point to be able to connect about what our experience was in that. So maybe you could catch me up briefly into like the evolution of that and maybe even like what came before like influences that were kind of feeding in before Punchdrunk started doing it. I'm just trying to get a sense of the history or at least you know, the evolution of what we have today with immersive theater, which has really taken off since a lot of the work that Punch Drunk has done.
[00:05:31.202] Kath Duggan: Yeah, I mean, oh, placing that in context. Yeah, because of course we weren't the first to do that. It was, you know, there's a whole world of theatre, of happenings, of events that people would be swept up within and things that would happen within the real world, art exhibits that you didn't expect you would come across, you would stumble across and would happen. So I feel like I can only talk about it from our perspective and how we've shifted through that. And so Felix Barrett, who's our artistic director, so he was at university at Exeter and it was there that he started to kind of try to pinpoint how to engage an audience, how to startle them, how to take them along the side of a motorway service station and leave them there and see what would happen and then get the next bit of performance in. My background is in contemporary dance and I started with them as a performer for a show called Mask of the Red Death down at Battersea Arts Centre in London and a lot of the language within the show is a physical language rather than spoken text. So I joined the company in 2007 And I continued with them performing for a while and then started to do workshops and masterclasses and kind of the learnings and the techniques of performance that we found within Punchdrunk, doing that within an education setting. and I worked a lot with what was then Punchdrunk Enrichment and lead projects within schools, immersive experiences within schools that would pop up that they wouldn't know was theatre and would kind of blow their tiny little minds. And then I kind of started to make the shift more onto the directing side of things, working as a resident director on the show and starting to create The more we kind of had partnerships with different brands at the time and it was more of the experimentation side of things that we were doing. So it was the work with technology. So we were doing a project called Silverpoint that was a collaboration with Absolute Vodka to try and find a way to make a mobile app game become real around the city. and by playing a simple game of a saccharine, almost like Candy Crush kind of game and collecting points and then narrative bleeding out of that. And then those characters coming off out into the real world for live performance interaction. So I've just kind of progressed with Punchdrunk in that way, kind of starting to develop the ideas for shows and then direct them and bring them to life with a team of other people.
[00:08:18.059] Kent Bye: And I know that recently Punchdrunk had released like a whole encyclopedia. Were you involved with that project at all?
[00:08:23.189] Kath Duggan: I was involved with that project only in being asked some questions of my time with Punch Drunk and how that was and on an anecdotal level of things that I remember. But it's so great to read through things and read other people's experiences of it, other performers' experiences of it especially because it's people that I've worked with and just looking at their memories from those shows and those times. Yeah, really enjoyed reading it.
[00:08:49.595] Kent Bye: Yeah, because it seems like these performances, these VR experiences that can be so ephemeral, sometimes they'll show up and go away. And so just to kind of help document the oral history and evolution. And so I guess some of the big questions that are facing both VR and immersive theater is how you combine interactivity and agency with a story. And with VR, it's particularly challenging because, you know, you can't really locomote around in a way that you can like in Sleep No More. So I'm just curious how you think about this narrative and interactivity and sort of inherent tension that goes between those two.
[00:09:26.668] Kath Duggan: I think it was particularly interesting with the creating the VR piece because it's almost thinking about what the restraints of that medium are to then be able to create something within it rather than thinking about we want to make a piece of theatre and shoehorning in the advances in technology but really looking at if we're making a piece of VR what does that offer us and what are the parameters that that gives us to create within. So one of the first things when we were creating the VR piece was how the act of putting the headset on, actually it's not necessarily about what you see instantly, it's about the fact that you're not seeing everything else, that you're no longer in the world around you, rather than focusing on what the film will be, what the world that they see is. It's using the framework that that platform provides.
[00:10:31.362] Kent Bye: Yeah, when I was at the Immersive Design Summit talking to Colin Nightingale, his comment was that doing immersive theater, you know, you have a certain bar that you have in terms of experience, and that I think initially it felt like Punchdrunk was a little resistant to jump in and do a VR experience because, you know, this piece was released in 2016, and VR was sort of on the scene by 2013. So you have two or three years of development and experimentation before the first project was created. So maybe you could talk a bit about that time period from what you had already been doing, from Punchdrunk opening up in New York City 2011, and then this whole new technology coming up in 2013, and then how this project actually came about.
[00:11:13.039] Kath Duggan: Yeah. Well, it came about, we had a two-year partnership with Samsung America. And I think it was through these collaborations that, well firstly actually they showed us the technology and asked us to create something within that field. So that was great but it also provided us with the means to do it. and the reason to look at it I suppose and I think just before that it was those brand collaborations that I think did encourage us to look at the technology because it can be expensive ways to create theatrical experiences if you're using technology and so I think as a theatre company when you're looking at the economics of creating work it's not necessarily the natural way to go because of how much these things cost to create, or can cost to create, which I think is shifting nicely to make it cheaper. So we created the Samsung Astors, they were going to be in Cannes for the Lions Festival, in 2016 and they asked us to create a piece of VR for a house that they were going to host lots of people in and lots of different experiences in, so they asked us to create something for that specifically. So I think it was about a six-week turnaround from being asked to do it to the event being on and live. And then since then, this piece has kind of travelled to different people have wanted it on. We've been, Samsung took it to New York, we took it to Art Basel in Miami, we took it to several different places, and then the Phi Centre in Montreal had it on there last year or the year before, and the brilliant team there have brought it to Venice this year to bring it back to life again, which is wonderful for us to have it back up and on again. Yeah, so that's the journey of it and actually we did one version originally and then we kind of re-shot it and transformed it for something that could be seen by many more people within a day. And so that same version is the one that's been around and it's a piece that is a mixture of live action film and physical performer interaction so we come and find a local cast in the area and work with them to rebuild the show.
[00:13:47.115] Kent Bye: Yeah, and so I think there's this challenge I've found with you know, I know Noah Nelson from no proscenium he covers immersive theater and how I cover these VR experiences with VR it's a lot easier because a lot of the VR can get out there and people can see it so I would usually say go see this experience and then we'll like dive into the spoilers, but I there's other experiences where I kind of have to like keep cards close to chest as to what my actual experience was just because it's could potentially like ruin it for other people and so I feel like when I was reading about Super No More back in 2011 I was curious enough to see that some people felt like on their second experience was better and so that as well maybe if I do these things I can get a really rich experience the first time, but then I robbed myself of that sort of mystery. So I would never know what my experience would have been had I not known that. And I don't regret it, but at the same time, I feel like this is like a personal choice in this immersive world. The challenge of marketing to get the word out about something. like, this is good, experience it, versus, like, well, don't give out too much information as to, like, ruin certain aspects. And so, how does Punchdrug navigate this immersive world and trying to find that threshold of secrecy?
[00:15:02.260] Kath Duggan: I think it's a really tricky line to balance. Talking about this VR piece now, there is the frustration of not just being able to go, and then this, and describing the situation, describing the experience, and then being able to unpick it with you and talk about the details of it all. So it can be a frustrating thing from within. And always that weird thing of how much information you can give somebody and not, because you don't want to give, you don't want to, there was a tricky thing recently with a show we made with Cabayroy, where you, you kind of can't tell everyone what happens within, but you have to be sure that the, Ready or you've given warning of things that might happen, but by giving warning you're raising expectation That we really don't want to raise and also these things might not impact these people in this way so you can say something is Like you don't want to say something would be You don't want to say don't enter this experience if you have a heart condition because you imply that it will be terrifying but at the same time we have to be cautious and warm people in the right way and So it's a really tricky balance to tell people what the experience is without giving away the spoilers, but like you say, it's the word of mouth that really helps us. It's the individual audience experience that's at the centre of all of our projects, that that individual audience will tell another individual potential audience, and that's how the word of the show can spread through that.
[00:16:38.393] Kent Bye: When I see what's happening in this immersive space, I've been covering it since May 19th, 2014, but my origin point I really put in that October of 2011 where I had saw all these people who had the experience and they're like, I cannot describe this in words, just go see it. And my reaction was, that's not good enough. And I was like, I'm going to try to find a way to find the words. And oddly enough, as I've been doing the Voices of VR podcast and talking to creators, this happens to be still a pretty huge open problem, which is like how to have a direct phenomenological experience and then to be able to describe what your personal experience is, but also the experiential design components of what we're able to actually modulate your consciousness in a consistent way, like a consistent experience across many different people. So it feels like as we go through these immersive theater experiences, there's like a language that's being cultivated that it's almost like this embodied poetry like you have a metaphor that oh it's like sleep no more and then you understand oh it's like all these different elements that you understand once you see it and it's like movies got to that point where it's like oh this movie is going to be like this movie plus this movie and then that becomes a shorthand so you have these references and symbols that you can get through these embodied experiences but Still at the same time, just within the context of an individual experience, I think it's still a challenge to essentially break down all the different components of the human experience to be able to talk about what your experience was. It feels like there's this language that's being cultivated, and as creators, I'm imagining you have to cultivate your own language within your company, so I'm just curious to hear a little bit more about that process of talking about phenomenological experiences and experiential design.
[00:18:18.587] Kath Duggan: I think it's actually can be quite prohibitive for an audience member coming in when they already think that there's like a word and they understand that particular word in a particular way so like immersive and like what is an immersive experience and it being such a broad category now that everything is put within that it kind of drags an audience towards or I think at least it kind of drags an audience towards thinking it's going to be a certain thing before they even experience it. And so the word itself is dictating how they feel about an experience. And how we talk about it a lot internally is through filmic language. The shows, it's a lot about close up, wide shot. Often if we think about an audience's journey, it's almost like a tracking, shot through space and how the performers relate to that, how we can encourage them at this particular point to have a wide shot and how we can encourage them to see something in close-up and see the details. So we do talk a lot about with filmic references. I mean we're always discussing it from the audience perspective. It's always about them and what they see and how they feel and quite often if we're working on a new project it'll be about what an individual audience member might see and feel and experience within a space. So it's often from the one person. And then in terms of performers within this work, there's a whole other bubble of language in the way that they communicate together. And of course, it's people that are from physical backgrounds and physical practices. So it's often not well-constructed sentences that they can communicate in. It's the sounds, it's the... and the... the feelings and the way they'll, like, squeeze their hands to describe something. It's how meaty something is and how dreamy and ethereal and light it might be. And it's words like that that the performers would use within something.
[00:20:35.977] Kent Bye: Well, it's interesting to hear that you think of the audience member as a camera that's moving through space because in both Unity and Unreal Engine, real-time game engine flavor as opposed to the 360 video, you're literally a camera that's moving through space. So it's like quite a literal translation. But I sort of see at a high level that there's yang and yin sort of metaphors. So yang being ways that you can express your agency, make choices, take action, kind of like a video game. And then the yen is more the passive receptive modulation of building and releasing tension, narrative tension or with music. And also there's a certain level of embodiment that you're receiving through your sensory experience. And so because in Sleep No More you're a ghost and you have no real agency that impacts the story, you have the freedom to be able to have a very controlled narrative that isn't being influenced by the audience members in any sort of interactive way. that's a good first step and where we're going feels like the live-action roleplay where people have the ability to Speak and be a character and you know the whole Westworld version of where things are going. I think there's a lot of competencies that the audience has to cultivate to even get to that point to really understand the dimensions of narrative and narrative tension. But I see it as this kind of like tension between like to what degree does the audience member, are they a ghost or are they a character? And also how much that you as a person being there is impacting the story. So that's at least how I start to think about it in ways of in VR seeing how still trying to like combine these two realms of like the video game world with the film world as well as like putting a sense of embodiment and the sort of more contemplative embodied experience into the whole mix as well.
[00:22:20.284] Kath Duggan: Yeah, it's interesting because lately, so even thinking about who the audience are at the centre of something, so like you say in Sleep No More, for example, they are ghost-like creatures that inhabit the space and they observe but they don't have an impact upon the story and the difference between those audience being observers to other shows that we've done where the audience don't have masks and they are placed as the lead character within that journey and people are addressing them as if they are that lead character and they're led through the journey as if they are and not that they have an impact on the way that things will turn out but they even then feel like they are more active within the narrative because they are being addressed. And I think because our work is a lot about the detail and the controlling of the environment and controlling what the audience will see, In the stuff so far, I think that's the reason to not have them impacting the show. So like, it's been quite interesting making this VR experience of how we could direct their attention or divert their attention within the piece and how you could make them feel as if they had complete freedom. but eventually by dictating where they look and what happens that you're kind of shutting that field of vision down so that we can control exactly where they're looking all the time although they feel as if there is the complete freedom. So it's through choice that we do that so that we can control every moment that they are seeing rather than let them impact upon It feels like a different form, a different style of narrative than letting them impact upon how the narrative unfolds through choice.
[00:24:28.180] Kent Bye: I'm wondering if you could talk a bit about your first experiences with VR as a medium.
[00:24:34.003] Kath Duggan: I'd watched very little, so I made this show with another associate director, punch-drunk Hector Harkness, and both of us actually had very little experience of watching VR, of experiencing it or watching it. And a lot of the pieces we'd seen were quite fantastical things, like doing the impossible almost, you know, flying through the sky, soaring through the sky like you're a bird and you're looking down on the world. or, well it's not impossible, but like being on a roller coaster. And I think that was one of the things that kind of drove us towards trying to do something that was more realistic and more intimate. And like I said earlier, it was a lot about noticing the things that for us felt wrong. about like felt like the things that were pulling me back to reality when I was meant to be immersed within a VR experience. So the fact that I would look down and it's a different body really I found it quite disturbing somehow that it doesn't look like me. or that I would perhaps be watching something that I didn't realize was made to be viewed standing and I was sat down and I would move my legs and they'd go through a chest of drawers and that I found like not being solid within a an alternate world kind of I found quite difficult to make it feel like I was really there or inhabiting this world of VR. Yeah so it was weirdly these and I think it was also the strange thing of putting it on. I'm so used to working in live theatre and being present with people in the room that you're working with either be it on stage or the audience being able to see them and it was quite a strange and actually quite vulnerable sensation of putting the VR mask on and feeling exposed to the room in a different way that I felt like, you know, you kind of feel like people could easily prod you or be laughing at you or, you know, a whole other world going on in reality in the room that you can't see. And it felt quite, the act of putting the VR headset on felt, made us feel quite, both of us quite vulnerable in the space. And I think that was one of the key driving factors for us was that feeling of the audience in a space with a headset on, how that makes you feel. And I think when you're doing VR, you get really used to that. You get really used to just putting it on and being in another world and everybody else going about their business around you and it feels quite normal. But I think when you do it for the first time, it can feel quite exposing.
[00:27:21.175] Kent Bye: So, in covering the Venice Film Festival as well as Tribeca and Sundance and South by Southwest, there's a number of themes that I see emerging in terms of trying to give metaphors for what immersive entertainment is, whether it's immersive theater or virtual augmented reality experiences. people compare it to memories or a psychedelic journey or a dream or like an initiatory ritual or rite of passage. So you have in film, we have a certain language that's developed, which is a cinematic visual storytelling language that's taken over a hundred years to develop. But I think of people like Carl Jung or Freud who are looking at the interpretation of dreams and seeing how there's a whole certain level of dream logic that's used that doesn't necessarily fall like rational thinking, but it's something that's more akin to a dream. And something like Bodyless here at Venice this year explicitly uses certain aspects of dream-like metaphoric mythological stories. The key director, Celine Tricart, is a lucid dreamer who uses dream logic as a metaphor. Jordan Peele's doing a lot of metaphors and us and Get Out, where it's a lot of symbolism. So I feel like there's a sort of movement towards dream logic and the language of dreams and memories that is this overlap to immersive theater and there's this challenge between a personal symbol and a universal symbol and trying to translate a story into something that goes beyond just the personal that people can kind of project whatever they want onto it but not be so arcane as to not understand what it might mean. And there's a mystery there. like a dream might be mysterious and may stick with you, but I'm just curious to hear a bit about how Punchdrunk thinks about this sort of grammar and storytelling language of spatial storytelling and immersive entertainment.
[00:29:05.434] Kath Duggan: Yeah, I think I can understand why people latch onto the dream landscape somehow because it really allows a fit with a non-linear experience. You know, we all travel through life in our own linear journey. And I think it's, for a lot of people, dreams, I suppose, are the only experience that feel kind of cut up, feel fragmented, and feel quite disparate. But also, it feels quite normal to them. They talk about it in a normal way. It's not unusual. I mean, of course, the way they describe the dreams and the things that happen within the dreams really are unusual. But they aren't necessarily perturbed by that. Whereas some people coming into an experience, if it's not linear, can feel perturbed. And I do think it fits quite well to be discussing these things as, like, dreamlike. Things that you can't quite grasp. And I think, for me, it kind of feels like it might help some audience members somehow because they're not hunting for the logic and they're not hunting for complete clarity of narrative and piecing everything together to gain a complete picture. If they understand that it's their own perception of these things and how they piece these things together, then it can give them a bit more freedom. That being said, it's the difficult thing that we also really want to make them believe it as reality somehow, because it then has a much higher impact. So if there's a way to allow them to be happy with the fragmented journey and still be in reality, then it'd be a win.
[00:30:51.434] Kent Bye: Yeah, it makes me think of how the Greeks had two words for time, the Kronos time and the Kairos time. And the Kronos time is that linear time. It's like the unending fatedness of Saturn eating its children. And it's like this fate that's bestowed upon you that you can't get out of. The Kronos is much more the quantification of time. Like, what time is it? Well, it's 1059. Or the Kairos time is much more the quality of the moment. So it's the right moment to act and that's more nonlinear and cyclical. So it's like thinking is patterns of cycles that are repeating and so you have returning to moments that are connecting you to memories and associations and it's also like when you're on vacation and you don't have a plan and you just sort of go out there and you have these series of synchronicities where your internal state gets matched to the external reality of what you're experiencing. And I feel like a lot of what I wrote about when I saw Sleep No More back in 2011, as I said, it felt like this initiatory ritual that was trying to cultivate this synchronicity of being able to make choices but to have things happen that you read deeper meaning into and you have this deeper flow state of like feeling like you're in the flow of the quality of the moment. So I feel like there's this sort of Kronos, Kairos, the linearity and non-linearity that I feel like at the heart of is like almost like cultivating our skills to be able to really get into those Kairos moments.
[00:32:10.848] Kath Duggan: Yeah, and it's really difficult to be open to those things and how you're feeling that day, how your day's gone, means you're going to enter an experience with a very different mindset. From one day to the next, you're going to be open to seeing how the journey goes and another day where you're feeling more uptight, maybe you're not going to be open to those experiences. But yeah, it's a really nice way of seeing it with those two concepts of time.
[00:32:41.597] Kent Bye: It seems like people's perception of time changes when they're in these experiences. They either think it's a lot longer or shorter than it actually is.
[00:32:47.722] Kath Duggan: Yeah, totally. And also, it feels like there's something about taking moments of experiences out into the real world that can really shift that concept of time as well and shift the perception of how the real world is moving around you. And weirdly, when these interactions happen within the real world, it's almost like the real world around you becomes more dreamlike. So it kind of somehow flips from within the show and you start to look at people and the way they move. And I think time slows down slightly because you start to look with more clarity. Yeah.
[00:33:32.668] Kent Bye: Yeah, it feels like that that's a big part of the immersive theater experience is sort of the modulation of your experience of time and that's a big consistent thing as I've been doing a lot of VR and sometimes people will have an experience in VR where they play for 12 hours and they think it would been like two hours so you have this time dilation effect that happens and so just kind of studying what is at the heart of that but I feel like that it seems to be a pretty consistent experience of the immersive theater world.
[00:33:55.743] Kath Duggan: Yeah, it is. And it's that thing of having eye contact with someone for a moment. You can't tell how long that moment is. And maybe after you've seen a show for three hours, you've been in there for three hours, and the moment that feels the longest is that one blink where you had eye contact with one performer. So it really can stretch and play with time in that way. And I don't know if that is... I need to experience some things and think about that more, if that is a thing that is consistent with immersive theatre or not. Yeah, I need to think that if that is a definite connection.
[00:34:37.401] Kent Bye: To me, I think the connection is, if it's a novel experience, you sort of enter into, like, we have category schemas, we have a predictive coding model of neuroscience, where we have a repository of things, where we project these models onto the world, and if it goes beyond our model, then we have to do this error correction code in our brain, which then releases endorphins, but it also, I think there's a connection there. If you have a lot of novel experiences that you've never had before, then your brain is doing extra work and then it sort of loses track of time because it's trying to create new models in its mind. And so you're kind of putting people in this state of awe and wonder in a perpetual way that they can't deny the reality of this experience and they have to find some way to make sense of it. So I think through that process, it invokes all these senses of awe and wonder and time dilation and yeah, just kind of modulating your experience of time.
[00:35:24.803] Kath Duggan: Yeah I mean for sure that's true and I suppose it depends as experiences become more and more immersive experiences are more commonplace within theatre it's not always a novel experience for people and I think that's why Time might not have that, but for sure it's the continual new experience of new things and new process. The time shifts dramatically. Your brain is working to understand. Or your body is. I feel like it's like a visceral process somehow that you allow your body to understand things. or they are dealing with these things, of course it's your brain really working away, but it kind of somehow feels like your whole body is processing these moments.
[00:36:15.995] Kent Bye: And so for you, what are some of the either biggest open questions you're trying to answer or open problems you're trying to solve?
[00:36:26.485] Kath Duggan: One problem or thing to solve is that as we use these technologies, how we can continue bleeding them, like cross-fertilising them with theatre, so that if we're using a mobile phone, actually how can we not get drawn into the mobile phone but how can we force audiences eyes up into the real world as they're doing that. So I suppose it's as we keep progressing with the use of technology I think it's continually playing with these technological advances that we cannot get too drawn into that and remember the skills that we have and how you can keep looking at the technology afresh by using the theatrical tools and skills we have rather than just getting drawn into what the technology is currently used for and does.
[00:37:24.605] Kent Bye: Yeah, and finally, what do you think the ultimate potential of immersive experiences might be and what they might be able to enable?
[00:37:35.170] Kath Duggan: I still kind of really feel like it's allowing the audience to see the world differently. to be able to step out of an experience and look at the world around them and the people around them through different eyes, even for a moment. Because I think by looking at people and the world around you through a different lens, even for a moment, you'll have a different connection with that world and behave differently within it. So I think, for me, it's all about how an audience, through those experiences, can look at the world around them differently. And I think the potential of that, in the same way that I don't think it's exclusive to immersive experiences, but I think that they can really push people to do it in a quite sudden way, somehow. But I think by the experience of looking at the world around you differently, will allow people to see what's really happening in the world around them and interact with it differently. I know that sounds really vague and wide but I think it can allow for different personal human interactions after having an immersive experience by seeing the world differently.
[00:38:59.479] Kent Bye: It's a common theme in VR to say like it's blazing new neural pathways in and so once you take the headset off then you're perceiving the world in a new way. So I think it's very similar. Is there anything else that's left unsaid that you'd like to say to the immersive community?
[00:39:14.357] Kath Duggan: I'm just very curious to see and watch where it goes, especially VR. Not knowing an awful lot about the world, really curious to see what happens when artists that don't know about VR and haven't experienced a lot of it interact with the technology for the first time to create new things.
[00:39:35.535] Kent Bye: Awesome. Great. Well, thank you so much for joining me today on the podcast. So thank you.
[00:39:39.378] Kath Duggan: Pleasure.
[00:39:40.850] Kent Bye: So that was Kath Duggan. She's an associate director of Punch Drunk and we were talking about Believe Your Eyes. So I remember different takeaways about this interview is that first of all, Well, this was a very amazing experience. Uh, I wish I could just like tell you everything that happened, but I think it's just better for you to try to see it yourself. But for me, I just felt like this state of all in wonder and a number of different ways. And, you know, it was just interesting to talk about the design process for Cath because coming as a dancer, working with an immersive theater troupe, they're really trying to break down this fourth wall of this separation between the audience and the actual experiences. starting with a lot of the immersive theater type of experiences of sleep no more where you are a ghost with no impact and You're in these masks with these mass shows walking around seeing these different immersive theater actors kind of run by you It sounds like they use filmic language to be able to describe different aspects And so you get like the wide shot to kind of establish new context and then the medium shot and close-up or the close-up is you really just you digging in and being able to open up different drawers and read different letters and for you to really maybe see an interaction between two characters and use those types of like filmic metaphors to be able to describe the audience journey because they're really thinking of the audience as this camera that's kind of moving through this space. Now, I think some of the other experiments that they've been doing with some of the other immersive theater shows where they've started to actually put you out into the world and then experiences that are really designed for one person, you start to have these interactions with people within the public where it sounds like this vodka app where you're walking around the locations of this place, but then these people are coming up and interacting with you and you don't quite know who's an actor and who's not. and they start to blur the lines of that reality. And so we were talking about this dreams and dream logic and how you kind of entered into this kind of altered state of paying attention. And usually when you're walking around, you have like these category schemas, you know what to expect. And then in this type of blurring of the line of what is reality, you start to have this experience of you not quite knowing what is real. And it kind of like starts to make you pay attention more is what she's accounting here. which is you starting to just notice things you didn't notice before and she wonders if that's kind of a universal experience within immersive theater as you go and have all these different novel experiences then you start to dip into this luminal space where you don't know quite what to expect and you feel like you're walking through a dream. And I think they really like those dream metaphors, especially like my experience of Sleep No More very much felt like I was walking through a surrealistic dream where there's all this embodied interactions and it has deep metaphoric meaning that you can get a sense of what it means. But it also, maybe you have like a symbolic action that's happening in front of you and it's actually really reflecting what you're feeling inside and you have this moment of alignment of what you're feeling inside and what's happening outside of you. the deeper you get into these deeper flow states of immersive theater, then you have those experiences of those synchronicities and those different qualities of those experiences of time. And then she was also reporting that people were experiencing the opposite effect, which is that you're out in the real world. And when you start to have those different types of interactions that are perverting your expectations, then it also makes you look at the entire reality as if we're living in a dream. Which I think is an exercise that you can go through and start to interpret everything that you're seeing as a dream. And then really start to question, like, is your waking life the real life or is your dream life the real life? And then your waking life is just the dream, which I think is fascinating to think about. what's more real, the dream or the reality. And I think Punchdrunk is at that intersection of trying to really blur the lines and really make you question what's more real. As I was talking to Celine Tricart, she said the brain is actually kind of like experiencing things in a way within, you know, for her as a lucid dreamer, where she can't actually physically tell the difference as to what's real or what not. And she has to do all these different types of reality checks. And so maybe that's the process of immersive theater as we move forward is we're all gonna be kind of walking through the world as if it was a dream, and then to start to have to do these reality checks to be able to check, are we dreaming or not? So the final thought is just how in the future, a lot of what you go through in the immersive theater world is on the rails. You can't impact the unfolding of the story. Logistically, it's very difficult to think about the branching narratives or the improv or what kind of structure you'd actually have to have in order to allow you to be a full character. And then we're getting into the realm of live action role play and the cultivation of the skills of the audience to know how they kind of fit as a character and who that character is and how that's defined. And it's much more easy logistically to be able to experiment with these live immersive theater shows where you have no impact. And they're starting to play with different ways in which that you're being addressed as an individual versus you being an anonymous masked ghost character where you're just doesn't really necessarily matter that you're there or not. They are pretending that you're not there. So the action is gonna unfold whether or not you are witnessing or not. and some of the directions that they're going with some of this more bespoke individualistic way that you're actually the center of the character addressed as an individual. And even though you may not be able to impact the outcome of the story, as we move forward in the technology, I think it's actually going to be easier for artificial intelligence and other ways within the simulation of a game engine to be able to allow people to experiment a lot more. and then maybe there'll be this dialectic process to kind of figure out what those narrative structures are and then to find ways to feed that back into like a live immersive theater context. You can do a lot with a small scale with just a few people and be able to track it. But once you get into like dozens or hundreds of people, then how could you reasonably expect that the audience is going to somehow shift the overall outcome of the narrative, especially when you have that asymmetry of the number of audience members versus the number of characters. So I think you could start to maybe play around with that a little bit within immersive simulations of virtual reality. But I think that's kind of like the bleeding edge for how could you actually have a group participatory process where the agency of either an individual or a group could aggregate to be able to actually change the outcome of what actually happens within a narrative. That's something that's really on the bleeding edge. Maybe we'll see that much more into the realm of artificial intelligence and different models for doing that. And there's some other AI experiments that were happening both at Tribeca and South by Southwest and here at Venice where they were starting to experiment with different levels of artificial intelligence. But that's a whole other separate series to kind of dive into what's happening in the realm of AI with this interface with VR. So that's all that I have for today. And, um, thanks for joining me on this series of looking at dreams, dream logic, and this symbolic spatial storytelling language of virtual reality. This was a fun little short series to kind of dive into and, and themes that I think came up between what I was seeing at Sundance and South by Southwest and Tribeca and Venice, especially at Tribeca and Venice with these specific experiences that were very explicitly using different elements of. Dream logic or dreamlike qualities or you know symbolism within spatial storytelling and playing with that in different ways So I will be heading off into Oculus Connect 6 and then you know some associated events after that I'll be going to the SVPR meetup that's happening on Tuesday night and then I'll be at a the indie dev night, I think on Thursday night, and then the San Jose State University on Fridays, having a whole day long virtual reality connect. I'll be giving an opening keynote there. I come back home and then I fly out to Amsterdam to go to the Vue Source Conference in Amsterdam, where I'll be recording some podcasts with web developers and doing a live recording of looking at the future of WebXR. So I'm excited to be out there next week at Oculus Connect 6. If you see me, track me down, love to connect, see what you're up to. And yeah, I'll be back in about a week from now, kind of dive into the content that I capture there this coming week. And if you enjoy the podcast, there's a couple things you can do just to support the podcast. One is just spread the word, tell your friends. I really rely upon word of mouth to help get the word out about what I'm doing here at the Voices of VR. And this is a less supported podcast. And so I do rely upon donations from Patreon to continue to bring you this coverage. And if you've been enjoying these deep dives and series, then really consider becoming a member of the Patreon. I do rely upon donations from listeners like yourself in order to continue to bring you this coverage. Just $5 a month is a great amount to give and just allows me a foundation and baseline to be able to continue to do a real-time oral history of the evolution of spatial computing and immersive storytelling. So you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
