I interviewed The Imaginary Friend director Steye Hallema at Venice Immersive 2023 after passing along a recommendation to watch my Storycon keynote on Presence, Experiential Design, & Immersive Storytelling. We do a deep dive into applying some of these ideas around the different qualities of presence to his project. See more context in the rough transcript below.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. It's a podcast about immersive storytelling, experiential design, and the future of spatial computing. You can support the podcast at patreon.com. So continuing my series of looking at different experiences from Venice Immersive 2023, this is episode number 33 of 35, and the first and only one on the context of community and friends. So this piece is called The Imaginary Friend by Steye Halema. This is a really ambitious piece that is trying to integrate lots of different components of virtual reality. So it's in the contextual domain of friendship and contextual domains of death and family and in response to bullying. And so the primary center of gravity of this piece, I'd say, is mostly in this field of mental and social presence. And so trying to cultivate this interaction of creating this pseudo relationship with this volumetrically captured character, you are embodied as this imaginary friend, you are his imaginary friend. And so there's different moments where you're asked to speak and communicate. It's not actually interpreting what you're saying, but it's trying to give you this feel that you have this interactivity and participation with communicating in this social context. And it's also got this mental presence dimensions where there's certain aspects of the narrative that are kind of happening off screen and you have to kind of piece together different dimensions for what leads to the next in terms of what the overall arc of the story is. And we start to unpack some of the nuances of that story in the context of this conversation. But in terms of all the different experiences at Venice immersive, there's some of them that are trying to check all the different boxes of all the different quality of presence. But this is probably the most ambitious of trying to integrate and fuse them all together. And I think some of the friction points that I had is some of the ways in which that some of these aspects of agency and interactivity aren't necessarily always lining up with different dimensions of the volumetric capture. that he's created here. But the secondary mode of presence, I think, is more of the emotional presence in terms of the story that's being told of trying to have this really well acted volumetric capture of this character of the kid throughout the course of this piece. And so telling the story of him and dealing with the grief of his mother, trying to deal with bullying, and so just the story that's being told there. And then there's this aspect of embodied presence where you're actually embodied into a variety of different types of embodiments throughout the course of this piece. And you actually get mirrored at some point by the main character. And so you see what your embodiment fully is when you see the other character that is mirroring your embodiment as this kind of imaginary friend depiction that is kind of this bird-like creature with wings. And there's a number of different types of cut scenes that go away to more explicit game-like components where you kind of have to bat out enemies that are coming at you. And then there's another component where you're kind of flying above and you have to shoot off these other enemies. So there's more explicit game-like components, but more of interactive and participation rather than actually more gameplay loops where you could actually fail since it's more of a narrative driven type of experience overall. And Steja is a creator who's trying to think deeply about all these different aspects. And as soon as I told him about my talk that I had given at StoryCon and recommended that he check it out, he immediately went to go watch it. And so he wanted to have an extended two-hour slot to dive deep into more of the theory and practice of experiential design. So this is one of the deeper dives that I do over the course of my interviews at Venice. Also digging into my own thoughts and reflections on experiential design. And so I would recommend folks to actually go and watch the StoryCon talk that I gave last year in Belgium that I think is one of my more comprehensive takes of talking about all these different qualities of presence and the kind of more philosophical aspects behind it. I've been talking a lot about the different qualities of presence. There's also the context. I've been organizing all the different selection this year amongst the contextual domains. And then on my website, there's actually a categories page that's starting to flesh that out across many different applications of virtual reality across these different contextual domains. You can go check out that to dig into more of that. And hopefully I'll put out some ways of kind of navigating how the different themes are resonating across the series. But it's also the story and the character. And I think in episode 1305, when I do the critics roundtable, the other folks that I invited in there, Paula, Alina, and Agnieza, they're all very much primarily focused on the stories and the characters when they watch a piece that's their first filter as they're looking at the story that's being told and oftentimes when they're explaining and deconstructing a piece they're going to be unpacking the different aspects of the character in the story whereas for me my first filter is much more in the mechanics of looking at the different elements of embodiment and environmental presence and mental presence, social presence and active presence as well as the emotional presence and all the different mechanics of how those things are all kind of interplaying together. That's sort of like my first pass phenomenologically as I'm going through experiences and how I'm remembering and describing these different pieces. But for other people, they're looking at other things. And I think that's one of the big reasons why I always love to have like a bit of a critics roundtable just to get other people's takes on some of these different experiences, because there isn't a lot of deeper critical discourse that's happening in other contexts for this work. And so I just want to provide a forum to have these types of deep dive discussions like I have in episode 1305, as well as with this conversation with Steya, where we do a deep dive into his piece and also talk about where he's going to be going in the future. So that's what we're covering on today's episode of the Voices of Yara podcast. So this interview with Steya happened on Monday, September 4th, 2023 at Venice Immersive in Venice, Italy. So with that, let's go ahead and dive right in.
[00:05:38.705] Steye Hallema: All right, I'm Stij Halema. I'm a VR creator here on the island of the Festival of Venice for the Imaginary Friend, which is a VR piece. Next to that, my biggest project is the Smartphone Orchestra, which is, I would say, that's a form of immersive storytelling, too. We synchronize phones and then we tell stories with the audience, with big groups of audience members, and immerse them in kind of like group interactions, social interactions. Although these are quite different technologically, for me it's the same thing because it's kind of like figuring out the language, how to tell a story that involves the person undergoing the story. It's really about doing it right on all layers. That makes serotonin in my brain, if it works.
[00:06:25.883] Kent Bye: Maybe you could give a bit more context as to your background and your journey into working with immersive media.
[00:06:31.527] Steye Hallema: Okay, that's interesting. There's a couple of lines going into that. I think, first of all, I have been a musician for a long time. I also studied music. I did the study, which was actually a combination of the conservatory and the art school. And I feel, for some sort of reason, if I'm playing live, I'm always really aware of the audience. And I think a good gig is a musician having its own material. And the audience have against audience energy and a good musician really makes something out of both. So I've always really like grown a sensibility to the audience. Also when I still play live for example with my own band I'm always thinking of stuff that I can do with the audience to create something together. Next to that I have always been part of my friend group in my youth. We couldn't play soccer, so to impress girls we took the creative route and we became some sort of weird multimedia theater group. And quite early, we had a dogma that we wanted to unsheep the audience, probably also because computers came in. We were kind of like the first people on the stage that would really use the computer on stage and see how we could use the audience in our place. For example, we made a piece called Wartel Kombat, which was a spoof of Mortal Kombat. Wartel means carrot in Dutch, so it was carrot combat. And what we would do, we would make games live on stage, video games, but then, like with real videos, we would film the audience in certain positions and they would become characters in the game. And we would divide the audience, for example, in half and they had to, like, both were teams and they had to fight for the outcome of the game. So, quite early on, we were really trying that. Also, in 1999, one of my good friends in that theatre group, Kees Duijf, he had this luminant idea of filming a Chinese massage ball, which is shiny, reflective. ball from the top, and so we could film all around us. And we managed to unfold that in After Effects and then make it interactive with a micromedia director. And so we could basically look around in a clip. And that, so already in 1999, made our brains go like, oh, wait a second, how can we tell a story with that? Then later, I found a way to make the quality of that better, basically by just putting my then still SD camera on its side and filming pizza slices all around, putting that together in After Effects, which is actually software to animate. And then I would make clips that you could scroll around. And then at a certain point, I found the Oculus on the blog and was like, wait a second. And I didn't get it. And then I bought it, made it work on my computer, took me about two weeks. And then something really special happened because I was standing inside of my own material. And that blew my mind and then I completely changed the script. And in that clip I really thought about how would I do transitions. And I found a way to do a transition with a hula hoop. So I would just put a hula hoop over the 360 camera And then that would be like a perfect transition because actually someone's coming towards you, pulling hulup over you. And it was also a spatial transition. So that really worked and then when it came out in 2014 it was by accident one of the most advanced VR videos. Which I didn't know because I was just making what I knew. And then I first got a job at a broadcast company in Holland at a media lab. And then Jont, which was a big company, Jont VR then, I thought I would be good to be creative director for them. So I became a creative director for them and then it really gave me the chance to make a lot of VR pieces. And when Jont stopped, I just went on.
[00:10:26.880] Kent Bye: Yeah, so we had a chance to catch up at Tribeca, where you were showing Emoji with Three Eyes, a smartphone orchestra experience that was a lot about cultivating social dynamics. And we had a chance to go through the history of the smartphone orchestra at different pieces that you had created. So had you been creating 360 videos in VR before the smartphone orchestra there, just in terms of the timeline? Or did the smartphone orchestra start before you even started into VR?
[00:10:52.803] Steye Hallema: No, the smartphone orchestra came later. Also because smartphones came later for me. Well, like when I was doing VR, to be clear, I didn't know what VR was. I discovered what VR was, I think, I think, yeah, 2014 somewhere. Yeah. And I had the idea. So I was already doing VR. I'm doing quotation marks with my fingers for everyone who's listening. I started kind of doing VR since 1999, when I was, I don't know, late teens or something. And the Smartphone Orksta idea really was also, it was kind of 2013, I think when smartphones became so commonplace that it started to be a thing, that everyone was just staring at their smartphones. And when I realized, wait a second, there's a speaker in the smartphone, and if we could synchronize the phones over the internet, we could do something really special. So Smartphone Orksta came later. But for me, in the sense of doing the Pips Lab, the multimedia theater group that was really about interactions with the audience, it's all the same thing.
[00:11:50.073] Kent Bye: So when you say that you've been doing quote-unquote VR since like 1999, do you mean some of the experiences that were interactive with your theater group or were there other things that you were doing that were trying to create like these expansive video experiences that you had to scroll around or what do you mean exactly by doing VR for that long?
[00:12:08.987] Steye Hallema: Well, like what I just explained, the way we found a way to capture the 360 surroundings, we would play with that since 1999. So we would, for example, have it as a start of the show. People would walk in and we would have this camera next to them and they would see themselves like really weird blurred and would turn around them just on stage. I made a music video in 2009 when I did a project in Canada. I was actually making an album there. And then I thought like this is the perfect way to show what I'm doing here. So I made a music video kind of documentary 360 video that really showed how it was in Canada. Then in 2011 I made a much more elaborate 360 video for my music. where I already started to think about, if I'm on this side of the camera, how do I interest the spectator to look at the other side? And still, this wasn't VR VR, this was really scrolling on your monitor, because that's the only thing we could do, because I didn't know of the existence of virtual reality headsets. And it came only like three years later that it became a big thing when Facebook bought Oculus. But when I made that clip, for example, I would stand in one corner holding a rope for like two minutes. After two minutes I would let it down and then on the other side a guitar on a rope would come down and I would come out of a closet and grab it. So there were like 15 times me. The shitty thing was, sorry for using that language, that when it came out, just the day before it would come out, Apple pulled the plug from Flash, and it was built in Flash, so we couldn't show it. And then I tried it again in 2014, and then I found the Oculus, and then all of a sudden, it hit me that I had to do VR for the rest of my life.
[00:13:49.849] Kent Bye: Yeah, so Jaunt was one of those 360 video companies where they're actually building their own hardware and also trying to, in some ways, create their own original content and distribution platform. But I feel like in the early days of VR, it was mostly very focused on gaming and real-time engines with Unity and Unreal. But eventually, in the ecosystem, there was GoPro that a lot of people were making stuff. And then there was also Insta360 entered in the scene at some point to do more of a consumer-grade 360 video. So after you left Jaunt, was there any other VR experiences that you were creating before you worked on Imaginary Friend, or was it mostly the Smartphone Orchestra before you come back to start to make VR again?
[00:14:29.983] Steye Hallema: Well, I've always been doing my own stuff. Also next to Jaunt, I made an opera, a VR opera, which was actually, I really liked working on that project. And this was, the audience members would come in, and it would be for, I think, 60 audience members, but 30 of them would wear headsets, and we would position them in a circle while the audience was sitting in a circle looking outward, and the circle of VR headset people would look inward, and they would play a game, we would teach them how to sing. So they were kind of a choir, and we had real opera singers that would harmonize with them in a beautiful way, and there was a storyteller. What I really liked about it, we had the storyteller sing on the track, So, for example, we would talk about birds, but then when the audience members with the headsets on, they would do like this breathing moment when they had to move their arms. So, like all these layers were coming together quite poetically. Then after 10 minutes, the audience would swap. And then the last end thing, the choir would sing. Really, really pretty. That was a project I worked on. Then I did some NGO and corporate things. I made the most boring, interesting project ever. I made Lloyd's of London. It's an insurance market, actually. It's an intriguing company. And I made Through the Eyes of the Broker. Just because it's a complicated route. For example, when you fly to the moon, that's where you insure it. So they have all these constructions and all these weird insurance companies. So I made a route for a customer going through that route with all the brokers. which was fascinating for me because that's totally not my world. Another piece I worked on was actually for an NGO to stop palm oil plantations in Indonesia. I actually had to lie at the embassy that I was making a project about orang-utans, which I cannot lie, I can only fantasize, so that's what I started doing. Yeah, that's all the clips that come to mind that I did. So I've always been working also alongside John and after that on VR stuff. It's also maybe good to mention that I'm working as a creative and in-house director for a volumetric studio in the Netherlands since, I think, the beginning of 2019. We use the 4DFuse systems, so that's kind of like my day job. I do that sometimes a couple of days a week, sometimes a couple of days a month, and I help clients with making sure how they capture volumetric video actually makes sense in their projects, because it's quite specific. I really like doing that because it's also like when you do all these autonomous projects, it's really nice to not be the creator but be someone who's facilitating because it's really adding to my creative practice.
[00:17:14.260] Kent Bye: So maybe talk a bit about how did Imaginary Friend as an experience come about?
[00:17:19.912] Steye Hallema: Well, when I was working for, I think I was still working for Jant then, yeah I was. Also, we just made Ashes to Ashes, which was quite an elaborate cinematic VR piece, like stereoscopic 360 video. And when I was working for the Dutch broadcast company and for Jant, I would, every piece, like I think this is really my niche, I researched first-person storytelling. I was really intrigued by the fact that as a player as a spectator in a VR piece, you have a role. You are present, so whether or not you have a role, you could be a fly on the wall, but that's still a role. And I really like thinking about this, so I researched that quite a bit. And then at a certain point, I kind of like, I came to the end of the possibilities of POV storytelling in that sense. And I realized that I could never have the same cathartic, empathetic feeling for, as a player, you could never have this effect for yourself. I might be wrong, but at least not in 360 video, as you have when you look at a character on stage or in a film. It doesn't happen if you have to have that for yourself. You could probably have, similar feelings, perhaps meditating, transcending in spiritual practices, but not in a story. So I was really thinking about this, I was actually on holiday in Mallorca, and I remember walking up the stairs when it popped into my head, like wait a second, If you're the imaginary friend of this little kid, then I have both. Because this kid is making you up to deal with certain emotional, social trouble he's going through. And you, as the spectator, who is the imaginary friend, I have a very special role for you. You're a fantasy. I have this very interesting place for the player because you are a fantasy. Not only you're a fantasy, you're a fantasy of a boy. So you're some kind of weird warped psychological version of him, which I thought was a really interesting thing to explore.
[00:19:16.840] Kent Bye: Nice. And so when was the beginning of this project or what was the turning point where you actually started to make it?
[00:19:25.152] Steye Hallema: Well, then when I had the idea, I went to a cafe every morning and started writing the first version of the script. And then we just made Ash to Ashes with Submarine Channel. Submarine is quite a big animation studio in the Netherlands, but also in the rest of the world. And they had like a digital part and Ash to Ashes was quite a big success, so I came up with this new idea and they all really liked it. So they wanted to be the producer and then we started applying for money and then we hit a wall because there was no applying for money yet in the Netherlands. We made Ash to Ash with a bit of luck with a media fund that did their last thing. So then it kind of like stood still for a while because we were really waiting for funding. Although I would still work on the script obviously because I was fascinated by this idea. And then three years later finally there was funding and I could kind of start.
[00:20:16.937] Kent Bye: So that this this piece has a lot of different qualities of presence that you have as you you know Have certain moments you're embodied you have a story that you're telling you have some interactivity and agency You have some voice interactions. You have some choices I don't know how many choices are of consequence in terms of it impacting the narrative and so maybe that's a good place to start because there's certain moments where you're asked to respond and And so is this a narrative that has slight variations that converge or are there actually different branches in this narrative? Are there multiple endings?
[00:20:49.418] Steye Hallema: No, there is no branching. Actually, I don't really believe in branching. I do believe in it, but it's my theory that if you make a story about branching, which is choosing, then the story should be about choosing. So I know only two good examples, personally, like Stanley's Parable and I think Bandersnatch is also very interesting, the Netflix kind of interactive fictional film. So I don't believe in choosing so it's no it will just I believe more like in and sort of I would say like like something is growing or thinning so Like your reaction doesn't change the outcome of the story But it maybe it changes the feeling of the story because I can change the surrounding in a certain way But in Imaginary Friends it's very simple. It's just you say anything and it works. It's just how we designed the reaction that a lot of people actually think there's voice recognition in it, but there's none. It's really just amplitude. And I think this is the thing, this is the language we're trying to always develop. How do you involve the player in such a way that it feels like he's really involved? Or she, obviously. Or they.
[00:21:56.736] Kent Bye: So you said you sat down after you originally had the idea, you wrote out the script, and then this piece has a lot of different components and moving parts, you know, from the volumetric capture seems to be a big part of it, and then lots of different interactive components, embodiments, so where did you begin to start to iterate in terms of this piece, because there's a lot of different stuff that you're fusing together, and so how did you iterate towards, you know, eventually coming to the finished product, starting with the script, and then where did you go from there?
[00:22:24.263] Steye Hallema: Well, the first concrete thing we could do was actually I did a residency at the National Theatre in London. And they have like actors, they have this thing that if an actor is playing a piece there, they could also be called for other projects. So I had actually really good actors to do a test. And then I was still thinking to do this 360 video. And we rented this Airbnb and we had this boys room in it and we decorated it. Great kids, great actors and I tried different sections of the script to see how it would work. And I realized that it wasn't working in 360 video. And I realized 360 video is still... I love 360 video, I really do love it. I think I should make stuff in it still, because I kind of know how to do it. But it also has a limitation. I think for drama, I realized every 360 piece I made, I kind of made the actors act comedic. they would play a little bit like, we're playing and we're taking you along, which I then realized was a trick to help you cross that border, because actually the medium is quite clumsy, because you're in a fishbowl looking around. So if I want to break the fourth wall, like involve the player, I cannot really do it, because it's not interactive, and you cannot really do anything as a player. You can just look around. So this kid was really like trying to involve you in the story but then you couldn't do something for real and then I felt the acting became lame. I actually never saw acting in 360 VR that I really liked. With Ashes to Ashes I think we came quite far but then also because it was comedic and we used this trick that it was on a film set and then all of a sudden the film set broke open and it was not a story you were just like there on a film set. And for the imaginary friend I really realized that I needed to be more interactive. So I started to really look into interactions. Obviously also embodiment I felt was a really important thing for the imaginary friend. You should have some sort of form that the kid thinks of when he makes you up. And I thought 6DOLF was really important because it brings much more presence to storytelling. If 6DOLF is there, you are really there instead of looking around. So then all of a sudden I had all these research questions. How do I do the 6DOLF? First of all, how will I design the embodiment? And that also opened up a very interesting way of thinking, because he's making up, so I could do lots of embodiments. Which in the end, I did less than I researched, because I felt it was a little bit too destructive to do that all the time, because then you become busy with checking out who you are all the time. Which also brings us back to how do you create the flow of the story, how do you go back and forth between going along with the story and doing interactions. But the main research question, how would I do the kid? And first I thought of animation, but for some sort of reason, maybe also because Wolves in the Wall was made, I felt that would be an already beaten track. Like I would make this character with bigger eyes, which would be really endearing. And I felt if I have a real kid, it would touch you in a different way as a player. So I started looking into volumetric video and first we tried the hybrid, volumetric video is how I call it. We tried kind of like how Scatter is doing it with Depth Kit, with the infrared camera that senses the depth and then with DLSR that creates the texture. And we did quite an elaborate test, I think we spent too much money on it. Which was great, and it looked amazing, but it had a couple of downsides. First, the image you create is hollow from the side. And second, it has the streaks. Because if you have a hand in front of the camera, then the camera doesn't know what's in between the body and the hand. Which I actually, on an aesthetic level, I love, but as a storyteller I think it has a certain feeling. For me it has a feeling as we're looking at a ghost, or the past. And I felt it wasn't suitable for the imaginary friend, at least not as I was envisioning the imaginary friend. And also, I wanted to capture a kit in all its jumpiness and realness. And I felt like it had to be too stuck in one position to be able to capture it properly. And also, still, the hybrid volumetric is a lot of work. So at a certain point, the production company said, then we need to hire 10 people for 10 days in India to do the rotoscoping. I was like, OK, this doesn't make sense. This is so expensive. then kind of at the same time I got this job at a volumetric studio in Eindhoven in Holland and I started to have some experience with it and I realized wait a second I might be one of the most experienced people in the world doing full volumetric video I have to try to pull this off so I made a choice to do the kit in full volumetric video. That was a really important choice and I kind of took a big risk to choose this and promise this to the funds we were then getting that it would work. But because there was not real editing software yet, you know, it came kind of raw from the systems. you really couldn't edit. So then by sheer luck 4DFuse came out with editing software and I became like the most fanatic beta tester in the world and I really stalked them and because I realized they could also do something else which is really important which was head retargeting. And head retargeting is really really really important for the piece. Head retargeting is basically we kind of like add some functionality to the head of the volumetric capture that I can use to make it look at the camera in unity. The camera is kind of like the player. So the boy really acknowledges you. Because with the first test I did with volumetric video, which looked amazing, you know, we had the like beautifully 3D photogrammetry room, the boy was really well done, the production company made like, they made it look really good. There was one big problem. The kid wasn't looking at the player. So the whole point of making contact with you was lost. If someone is not looking at you, it also has a psychological effect, doesn't it, Kent? Because now, for the listeners, I'm not looking at Kent while I'm talking to him. It's probably very weird. Yeah, it's pretty weird. So that was a big problem I had to solve. I mostly solved it with body retargeting, but that wasn't always possible, so I used a lot of head retargeting, and it's much more subtle, and in that sense also much more realistic. So that was a big thing and when that all kind of worked then you have all the layers which you just named in the question I've been talking for a long time five minutes ago all the layers like I realized like how And I don't mean this in a demeaning way, but how easy it is to make a film It's just two layers in the end that you're editing together. It's sound and image. Okay, you have maybe have the dialogue and the music and and sound effects. But in VR you have dealing with presence, telling stories with presence, you have the interactions, then you have the story, then you have the embodiment, then you have the sound. So there's so many layers that we have to find a way to mould in something that actually flows. Which is really a challenge, and maybe because I am a musician, I've been an editor, I've been an animator for quite some time in my life, I really take pride in making the musical flow of the thing good. But then you have the interactions, you have to make, sometimes the player has to do something, so you have to wait. It's quite a challenge. I personally feel it came out quite good. Definitely room for improvement. But yeah, I think I managed to get it to a certain level that the experience really flows. And that was also a really big challenge. Also because Unity, where we made it in, works really differently than a sequencer, like an editor, like for example Premiere or After Effects. And it took me a while to figure out what's the difference. In Unity, you basically have a space, and in that space you place objects, and you couple things, functionality, sound to those objects. So there's no timelines. Until I discovered, actually, you do have timelines in Unity, but they're just basically a way of working with Unity. And when I discovered there were timelines in Unity, that was also when I got a better grip on it. The first version I made was kind of like a narrative prototype. I let my girlfriend do all the voices of the kid. She's kind of a voice artist. She's a singer, but she's super good with accents and everything, so she could do this. And then I would put these as audio tracks on the timelines, and then I would design the space, and then I would put in that space, I would put text where the interactions should come. And then I gave it to the developers, and they would make the interactions. And this way we could create an interactive prototype, or storyboard is a better word, storyboard, which kind of had all the narrative elements already. Because if I, as a storyteller in VR, I cannot just tell a story, I have to like, the interactions need to have meaning, they need to come from the story, and I need to test them on an audience to see if what as a storytelling I think I'm telling is actually being conveyed. So that's what we did with this interactive storyboard. And then you have this through line and then you can add the layers as the organs and the skin and the muscles on the skeleton.
[00:31:52.768] Kent Bye: So it sounds like you had a pretty detailed interactive storyboard where you're starting with the script and then you have your girlfriend record these voices that then you can put into an immersive experience with some notes for the developers. And were you showing people this storyboard previs interactive experience as sort of a demo? Were you able to create the entirety of the whole experience just to get a sense of the flow? Or did you really need to start to build out each of these different scenes and interactions before? I'm just trying to get a sense of the iterative process as you're trying to refine the script and refine the story as you're adding these additional layers How much is that feeding back into like the core story that you're telling?
[00:32:32.188] Steye Hallema: Well, I think the answer is both. It did really give me a feeling of the whole thing, and at the same time it does not, because every layer added changes something. There were some parts, the end when the boy flies up in the sky, it becomes quite dramatic. And I didn't have the music yet, so I couldn't really make the dramatics work. But I had an interesting discovery when the narrative script was kind of, oh, sorry, the interactive storyboard was kind of done, that I always set out to make this kind of an emotional experience. at least to let people be really emotionally involved with this kid. That was my aim. And then I had the interactive storyboard and something else happened. It wasn't as emotional then, but something else happened. I was involved in the play of this kid all the time. And then I thought like, but this is actually really cool because this is something I have never experienced yet. Only in reality that I'd like a kid would like involve me in his game and I would go along. So I thought like, okay, we're on to something. We're really on to something. Like this is what I want to explore. And it was only later when we started adding the music and all the visuals became good and we put in the volumetric kit, who was a real kid, that I reacquired the emotional depth of the whole piece.
[00:33:46.593] Kent Bye: And as you start to build out the experience, going in and capturing all the volumetric performances, in some ways, you get kind of locked into what kind of stories you can and cannot tell based upon what you have been able to capture, because you're kind of limited with what they gave you in that performance. So did you end up having to do any reshoots or pick up later as you're developing the experience? Or at what point in the development process did you actually, like, capture that performance? Because that seems to be a pretty key part of the entire experience.
[00:34:15.180] Steye Hallema: Totally. Well, again, both. Because I had the interactive storyboard, I knew what I wanted to capture. I knew the story was flowing, and I knew how to do it. But then there's a real kid, and this kid is young. So I did reshoot some stuff, especially the beginning, because at a certain point you have a feeling for how it feels, and I felt the beginning needed something else than I captured. But also because I think I had some experience with volumetric video, I brought a lot of ruses to the studio. One of the things I discovered when doing the first tests, like imagine a volumetric video studio, there are 32 cameras pointing inwards in a green tube. So having a kid in there acting like he's addressing some imaginary friend of his is a very abstract thing for a kid. So I came up with an inter-actor. So actually my ex-wife, the mother of my daughter, she's a really good actress and she also is the director of a youth theatre school in Amsterdam. So I asked her to kind of be the imaginary friend. So we would film her in the back side of the studio where all the machines are and she would be shown on the television in the studio. So the kid would act towards her and she would react and she would really provoke his reactions. So that for example made his reactions really real. And to be honest that worked really well but at the same time it was still super stressful for me because You know, I had six Wednesdays to do this whole thing. So basically, at the end, this production costs a small house. And it was basically hanging on the shoulders of an 11-year-old kid. And I couldn't put it on his shoulders, so I had to keep all this away from him and having to do what I needed for the thing. It was one of the most stressful things I've ever done. It was only when we were done, I realized... it worked, you know, it could not have worked. And the kid gets older, you know, he progresses, so you only also have this, like, this small window of opportunity that he's that kid at that time. If they're 11, you know, you're really young, you know. Three months later, there's someone else, like, all of a sudden, a tougher kid, you know, because he made, like, something happened. So that was stressful, yeah. But also fun, you know, it's fun working with kids, yeah.
[00:36:32.866] Kent Bye: So we're here at Venice in September of 2023. When did you finish capturing the volumetric performance over those six weeks?
[00:36:41.368] Steye Hallema: November last year, 2022. We started in July and I think November was the last shoot. Because capturing sound with a jumpy kit in a studio that films from outside is hard because you can't really hide the microphone anywhere apart from underneath his clothes. So you have a lot of rumbles. I wanted to redo some audio with him, but I tried that in March So five months later than November and he was already another boy. He was tougher You know, I couldn't get this kind of cute thing that he had out of him anymore. So I couldn't use these recordings
[00:37:15.161] Kent Bye: So then you're starting to add all these other interactive components because as an imaginary friend, you're sort of like this bird-like creature. So you have different embodiments and you have different interactions where you flap your hands to be able to fly up and move around in the different scenes. You said just a bit ago where in film you have like the sound and image and different degrees of the sound, of the sound effects and music and whatnot. But with VR, you have all these other layers. And so which layer did you start to add in next after you have the performance and you have it in Unity, then where do you go from there?
[00:37:45.622] Steye Hallema: I think we kind of worked iteratively on all aspects. Embodiment, yeah, kind of, you do it in sprints, so we already were testing embodiments in the beginning just to see what it could bring as a storytelling device. Same with interactions. I think the interactions are very important, especially because as a storyteller there's this challenge to go back and forth from interaction to story. And so I only used interactions that were easy enough not to explain. Perhaps I could explain them from the story, so the kid would say something to you that would explain the game mechanics, but not like, I'm explaining this to you, but it would be embedded in the dramatic flow. And getting that right was probably the first thing to do, so to having the voice interactions. I think having the voice interactions work with volumetric video was a big challenge, which I think has actually huge potential. First of all, how to record the kid in such a way that he asks you a question, then he waits for you to answer, and then he reacts. So my first, actually I did reshoot this scene. At first I made a version and I kind of made it work and then I thought I had a way to do it and I was kind of looping him in all kind of short loops and I tried to do that and it wasn't working. I was making him too mechanic. I was actually going against the strength of volumetric video that it's so real. To elaborate on that, I think Follow Magic Video is amazing because it's never uncanny. Because the micro-movements of bodies are so real and our brains are so, in a million years of evolution, we're so good at reading humans, that a lot of motion capture and then they put a body on it. It doesn't work. I'm very rigid about it. It just doesn't work. It makes you feel like this person is not real, it's weird. Especially if they use photogrammetry and then animate photogrammetry faces. It's horrific. It does have a quality if you make something that's kind of like... So, because I was cutting it up so much, I was making it so mechanic again, that it wasn't working. And then my developer, he kind of didn't understand my idea, so he already made a version where he was just using how I shot it, and I looked at it, it actually was working perfectly, I was just having longer loops. I would just have this, he would have this question, I would just wait for you for a long time. and then at the end perhaps we would make a loop and then we'd have the answer and we'd just play the whole answer. So it was great, it was pure serendipity that he interpreted it like it should be done like that and it worked. So we had a lot of luck actually on this. We had maybe seven miracles happening that made this work.
[00:40:35.070] Kent Bye: Well, I think one thing that I noticed, at least with the volumetric capture as you go through these different sequences, is that you do have to, at least I have a recollection and memory of these pretty harsh cuts that you have to do with the volumetric capture, because going from one mode of embodiment to another mode of embodiment, it's difficult to seamlessly morph that, and so you have to have these jump cuts. So I feel like the jump cuts, I mean, it sort of reminds me that this is not a person, it's a mediated recording of someone. Yes, it's a little less uncanny, but the plausibility gets diminished once I know that I see this kind of implausible cut that reminds me that I'm in a mediated experience rather than I'm talking to a person.
[00:41:14.338] Steye Hallema: Well, that is sad and wonderful to hear at the same time, because it did also work. That's what you're saying. You say, like, the illusion is broken sometimes. And then why is it broken? It's just technical limitations. You know, there's Fallen Magic Video running on the Quest 2. So I had to minimize the amount of volumetric capture I use. Also volumetric video is super expensive. Although I'm working for a studio and they gave me quite a bit, I still had to, I couldn't keep on filming. So sometimes I had to do a jump cut. There's actually also versions where I do a jump cut where I think it works. Please give me your opinion on this. I came to the conclusion that if he would ask you something, and you would answer, then I could do a jump cut. Which was great, because going from a question to an answer, I could morph, but I couldn't get the morphs working at all. So there's an impossibility to make that work. I don't know if you agree with me, like in the scene in the school, then he kind of jumps. Did that work for you or not?
[00:42:16.772] Kent Bye: Well, just to take a step back into like the different voice interactions and agency that you have in this piece, I tend to think of the different qualities of presence like there's embodied presence and environmental presence and emotional presence. Those are the more receptive where I'm just like passively watching. and I'm receiving what you're giving. And it's sort of driving different emotional reactions. And then there's the active presence and agency with the mental and social presence and plausibility and actually speaking language. And that's more of the outward expression, young, interactive components where you're asking me as a user to participate in some fashion. And so in this piece, you have this balance between the center of gravity is that you're telling a story and that I'm receiving it. But then you're asking me as a user to say stuff. And so at first, I'm like, OK, well, I'll respond. And in these pieces, I try to do a yes and improv approach where I'm not trying to break the experience. I'm trying to have the best experience I can have. So if you say, are you my imaginary friend? Yes, I am. Are you real? Yes, I am real. And trying to respond. But then as soon as you introduce the possibility for me to answer, Well, first of all, I answered. And then he asked me a second question. I didn't know if it was rhetorical or if he was actually waiting. And he said, no, I'm waiting for you to answer. There's this moment where I don't answer. And he says, no, I really need you to answer. I was like, OK, well, I answered. And then I see this visual like particle effects coming out of my mouth telling me that it's registering that I'm saying something. At that point, I don't know if it is interpreting what I'm saying or not. And I don't know if it's actually doing natural language processing and it's engaging me. So then I'm like, OK, I can speak at any moment. So then I start speaking and there's no interaction. So I think that there's a sort of tricky balance of like, well, if you're going to introduce this possibility for me to speak and you're going to ask me to speak, then I want to speak it all the time. I don't want it to understand me. And then immediately I was like, well, it's not listening to me and it's not understanding me. So then I'm like, that's not real agency. So I kind of knew in this experience that it's sort of like a magic trick that I'm not That's why I asked if it actually was branching at any point. Because when you play an experience that's interactive, you don't really know unless you play it multiple times to what degree is there a branching and agency. And what degree am I actually being asked to participate in this? And so it's a tricky balance when you're including both of these. Because when I test it to the limits of like, OK, well, you want me to participate? Well, here you go. Here I'm going to participate. And then it doesn't receive that. So then it even more diminishes that plausibility that what I'm doing is actually interacting and meaningful. Should just receive this because it's not really gonna matter anyway
[00:44:47.958] Steye Hallema: Well, this is why I love talking with you, Kent, because we're trying to figure this out. And these conversations that we're having, where you, with your wonderful analytic brain, formulate this so clearly, this is how we're going to find the answer. Obviously, I couldn't find this answer because of the technical limitations and the budget limitations. I would love to make a piece where you can speak all the time. I just wouldn't know how I would do this in this setup. And I think there's elegance in finding form. Maybe then I would have to make a piece that's only about speaking to figure this out completely. I kind of compromised in that sense. I felt like speaking does give you a lot of agency. and makes you feel you are part of the story. So although it's maybe still a bit brutal, I will still use it and hope for forgiveness of the player. And I think there's also degrees of forgiveness in players. I think some people are much more touched by the relationship and they forgive that it's not perfect. Maybe someone like you is like knowing VR like really well is looking for via much more analytical routes. I am comforted by the fact that for both there's something very interesting to get here.
[00:46:09.238] Kent Bye: And I was thinking about this after this experience actually like the differences in spectrum between what true agency is versus what is like participation and interaction and I feel like this is more on the scale of interaction rather than agency because agency means you actually have choice and control and free will over your actions and As an experience, you have narrative agency, so are you able to control how the narrative is playing out? You have embodied agency, so you can move around your body. You have environmental agency, so you can navigate a space. You have social agency, so you can actually interact and participate in conversations. And in terms of receiving that agency, it's this spectrum between what's a game versus what is a story. And if you have agency, I want to be able to fail. because if I can't fail, then it's just basically an interaction. It's participation that I'm doing a mimicry of something that feels like I'm in the magic circle of agency, but really, like there was a moment where there's these monsters coming at me, I'm like, I wonder what happens if I don't do anything? You know, can I die? I was like, can I die? Because at that point I'm like, I'm in a game right now, can I die? And it was like, no, I can't die. I'm not getting hurt. There's no hit bar. If I don't do anything, it's not going to make any difference. So it's sort of like,
[00:47:22.334] Steye Hallema: Yeah, it's all true. But what if I would have... Obviously, we tried to have game mechanics that you could win or fail, but I couldn't get that in a meaningful way in the story. There was no way that I could do that. And in terms of agency, I think that you are there for him is already a really big part of the story. You are there for him, and that's why you are an imaginary friend. Also, you are kind of stuck because you are made up by him. This is your position. You're kind of stuck because you're in his fantasy. Basically, you put the consciousness of a human in the fantasy of a boy. That's already like a really weird combination. to make, so your presence is basically helping him already, so that gives a place for that, like a natural place for the player, but at the same time, if I give you too much to do, too much of a free will, then this imaginary friend would grow out of control, which I think is a really interesting way of thinking for a sequel.
[00:48:26.942] Kent Bye: Well, here at Venice Immersive 2023, there's a piece called Topomancer, which you're entering in different text and then the experiences are responding to you. So I think things with large language models and chat GPT, there is a capability to receive input. The challenge is trying to get away just from the typical querying of the statistical response of chatGBT as a large language model that surveys the entirety of the internet rather than coming back with a specific character. There are things like nworld.ai which allows you to create more of a specific character with a bounds of knowledge and more of a narrative context where you're able to interact with kind of a filter to this large language model that has the ability to drive a specific character. So when I see the different experiences here at Venice, I saw all the 43 different experiences and I find that it's helpful to kind of, as I'm covering the pieces and trying to give these first impressions, try to inform the people that are attending here the different center of gravity of what presence it's really focused on. Because some experiences are really about the environmental design, some about embodiment, some around the ideas that are being communicated. Sometimes there's social presence dimensions, the emotional presence coming more from the film world where it's trying to tell the story. And then there's more of the interactive agency experiences that more like a game. So I find that whatever experience there are, it's going to have a center of gravity. And so this center of gravity is mostly a film because it's more passive because the elements of agency are not having consequence, but it's also the mental and social presence where you're creating this sort of social dynamic between you and this imaginary friend. You're speaking you're communicating at least you give the illusion that you're speaking and communicating There's moments where you have shared embodiments with this other character where you feel like you're in a group experience with this Protagonist and then there's overall the ideas that you're talking about what is real and what's not real? so I'm kind of left with these deeper philosophical provocations and thinking about this and So, really great experiences do all the different qualities of presence, but they tend to usually fall down into some sort of center of gravity, whereas the games, you know, they have much more of the game mechanics, they have an ability to fail, and it's a different conceit for how you structure the experience, because you have to have a gameplay loop that is actually satisfying, because it's not actually satisfying unless you have to fail and have to improve to actually solve the game. So I think it's a tricky thing to kind of blend all these different modalities together. Your piece is probably the most ambitious in terms of trying to fuse all these things together. But I think it also starts to reveal the friction that comes up when you do start to push them together.
[00:51:01.507] Steye Hallema: Yeah, I agree. I also agree with your analytical system. I do, however, disagree with you where that you say it's more like a film, because I think the presence of the spectator matters so much, the player matters so much for the story, that that sense of presence, for a lot of people that suspension of disbelief in sense of presence does really work. So in that sense, it's definitely not a film at all.
[00:51:27.265] Kent Bye: Well, the reason why I say that is just because when a director creates a film and they export it and they show people, there's nothing that the people can do that change the outcome of that film. Whereas in a game, you have many different opportunities to express your agency and go down many different paths. So in terms of like comparing film versus a game, this is more like a film than a game because more or less people are going to have the same arc of the experience, whereas their interactions may flavor the experience, but it's not going to change the outcome of the narrative.
[00:51:58.233] Steye Hallema: I do have to say that I find your vision of a game a bit limited. I think you don't have to win or fail in a game at all. I think if you're playing in a certain way it can already be a game. It also proves your point, but for example a game that really inspired me is Journey. where you can never fail, basically. Sometimes it just takes longer to get there. The only thing you do is try to get to the peak of a mountain, which for some reason they make really clear without explaining. When I finished that game, it felt like I had a really interesting, weird dream, and I came out of it better. So I could not fail that game, and it was a wonderful experience.
[00:52:39.240] Kent Bye: Yeah, I guess, I mean, there's certainly this is a long standing debates as to how to like define each of these things. I guess I'm trying to figure out as we are fusing all these different things together. Like there's experiences here like Pixel Ripped or Fisherman's Tale or the Utility Room. Those are center of gravity more like game-like. And and you know like because oftentimes when I say well, there's no Agency in a film. Well, you can watch a film and it can actually be deeply profound and can change you but Nothing that you're choosing is changing the outcome of that film for other people to see it so there's like this difference between how you're receiving a narrative internally and how it's changing you and transforming you and feeling you can and have control over what you're focusing on and really allow yourself the vulnerability or attention internally within your body as you receive it, but there's nothing that you're exerting outward that's going to change the outcome of that film if you watch it again and again. It's still going to be the same structure of the experience. It's a time-based medium that's trying to modulate your emotions, but it's
[00:53:39.480] Steye Hallema: It's a linear narrative, basically, yeah. Well, to go back, I didn't see Topamensa, unfortunately, but it's kind of a form that I've been exploring quite a bit, because as someone that is hell-bent on making experiences that actually the person undergoing the experience is the main character, this is a form I'm really exploring. This is why the pile of books next to my bed are all about psychology. It's like if we make something about the player, then we need to know who this player is. And there's different approaches to it. I really like this form of getting to know the player and then doing something with that. I think one of the best pieces I ever made was the Haunted House, which was actually a real haunted house, but it was not about ghosts, it was about something else. So you would go in, but would do the psychological test with you, which is actually quite an absurd psychological test, but you would be filmed and recorded. Then if you would do the psychological test well, you would get a map of the haunted house. You would be let in. The door would close. You would be in pitch black. The map was completely useless. But you would hear a voice in the distance saying, I'm here. So at a certain point, people start walking. And you would walk through this maze. At a certain point, you wouldn't be able to go further. Bang, this door would close behind you. And tring, this phone would ring. A spot would be lighting the phone. You would pick it up. And for example, if I would do it, it was like, hello, I would say. And then I would hear my own voice saying, hello, who is this? And I say, this is Stijl. And then the voice would say, that is not possible, Stijl, because I am here. Because we recorded these words before and then another door would open and you would go into another room and you would see yourself walking through the maze filmed all Blair Witch with night vision cameras and then all of a sudden the last shot of the film you would see yourself in that room looking at that film But there was someone behind you and people would like the coin would drop they would turn around and for example If I would be the ghost I would stand there with a printout face of themselves and they would be scared shitless by themselves Which was kind of like the idea two years ago. I made a commercial thing for Philips Philips is a company that made light bulbs and then they started to making medical equipment apparently that's more profitable now and And they wanted to do like this, I don't know, this media thing. So I made a journey through your own body for them, which is kind of the same thing. So first you have this questionnaire that you fill in, and then you go through your body. And it was a bit cheesy, but sometimes it's nice to try cheesy. And it actually really worked, so in this questionnaire before you would have to record names of people that are really close to you and then you would be in your heart and you would shine around with a torch and actually it was like a sky with a lot of stars and you would shine at the right star constellation and it would say the name of someone you really love and people would really cry. So that's kind of like the same form and for the next piece I'm trying to make, which I would love to talk with you again and again, is Inwards. And Inwards, the tagline is a journey through your own subconsciousness, which is kind of impossible and never impossible because as long as you do anything you kind of like use your subconsciousness to understand. But it was inspired by a book by Jodorowsky, the Chilean director that made El Topo in the mountain. he has this form of storytelling that's called, sorry, a form of therapy, actually, called psychomagic. And he kind of has this weird ritual. So if you're lovesick, you have to stay three days at home, only eat white rice, and the last day, you have 12 candles you put in a circle, you lay in the middle of it, you get seven little plates, you put cups on, and you overflow these with honey and blood, then you press them against your chest, as hard as you can. Oh, there's actually a photo of the person you're missing in between. As hard as possible until you cry out for 15 minutes. Then you break the little plates with a hammer with 12 strokes. And then you take a bag that printed out Virgin Mary on it. You put everything in. You clean your chest with lemon. You also put the lemon in the bag. Then you paint your face silver. You go to a pub, get pissed, and then on the way back you have to throw the bag in the canal. It's full of rituals like that. I was reading it and it's hilarious, but at a certain point I was like, wait a second, this is VR. This is it. Because it's given me basically a way to answer so many research questions that I have. If my story is about the player, what do I offer the player? Then we get into some form of therapy. which is a very dangerous path, because I'm not a therapist, I know nothing, although I have a pile of psychology books next to my bed. It's also like, if I make a piece that's about a player, how do I make it fit to this player? So there's, for example, the option of having a questionnaire, but maybe we could measure more subconsciously, like letting you take a route, a certain thing, and make us map you in that kind of way. That will be kind of like my third piece. I'm trying to take the same form. Perhaps we use AI, I'm not sure it's necessary. One thing I would really like to do is use the Oracle systems as building blocks for the world. It's going to be like a road game, so you have the agency to wander around. But the worlds are built, and perhaps also the actions are built from oracle systems like perhaps astrology, the tarot, or perhaps the I Ching. I personally really like the I Ching, I think it's a very elegant oracle system. Because these systems are evolved over centuries, sometimes even millennia, to tailor, I think, to our subconsciousness. dialogue methods to make you... I don't personally believe in the fortune-telling power of them, apart from the fact that maybe if you understand your place where you are now, where I think this Oracle system can help, you might be able to have a better future, you know, because you understand yourself better. So I'm dying to start making this piece. And I think it's also really a form that will be clear. I think also, for example, the imaginary friend is bigger than the imaginary friend I made. I think the imaginary friend is like in literature, you have these forms. I think the imaginary friend is going to be a form in XR storytelling, because it's such a clear thing you can use to tell many stories. There's this wonderful couple here from Israel who also use an imaginary friend to... And their interaction is actually by a deaf kid that uses sign language. So I already see, like, Wolves on the Wall is a great example. Which I think you're not clearly an imaginary friend, but you do have that role. Yeah, so there's many, I think there's many ways of using this. It's kind of a literature kind of approach to VR. Actually, this is a statement I would like to make. I think VR is maybe very close to literature in a way. What I find interesting in literature that's mostly there's a lot of room for the internal voice of a main character, which is there not so much, for example, in a film or a theater, because in VR, XR I should say, there's so much role for the player that that becomes a bigger thing, which is interesting. They're, I think, at the ends of the spectrum. Like, a book is magic. There's just scribbles on a piece of paper. And you conjure, as the reader, you conjure all these worlds. It's just pure magic. And then, in that sense, VR is super lame. You create the whole world. Where's the poetics? What do we not show? That's also part of, I think, figuring out this language. What do we not show? In Imaginary Friend, I go back to a question you asked maybe an hour ago. My first route was 21 scenes. And I realized when I was trying to storyboard that just for myself, that I'm so conditioned by film. I had so much expose, just looking at stuff happening, that I realized this is, I'm so conditioned, this is so annoying. So I took out 14 scenes and I just kept the scenes where you're really like playing with the kid and there's interaction and there's a pure relationship. At a certain point I had to take out scenes. So for example, the kid is being bullied. But I don't really show that, because then you're just looking at the kid and you see him being bullied, which is a film scene. I thought that was boring. So what I do, like, he's in the classroom and he's completely disappearing his own fantasy with you, that you can fly and you have a funny stance on his chair. And then his classmates are like, what's happening here? And then the teacher shouts and he's like broken out of that fantasy and all of a sudden you realize, oh, what's happening here? And then also you drop as the imaginary friend because he gets back to reality. And then the next scene, you are on the roof on the other side of the street of his window and he's bullying you. So I kind of find a new way to juxtapose. I think the beautiful thing with film is that you have this shot and that shot, Eisenstein, and your brain is like doing the math. I felt like this is a way to do the same thing, but then in VR. Also tying into the theme that kids use imaginary friends to practice social situations on, to deal with social situations. So I thought like making him bully you is much more interesting because then you also feel bullied, which is a much more interesting place to understand bullying than when you look at bullying. Same thing I do with when he goes to the doctor, which is actually a bit misunderstood, because after the not doctor scene, actually, sorry I make this confusing, so let me first explain what I'm doing. So you have this game and then you end in front of like a hospital with another imaginary friend. And this imaginary friend is actually telling you what they do in the hospital, that they kind of like make you sane again, basically, but then he disappears, because actually he also has a kid making him up, doing therapy, losing the imaginary friend. So you see him disappear. Then the next scene, you are with the kid, who is basically doing therapy on you. And then you kind of like, you don't disappear, but he's putting you in a jar somewhere. So I felt like this is also a more elegant way of telling the story. You have to figure out what happened instead of showing it. And I think a lot of VR for me is way too literal. You're looking at stuff that's happening and no one is taking anything away. So my brain is just seeing what's happening and I'm super bored.
[01:03:58.711] Kent Bye: It's quite interesting to hear a lot of stuff there that you just put on the table and I think going with the imaginary friend thread there's Yeah, there are these like little embodied interactions where you're flying around and you go to the hospital scene after there's like another You know, there's like three different gaming like scenes But two of them are basically the same mechanic where there's things that are flying at you and you're kind of helping protect with your wings things that are flying at you and you're kind of knocking away and then on the way to the hospital you're flying in this drone and you have these entities that are coming at your friend as the imaginary friend and you're aiming and gaze detection kind of shooting them and then you are coming up onto this light pole and having this interaction with this other imaginary friend. I'm not sure if I in the moment had gotten that because he's getting therapy and being told that imaginary friends aren't real that that's why that the end is that you stop being his friend because of the therapy that all of a sudden he's convinced that his imagination is Not actually I I remember the ending being that the kid telling me It doesn't matter whether or not you're real or not. I still enjoyed hanging out with you. I Still like you yeah, and which which that for me sort of like made me feel like I was not like a real But anyway that maybe you can elaborate on that
[01:05:11.750] Steye Hallema: Well, you came back to me with that and I truly deeply enjoy talking with you because we get into the nitty-gritty what's actually happening here. You know, in the end I'm using my intuitions to do it right and I have all these layers and a budget and a production meeting and whatever, you know, like this, like making film is hard. But I think what you point out is that actually you are real and I'm addressing that wrong because he's saying it doesn't matter. I think that's a very interesting perspective.
[01:05:38.560] Kent Bye: Yeah, just to elaborate, because when he says, whether or not you're real or not, as soon as he says, I may not be real, it makes me question and then create this contrast to be like, no, I, well, I guess this is from the perspective of Alfred North Whitehead and process philosophy having like the creative process as the underlying nature of reality and our imagination being the organ of perception of reality. And so there's a certain way in which that the imagination is actually sort of a real part of our experience and our reality that I do see as real. And so as an imaginary friend, I have this orientation where, no, I actually am real. And when the protagonist friend is telling me, oh, whether you're real or not makes me in that moment say, oh, maybe I'm not real, but no, I am real. And so it sort of creates this dialectic of like a pushback of like, ah, you're just like basically undermining my reality of my own knowing that I exist and that I'm real.
[01:06:28.139] Steye Hallema: or making you feel furious that that is not acknowledged and makes you feel more real. But I get that totally, it's like totally Descartes as well, you know, like I think so I exist and you are a consciousness being put in his fantasy. From the perspective of the kid the imaginary friend might not be real and I think for me one of the themes in this experience is that dealing with imagination is one of the top because the imagination is a double-edged sword. It's our superpower as a species and it's also the cause of all the suffering in the world. So dealing with imagination is kind of one of the themes. It's kind of inspired by my older sister who claimed when she was younger to be paranomically gifted. which I'm completely agnostic about. I don't know if that's real or not, but it's her reality, so who am I to say anything judgmental about this? It did scare me and my other sister sometimes, because she would say like, there's a green man in the corner there. But she clearly has a different sense of reality, like how she's perceiving reality is really different than I perceive reality. The stories about our youth are so different from mine, so I've always been really fascinated by this and I thought that would be kind of that's always been like for me one of the things I'm talking about. So from the perspective of the kid, that makes sense. From the perspective of a very intelligent human being, being the player, that's a different thing. So yeah, interesting conversation. I don't know which perspective to take. You're right. I think you're right. Although I leave room for your perspective because I don't care if you're real or not. I like you. So maybe his position is, it's not about being real or not, it's about the connection and the friendship. Which is also, I think, a good team in the thing. It's about friendship.
[01:08:23.073] Kent Bye: But is part of the idea that he has to go see a therapist and he comes back and once he sees a therapist, then now his relationship to his imaginary friend is being fractured based upon as he grows older and the cultural pressures of what the culture sees as what's real or not is influencing him to sever this relationship or at least create some sort of barrier.
[01:08:42.651] Steye Hallema: Well, it's more like, okay, so he's being, like, so he has his imaginary friend, but he's also bullied because of this imaginary friend, so he has to deal with that in any way. And then in the scene that he does therapy with you, he's actually, you know, talking about that thoughts are coming and going, like some stuff a therapist would say just to make you, basically cognitive therapy that you can look at your thoughts and they come and go. which is not a typical thing for a kid to understand yet, I think, but it's more like from a meditational perspective, which is, I think, used in therapy a lot, that being absorbed by your thoughts and making that reality is kind of like samsara. That's actually what I like to put in there. So he's realizing that his thoughts are thoughts and that his fantasy is also just a fantasy. So he could also let go of his fears, which are also fantasies, but that also means that he lets go of his imaginary friend, which is also a fantasy. And then that doesn't really work for him yet, and that's in the last scene. He's really happy that you're back because he actually needs his good fantasy to conquer the bad fantasy kind of thing. until he comes to a place where he kind of like realizes that his feelings that he didn't understand are actually okay. Which for me are derived from a peyote trip I once did.
[01:10:11.076] Kent Bye: Yeah, I wanted to come back to this thing you said about your next projects looking at these different Oracle systems. And I'm just curious if you have come across the introduction that Young has written for Wilhelm's Itching.
[01:10:23.466] Steye Hallema: I did read it. I can't really remember what he said anymore, but I think I know if we have this conversation, perhaps stuff comes back. Yeah.
[01:10:32.845] Kent Bye: Yeah, just because, you know, Jung was someone who was working with the I Ching quite a bit and he was sort of wrestling with the philosophical implications of it and also digging into other hermetic practices and alchemy, astrology, but more looking into like the philosophical traditions and trying to reckon with what philosophical orientation have to be in order for these types of oracle systems to make sense. And he was looking more into like the philosophical traditions of idealism and neoplatonism and Just the idea that there is this... Idealism is more Kastrup, right? Idealism is Kastrup, Bernardo Kastrup, like... Well, it starts with Plato and there's a whole lineage, but yeah, Bernardo Kastrup is an idealist, but there's a long lineage of Platonists that precede Kastrup. He's just more of a, he's a modern instantiation of that, but there's a German idealist, there's Plato, there's Neoplatonism, there's a whole, Hegel was a Platonist, so there's lots of different Platonists throughout the history of philosophy, but yeah, Kastrup is a modern instantiation of a Platonist.
[01:11:26.538] Steye Hallema: Okay, cool. Please keep taking me along because I have a very fragmented knowledge of philosophy.
[01:11:32.840] Kent Bye: Essentially, Aristotle said that there's four major causes. There's the material cause, which is basically physical matter interacting with things, the materiality of reality. There's the efficient cause, which is the interactor of pushing something. Or you could say, what is the beginning of all the universe? Is it God? Is God the ultimate efficient cause? But efficient cause and material cause are the two major causes that most analytic tradition of philosophy and science are looking at trying to understand the world through those two causes. But Aristotle also had formal causation and final causation. The formal causation are more the mathematical structures of reality that are somehow potentially influencing reality. And when you look at quantum mechanics, there's more of the quantum wave function probabilities that create these archetypal potentialities of possibilities. And it's those realms of possibilities that you go from possibilities to actuality. And that's sort of that formal causation of those realms of possibility and archetypal potentialities. And then you have the final causation, which is the ultimate purpose and cause. But you can also think of it as your intention of purpose for where you're going. So a lot of what Jung is wrestling with is like the fact that you have to come up with a very specific question and that creates this sort of final causation that from more of a neoplatonic formal causation perspective is then subtly influencing the probabilities of selecting the right tarot card, selecting the right I Ching or in horary astrology it's, you know, asking the question that then is answered and received by the horary astrologer that then is providing the answer. The divinatory practices are all about intention of a question. So in order for these to work, you have to really meditate on a specific question to be answered. And then the divinatory practice is then, through this probability of these different archetypal potentialities, then you sort of go on in providing the answer. So Jung is trying to come up with a philosophical orientation as he's been wrestling with the philosophical implications of I Ching in this introduction that he's writing.
[01:13:29.086] Steye Hallema: Cool, yeah. I'm listening to you fascinated. Continue.
[01:13:33.582] Kent Bye: Well, just as you're designing these different practices, then it's about how to create the receiving end of whatever those range of possibilities are. In the TRO, there's a certain number of cards. In the I Ching, there's 64 different hexagrams. And there's bounded divinatory practices that have that set of possibilities. And then for each of those possibilities, then creating some sort of immersive experience out of that. So I think the question, though, is the process of, taking that input and then creating a variety of those different archetypal possible experiences that then you could take people on. So I don't know if you've thought about how to translate this whole corpus of different types of archetypal experiences that then are trying to focus in on what people may want to experience.
[01:14:16.638] Steye Hallema: Well, I'm really at the start. We just got funded for the research trajectory, so I'm really starting at this. I do now ask Li Ching a lot of questions, just to see what happens, but I'm really at the start. And this led me to Jung in a certain way, because Jung is obviously talking about the archetypes. I do have to say, though, I found one book, I think it's called Archetypes or something by Jung, and it's impossible for me to understand what he's saying. I think I'm at page 40 now and I have no idea what he's talking about. I have no idea. So I think maybe you can give me some better reads for my humble brain.
[01:14:55.015] Kent Bye: Yeah, I mean, there's been a number of different books that are looking at different ways of talking about the archetypes. James Hillman is a really good introduction. He's a Jungian scholar that talks a lot about what he calls archetypal psychology. Richard Tarnas' work with both Passion of the Western Mind and Cosmos and Psyche explores different archetypes. So I feel like reading the secondary literature on Jung, I mean, Jung has got a huge corpus of people that are inspired by his work in different ways. And I think also just these different cards in the books about Tarot or I Ching, you know, each chapter of that is like, especially for I Ching, it's more of a poetic painting and interpretation that then has lots of information. So yeah, there's lots of different resources that are out there. But yeah, I think in terms of, you know, just talking to Juanita, the director of Floating with Spirits, having this sort of mystical animism that she's trying to do in her work, I think there's a lot of possibilities for other ways of integrating that into You know the experience because there is an experiential quality where you're trying to create these contexts and situations for people to you know It's a big thing that I think is a possibility with VR as you go into VR and you come out knowing more about yourself Well, how do you do that? How do you close the gap and that's such a broad problem but this challenge experientially of how to Give someone an immersive experience as they come out Are they able to learn more about who they are?
[01:16:15.710] Steye Hallema: This is exactly what I would like to try to achieve with Inwards. I'm not sure if it's possible, but that's definitely one of the aims. That's one of the research questions. If this story is about the player, what can I give the player that's actually helpful or meaningful for this person? Totally. Well, what you're talking about, all the philosophical way of looking at it, I partly watched that talk you gave at, I think, StoryCon, and I was intrigued by your system. I think you kind of did it, but I would love to hear your take on the imaginary friend using your system. Because you say, I use everything, so maybe we could go through everything, and you say, like, this is like this, this is like this. I would really like to hear your thought process on this.
[01:16:59.600] Kent Bye: Well, yeah, so I guess that starting with the emotional presence, which is that it's the kind of film tradition where you're using the linear aspects of media and crafting stories to be able to tell the story. It's a volumetric capture. There's kind of limited agency where your agency isn't sort of directing different branches. And so more or less, it's a linear story that at the center of gravity, that's like more from the film tradition in that cinematic sense. You do have a sense of embodied presence where you're as a character You are embodying different embodiments at different parts a lot of times you're a ghost But other times or at least you have wings, but you have different bodies that you kind of cycle through and there's also Moments where you're having the character flap the wings and you have some raising of your height. And so you have this experience of like being embodied in this bird-like entity that is able to fly up and down. And you see a reflection of your embodiment when your friend also embodies the same embodiment and that's a moment where mirroring the same embodiment deepens your sense of who you are as a character because you really can't see yourself until your friend is mirroring your embodiment. And then in terms of like the agency, interactivity, there's a lot of moments where you're asked to speak, which sort of gets into this mental social presence because you're speaking and using the conceits of language. But it's not, like you said, there's nothing that is being interpreted, but you don't know that when you go through the experience, you don't know if you say yes or no. I was sort of doing the yes and, and agreeing, so it was sort of like giving the illusion that I was communicating. But if I would have said no, and it would have continued, then the illusion would have been broken immediately. So there's this plausibility that is the mental presence of what Slater calls a plausibility illusion, where you sort of believe what's happening. The jump cuts sort of plays with that, and there's different moments that kind of break presence with the volumetric capture, knowing that I'm in an immediate experience rather than something that I can truly interact with. So you're taking one form of the volumetric capture, which is traditionally very passive, and you're asking me to interact with something, but you kind of have to break that technology in order to give the response that would be given and appropriate and the agency is less about free will and choice of like exploration of activity and that there's different ways that you're kind of engaging and moving your body and being active by playing these little mini games that are spread throughout and I think the reason why I said the even though it's a more of a film piece I said it's more of a mental presence and social presence piece is because there is this experience of like being in a Relational dynamic with a friend and so there's this friend dynamic. It's a imaginary friend so it's a different type of like a traditional friendship, but you have these different interactions and you're speaking and You have the ideas that are being communicated as to what is the nature of reality? Am I real or not? And so I'm left with this deeper philosophical explorations in the piece. And then there's a story that's there that's unfolding, but it is sort of cryptic and you have to kind of puzzle it together. So there's another part where you're kind of having to solve what the narrative is because you kind of have to work for it. In this conversation, I get a little bit more of the puzzle pieces that I may have not picked up while watching it because it's not immediately obvious about all the different stuff that's happening. You have to pay attention and sort of piece together what the missing parts are.
[01:20:12.160] Steye Hallema: Yeah, what was interesting for me, piecing this whole piece together, I'm taking a completely different perspective on this thing, but it does, I think, relate to your more philosophical and spiritual take on this. I had a really weird trip at the end of making this. So when I had the idea for the Imaginaire Friend, the only thing I did was, how do I do this concept as good as possible? Through this process I have made artistic choices, for example on a storytelling level, using that his mother died as a driver of his position of the drama. My mother actually died when I was quite young. and was ill for quite a while, so I grew up with a sick mother, which when I was a kid was not a per se bad thing. Kids take the world as they are. But just before finishing this, I had a big relationship crisis with my girlfriend, which I'm already very happy with for 10 years already indeed. and all of a sudden the love I have with my girlfriend felt unsafe in a way and that really triggered traumas of my youth in that sense that if that relationship with someone I really love becomes unsafe then there are systems in me start to arise that I can only believe that this will end kind of thing So it was a really weird trip for me to realize. I was at the psychologist at a certain point and she said, okay, in you is a little kid. Now let the kid sit on the chair and sit on the couch. And like all these old stories of mine that was actually were very painful for me kind of came out. And I realized, shit, I have been telling a kind of an emotional story about myself, which I was definitely not aiming to do. Which is so weird. It's really weird. But also very fortunate because there's a lot of people here that come towards me like, I was so touched by your story. So it's also in the same time I created my own solution for my own trauma and people are comforting me in a way. It's beautiful. But it was really weird. So I'm saying this also because Like our subconsciousness kind of like, I don't know, they do other stuff than we think we are doing. It's just to quote John Lennon, like, life is what happens to you while you're making other plans. So I think one thing that VR could bring perhaps is because it's such an experiential medium, is tell stories that make us understand our subconsciousness better. I think that's the same thing you just said, and I was just trying to give an example of how at least making it worked for me.
[01:22:51.385] Kent Bye: One thing it'll say is that in my experiential design framework, I have like the qualities of presence and the character in the story. And like, I tend to be in the qualities of presence when I'm watching stuff, I'm paying attention to the mechanics of the piece. And Paula, who I like to talk to is all about character and story. So when she sees a story, like the way she would describe your story would be way different than I just described it. Because I'm focusing on the mechanics of the technology and the experience and the phenomenological experience of it. And the story comes secondary for me, or at least I sometimes have a harder time recounting each of the stories. And so when I have conversations with the creators, the creators sort of remind me of different aspects of the story because my memory of it is sort of more about the experience of it. So sometimes the story will hit me in different ways. And so, you know, to reiterate the fact that, you know, there are other dimensions of the story that I might've missed that I would have to watch again because the first pass, I'm looking at all these other things and trying to make sense of it. And during the virtual festivals, I would have the ability to watch things twice. So it able to get this first impressions of all these different mechanics and then go back and see what I might have missed and I think that's sort of like what I experienced a lot. Unfortunately, I only have enough time to see things once if that's for a lot of these experiences that's already people have trouble seeing all the work but To then see it twice, I think there's a lot to continue to digest and process. So if I were to watch your experience again, I think now that I know what the overall arc is, there'd probably be an ability to kind of settle into more of the story elements and how did that story work in terms of that. But that's sort of just my bias and my orientation.
[01:24:17.883] Steye Hallema: Yeah, well like that ties back to like if we create something that's about the player, how do we know who the player is? How do we create a shoe that fits everyone? It's impossible. So it's interesting to see like really the different reactions and that's why testing in VR, like testing from the start, like what you're trying to convey is so important. What I personally really find very special, I think 2 out of 10 people that do the imaginary friend cry. Often times it's really wet and it isn't sweaty wet, it's like teary wet, you know, it's like thinner wet. Which I think is really, for me that's really special, that it's able that this... Like I think I'm actually really proud of that because I think this medium is still so we're just figuring out like how it works That it will work to make it emotional in such an impactful way for a certain group of people because we're all different It's something that really makes me happy. I'm happy that I make people cry.
[01:25:14.193] Kent Bye: I'm a monster Awesome and and finally, what do you think the ultimate potential of virtual reality and immersive storytelling might be and What it might be able to enable
[01:25:27.992] Steye Hallema: I have this more, I don't know, technical, futurist ideas. I assume that some sort of pair of glasses will replace the smartphone at a certain time. And then I think a lot of storytelling ways we developed as a scene will be used to tell stories with this new device. Personally for me, I really like telling stories that involve the player and that's why I use VR. So that actually doesn't make me care too much about VR. Like I'm not in this scene because I think it's cool if everyone wear headsets at all. I think there's a beautiful opportunity to share experiences and that doesn't have to be with a headset at all. But I think just the fact that we have a medium that helps us think about to share experiences is very helpful in many ways. And at least for me, because I'm trying to make sense of the experience of my life, it helps me express that in a way that people can relate to. So it makes me understand better. So for me, that's the biggest potential of VR. It's just my trip that I understand life better, my experience of life. and able to share that with other people in a more direct way, I think. I do want to make a movie once, because I think I have a good story, but I'm not so interested in film, for example, at all. I'd perhaps like to write a book, but I think you are going to write that book, because you're just better in that, about how virtual reality works, because I find that very interesting, especially like... the spiritual implications which I was really wonderful discovery that that's also like part of your your research field yeah so like VR is just my medium and I will do that and what it will become I don't know I think for me it's a way to share experiences I think that's the interesting part
[01:27:25.803] Kent Bye: Yeah, afterwards I had sent a couple of interviews I had done with Matt Siegel about the process philosophy and the deeper aspects, but also in his book that he wrote as his dissertation is also part of his book. He talks about the imagination and the philosophical approaches of the imagination. And so, yeah, I feel like, and that's where I got that imagination is an organ of perception of creative process. That is the underlying nature of reality.
[01:27:48.979] Steye Hallema: Is that connected to quantum mechanics, that consciousness actually is changing reality?
[01:27:56.378] Kent Bye: more of a moving away from substance metaphysics and process relational metaphysics saying that the underlying nature of reality is processes relationships and potentia so in that, you know from Whitehead's perspective, it's a Process it's a creative process that's unfolding That's the underlying nature of reality and that everything is emerging out of creativity and novelty and striving for novelty and God is creative Yeah, God is a function of that creative process. The creative process is actually even deeper than God. So Whitehead's theological approach, this process theology, is taking God out of this omniscient part and saying God is more of a function of a creative process. And so it sort of like creates a different orientation where it changes your relationship to that creative process, but that there's the underlying nature of those creative processes that are the deepest level of reality. So your themes of imaginary friend is tapping into this imaginal space, which is actually a part of a deep, deep reality, which is why at the end when you sort of question that, I sort of push back strongly because it's like no, no, no. There's a deeper aspect of the role of imagination and the deepest levels of reality that are nothing but the deepest levels of truth.
[01:29:04.083] Steye Hallema: That's that's why you like in in Donald Duck language explain like this how this works I think like I am really attracted to this new way of thinking of that consciousness is actually changing reaction I think I'm interested in the idealist philosophy that actually Consciousness is actually maybe like the matter that universe is made of I'm saying that wrong because I'm really using the word matter But that's at least it's a big force in the powers that go around in this universe or universe But I'm also still like I grew up in this world that takes substance philosophy and the materialistic worldview very seriously. So could you like explain this to me in a way that that like yeah people can understand?
[01:29:48.529] Kent Bye: I'll try to do my best. There's a great TED talk that David Chalmers has done where he talks about the nature of consciousness and whether or not consciousness is a fundamental nature of reality, so it's at the deepest level of reality, or if it's an emergent aspect of the physicality of our reality. So it's something that's coming out of our physical neurons. If it's emergent out of that, or if it's something that's much deeper and fundamental. So is it fundamental or is it emergent is kind of a question.
[01:30:13.455] Steye Hallema: Why do they base that idea on? Because it's much more logical, because we're living in this materialistic world, to think it's arising from neurons. So what's the underlying thought?
[01:30:24.101] Kent Bye: They haven't come up with like a proof of that to for sure be able to demonstrate that that's where it's coming from There's there's too many gaps and it could be a matter of neuroscience not having high enough resolution to understand all of the Holistic firing of all the neurons and maybe once they have better technology, they'll find that but right now it's basically like it could be just as well that the brain is a receiver for consciousness that's coming from a deeper level of reality and So this is sort of the nature of consciousness is a big open philosophical question that is unresolved and because of that unresolved that's why David Chalmers calls it the hard problem of consciousness is because there's too many open questions around what the nature of consciousness is which opens up the possibility for these other Now, Chalmers himself is more in the realm of substance metaphysics, and this is where we get into the more alternative metaphysical systems where most of Western analytic philosophy says that the underlying basis of reality is stuff, the materiality, the things that are actualized. Well, if you want to say that the building blocks of all reality is stuff, when you get to the quantum realm, that breaks down. Because it's all about these realms of potential and that's where you get Interpretations of like Everett's many worlds interpretation of quantum mechanics saying that anything that's possible actually does get actualized But it's in an orthogonal universe that we can't see So it's a way of kind of resolving that the underlying nature of reality is stuff But saying it's in this parallel universes this kind of multiverse theory that sort of resolves that problem But then we can't observe those so we can't falsify it yet so then
[01:31:53.657] Steye Hallema: Interesting in that podcast that I listened to from you from Matt Siegel that there's not per se like multi universes, but there might be just the potential That I felt like was quite plausible Could you elaborate on that?
[01:32:06.464] Kent Bye: Yeah, just so in the quantum wave function is this realms a possibility of what is possible and so when the wave function collapses you go from what's possible to what's actual and So it's like the collapse of all these possibilities down to what's actual. So most of substance metaphysics is like saying the only thing that's real is the things that gets collapsed into what's actualized. But the process philosophy is saying, no, there's a deeper level of reality, which is these realms of relationships, mathematical equations of the quantum wave functions, these potentials, and for Whitehead, these underlying processes. So there's an underlying beginning, middle, and end of like an unfolding nature, whereas substance metaphysics says that the nature of reality is static and unchanging. But that goes against our intuition that things are constantly changing. And so there's this, rather than centering it in that, the change being a property of the substance, you say that the underlying nature of all of reality is this dynamic flux that's constantly changing. So it's more of a Eastern philosophy, Chinese philosophy, indigenous philosophy. There's more of this sort of Eastern way of approaching the world that is much more in this process relational mode. And Whitehead is creating the mathematical and philosophical metaphysical system that is more in alignment with some of these more Eastern approaches.
[01:33:13.833] Steye Hallema: Yeah, Wu-Wai, for example. I think that's called Wu-Wai, right? Like, it's just everything flows kind of theory. Yeah. Yeah, which makes total sense, obviously, because everything is always changing.
[01:33:24.480] Kent Bye: Yeah. It goes back to like Heraclitus, who had said you can never step in the same river twice, because you are constantly changing, the river is constantly changing. So it's just the idea that this dynamic flux. And so, you know, Heraclitus could be pointed as to one of the original process philosophers in the Western tradition. There's an interview I did with Grant Maxwell that goes through like 13 different process relational thinkers and a couple interviews with Matt that digs a little bit more into that. But for me, I feel like VR is bringing up these deeper metaphysical questions of what is the nature of reality? And so if you say the nature of reality are these relationships and these potentia and the imagination, these creative processes, then you are basically saying that you can get to the deepest levels of reality that way. So anyway, that's why I got these different provocations there.
[01:34:09.430] Steye Hallema: Well, I think it's interesting and I think it's also so interesting how you couple this to VR. Because this is like a process medium. You know, all the time you are in a place where you can change something that has an effect. I think that's already really interesting to couple these two. What is a piece that you would really like to see being made?
[01:34:29.166] Kent Bye: Yeah, well, I would love to see your piece actually. That's exactly what I've been waiting for.
[01:34:35.157] Steye Hallema: Okay, cool. Well, I'm looking forward to make and have a conversation with you again, Kent. And I would like to say on record that I think it's really great what you do. You're kind of like the chronicler of this scene. Like, I love this scene. I just love to how everyone is using their intuitions and talking and we're all one eye and we're trying to desperately open the other eye or maybe we just have one eye, you know, and we're trying to do this together and I really appreciate how you are recording all of us and our thought processes and
[01:35:03.277] Kent Bye: thank you for this it's really it's really cool awesome yeah well thank you so much yeah you're very welcome thanks for listening to this interview from fitness immersive 2023 you can go check out the critics round table in episode 1305 to get more breakdown and each of these different experiences and i hope to be posting more information on my patreon at some point there's a lot to digest here i'm going to be giving some presentations here over the next couple of months and tune into my patreon at patreon.com such voices of vr since there's certainly a lot to digest about the structures and patterns of immersive storytelling, some of the different emerging grammar that we're starting to develop, as well as the underlying patterns of experiential design. So that's all I have for today, and thanks for listening to the Voices of VR podcast. And again, if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listen-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
