I interviewed Below Deck co-directors Martina Mahlknecht and Martin Prinoth at Venice Immersive 2024. See more context in the rough transcript below.
Here’s their artist’s statement:
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.458] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. In today's episode, I'm going to be diving into another piece that was in the college man alley called Below Deck. So this was one of the more innovative mixed reality projects that I saw this year, just because they were blending this kind of theatrical staging with 360 video with some other mixed reality components. And so there's kind of like these three different layers of like there's 360 video. And then on side of that, there is a mixed reality components. And then on side of that, some of the theatrical staging. Yeah, at the end of it was having you question the nature of reality as they're playing with these different fusion of these different lighting with theatrical staging, with virtual reality and mixed reality, kind of like one of the more sophisticated and more ambitious fusions of these different layers of reality. So that's what we're coming on today's episode of the Voices of VR podcast. So this interview with the team behind Below Deck happened on Friday, August 30th, 2024. So with that, let's go ahead and dive right in.
[00:01:17.546] Martin Prinoth: Hi Kent, nice to meet you here in this space. We are Martina Mahlknecht and Martin Prinoth. We are from Germany but originally from Italy and we are presenting here in Venice, immersive island, our project Below Deck. It's a mixed reality experience and maybe you want to proceed?
[00:01:38.364] Martina Mahlknecht: Yes, I'm Martina Malknecht. I'm the second part of this duo that we built together. The name is Tosu. This means like we are coming from the documentary field and Tosu is from the retro-Romanic language that is just spoken in a few valleys in the Alps. It's the oldest language in the Alps and it means like to document or to take something with your hands. And we're working together since 2020 now. And this is the point where also the research phase for this project did start. It's the third part of a trilogy about living and working conditions on the high seas.
[00:02:17.909] Kent Bye: OK. Maybe each give a bit more context as to your background and your journey into working with immersive media.
[00:02:24.747] Martin Prinoth: Yes, I'm coming from originally from documentary filmmaking and my first contact with VR was in fact in 2023. It was one year ago when we started the college here in Venice at the Biennale College Cinema VR. It was in 2022 in autumn, right?
[00:02:48.685] Martina Mahlknecht: It was in January 2023.
[00:02:52.091] Martin Prinoth: Okay, so it was in January 2023. So this was our very first contact in planning and thinking about VR. And so this was a very powerful tool and workshop for us to meet those experts and other colleagues doing VR. And I would say that this is a good point also to thank maybe to Michelle Relak, who was introducing us into this new media for us. And it helped a lot to create this new experience. Because coming from documentary filmmaking, we were thinking about making a 360 degrees film. And this was the only thing we knew in this time. But then we heard a lot of things also from the tutors like Avinash, who gave us insights into these new techniques and possibilities to work with. And step by step we developed this piece and we get a lot of new prospects on this means of VR. And so we created a 6DoF experience out of the first idea of creating just a single experience in 360 degrees.
[00:04:03.583] Kent Bye: And just a little bit more on your background and context of like what kind of disciplines that you're kind of fusing into this process. You mean in terms of your training in film or theater or what kind of disciplines that you're coming from?
[00:04:17.556] Martin Prinoth: Yes, I'm coming from documentary filmmaking. I learned it at the University of Fine Arts in Hamburg. I'm originally coming from the northern part of Italy, from the Alps. And I moved there many, many years ago now. And this perspective on doing documentary is all about the story. It's all about the content. But in the college cinema, we learned a lot of these kind of techniques, how to tell a story, a narrative, in a other way. And not only in the linear way. This was very, I have to say, it was really a hard issue to come to this, to the end, to understand that VR is not like narrating a film. like when you're shooting then you are cutting or first you are writing then you're shooting then you are or even more before you are researching then you're writing then you are shooting then you are cutting but the process in VR is completely different you have to think about how do you want to have the project at the end. Would it be an installation? Can it be a single experience, a multiplayer experience? Should it be three DoF, six DoF or what else? So this is really something that you have to know, in my opinion, you have to know it at the very beginning. And then you can go deeper into this field. And for this, the college cinema was the most powerful tool we did and we had. Yeah, I'm really glad that we participated there.
[00:05:53.546] Kent Bye: Awesome. What about a little bit more about your context and background and your journey into this space?
[00:05:58.382] Martina Mahlknecht: So I'm coming from the field of theater. I'm doing a scenography. I studied this as well in Hamburg, like Martin did, where we met. And we are like going for a long time. We know each other and we know our artistic path. Then we decided to work together because we had this sense that this discipline coming from space and this narrating discipline, they somehow would really match to tell stories especially in an immersive way maybe. And we did see, this was something that happened during the college cinema, that it's an interesting thing how a narrative could be told with a spatial dramaturgy. The viewer can move through a space and like walking is seeing their own experience and their own narration. And of course it was a long path, not knowing the tools yet, looking what's possible, what's not possible. And then we figured out that this pass-through mode, we like that really much, that you have this connection to your own body, that you have a quite intuitive sense for yourself. and that for us it's the most important thing to keep this connection also with another viewer. So we decided for here in Venice to do it with two visitors, but let's say in a smaller scale, but to have in mind that it could also be done with a broader audience possibly. Let's say it's an experiment. And what we also like maybe about this pass-through mode is that you are still connected to the physical world, to your physical body and also to the physical atmosphere. And so we came up with this idea to add lights in the physical space and to synchronize it with the narration that's happening in your headset. So it starts at the same time and then it develops like a narration together and Usually the viewers should not be aware what's coming. It's like merging one into another and you're not really aware what's physical space and what's virtual. And this is an interesting thing where can these two layers merge and you maybe don't even notice, take notice about it.
[00:08:10.308] Martin Prinoth: I would add that for us it has a really natural atmosphere. It feels really natural to be inside Below Deck. We try that the audience can also feel comfortable inside. And what we see that this experience is kind of a blending between theatre, performance and documentary filmmaking. So this is what we really like, to blend these two genres, let's say, like this. And I think we will proceed in this direction. We will try other and make other experiments. So this was our first try and we are very looking forward to do it.
[00:08:48.482] Kent Bye: Yeah, I'd say it's a very ambitious, interdisciplinary fusion of all these different parts from both the formats, but also the design disciplines from theater and immersive theater and filmmaking and XR design. And so, you know, as you're having your existing practices, you know, before XR came along, what was the first turning point where VR came onto each of your radars? And then VR, then mixed reality, because mixed reality is now kind of the latest iteration of where VR started, but now it's more of the XR having more augmented reality. And so it's kind of like all these blending and blurring of realities. When did that come onto your radar as something as interesting to want to explore as a part of your own process of telling stories?
[00:09:31.101] Martin Prinoth: So the first was in, it was like, I had the very first I had in Sao Paulo in Brazil. It was seven years ago and it was terrible for me. My first experience there, because it was like you could fly around and walk around and jump in each direction, but you were sitting on a chair. And this was very problematic for me because I can't do even the This kind of being on a boat, like shaky, it's terrible for me. So this experience was really hard. So for, I would say one year or two years, I was saying, no, this is nothing for me. I wouldn't do this. I would never do this kind of stuff. But then I experienced a 360 degree film and this was amazing. Like doing this experience, feeling inside the story, feeling inside the space. And then when we come up, I don't know, three years ago, with this idea to make a VR experience, then the first idea was to make a 360 degree film. So this was really the first impact and the most natural way to come into this media. But as I said before,
[00:10:43.047] Martina Mahlknecht: I think the 360 film you are talking about was the same that I saw. It was this piece by Omer Fast. That's a visual multimedia artist. And he's using also space in a very important way or in a very... Theatrical maybe?
[00:11:00.119] Martin Prinoth: Theatrical way?
[00:11:03.062] Martina Mahlknecht: And maybe this was an inspiring moment that we thought when we later came up with this idea about telling the story below deck about an area on cruise ships where passengers or press even does not have any access then we thought oh this could be a story that could work for VR it's a space you don't have any access then in the end we decided not to shoot really below deck because of this reason it's not possible to enter but we decided coming from the documentary field to work together with real people and we were very lucky that in Hamburg we find five people that during corona decided to leave their jobs and and to move to Germany and so we had five people we could work together we did a long research phase we exchanged about what they lived on the ships and out of that we developed the scenography
[00:11:57.579] Martin Prinoth: I want to mention the names of the protagonists, can I? Because this, like Martina mentioned just right before, these are people that are unseen, people that their stories are not heard by a lot of people, so I would give many thanks to Maria Antigua, to Mary Grace Wesch, to Manole Castillo, and to Gadi Santos and to Vanessa Fauvel. These are the five protagonists and they shared their stories with us and this was a gift for us. And it was so important for us, this corona pandemic. The good thing about this pandemic was that we could find these people, otherwise it wouldn't be possible because they were stuck in the port of Hamburg and so we could meet people And we could get to know these people. And a big thank you also to Manuel Castillo who lead us to the other protagonists.
[00:12:58.837] Kent Bye: And you mentioned that Below Deck is actually a third in a trilogy. And so maybe give me a bit more context for what happened in the first two chapters of this topic that you're exploring.
[00:13:09.306] Martina Mahlknecht: The first two parts of the trilogy were, so in general the topic is the living and working conditions on the high seas and we were so interested in that because it's so connected to our everyday life. The first part was with workers on container ships and they are like container ships are providing us with 90% of all goods that are around us or that we are using in our everyday life. So this was the point that we said we want to work with these people, we want to make it a bit more visible or to give it a bit more visibility. And as Martin said before it was in 2020 when a lot of seamen were stuck in the port of Hamburg and we had this chance to have personal contact because usually so many years ago it was like when ships came to the port Seamen were getting off. They are getting to the city. They had some days off and you could meet them. You had a chance to meet them. But nowadays harbors are working really differently. They don't really get off. They have really short breaks to take a break. So this contact, even if you're living in a harbor city, is very limited.
[00:14:14.548] Kent Bye: And were you also working on this project as well?
[00:14:20.875] Martin Prinoth: Yes, this was our second work we did together, right? The first was Hack Me Baby and then this trilogy and did you talk also about the second part? So what was the format of what this was? So the first part of this trilogy was an immersive film installation. It was also XR. We shot with six people from Kiribati. It's an island atoll in the Pacific Ocean, the biggest in the world, but a few people know about, you know about.
[00:14:55.097] Kent Bye: Curacao, is that what we said? Or what was the name of it? Kiribas, Kiribas. Oh, no, no. Marshall Islands. What's the name of it?
[00:15:01.221] Martin Prinoth: Kiribas. It's the biggest atoll in the world and in the Pacific Ocean. And we had the opportunity to work with six people, six seamen. and we shot with them a few scenes and then we integrated this in a room setting. It was a huge room setting with a light installation and sound installation and people could step into the water with boots on and watch their stories on tablets. And there was a second part where people could experience a big screen where the crew members performed a very traditional dance. It was the collective moment from this work. So this was overseas. The second part was
[00:15:47.388] Martina Mahlknecht: If I may add, this is a topic that we are always coming up in our work. Let's say this experience you have alone and a common experience. And in our last work, it was always like you have a part of it is like an individual experience and part of it is a communal experience. And this for us is like has a strong power to have these both moments.
[00:16:10.717] Martin Prinoth: Then the second part was the staff your dreams are made of. It was about crew members working on cruise ships as well, also below deck. It was a one-to-one reconstruction about a cabin into the crew area. where people could enter with four people each time slot enter this cabin and experience this tight room and listen to the stories of these crew members. It was quite similar to an installation in a museum or in a theater space. And now below deck is the third part. It's our first MR experience.
[00:16:52.931] Kent Bye: Glad to be here with this. Great. Okay, so then you move into Below Deck, which I feel like is kind of like a continuation of the stories that you're exploring around the life on the high seas and different worker conditions, essentially. But also, this project particularly has a lot of the technological innovations of really finding a way to... blur the lines between the captured 360 video with the mixed reality with also the theatrical staging of the place in which it's being displayed the lighting effects and everything so there's like the nested contexts of like the video and then the next context of mixed reality and then the next context you have maybe some VR components but then also like some elements of the theater and then how to jump between these different layers and I feel like this project is you know it's called below deck and you're talking about the lives of these workers on the high seas but you're also blurring and blending the realities between you know each of these different contexts and so I guess when you're thinking about doing this mixed reality documentary exploration with some theatrical elements to it where did you begin with how to construct this story that you're telling here
[00:18:06.450] Martina Mahlknecht: Interesting question. That's always difficult to tell afterwards where was the beginning and where was the end. It was a lot of trial and error maybe let's say. We knew what components we like but we played a little bit around with that and we worked with a small stage model and paper and all this physical stuff and yes and we tried what could be a setting what could be a setting to tell the story and to What was important to us also is what is happening if you're putting on the headset? Where are you finding yourself? Are you immediately in another space? Or could you also start in the setting that you are in and then somebody is like taking you from there to somewhere else very naturally? And this in the end was then maybe a very important point for the decision to have these two areas. We have one area in front of the installation, which is part of the exhibition setting, of the onboarding, right?
[00:19:09.762] Kent Bye: So you're kind of entering into the portal of the ship. So when you get into mixed reality, you see the big, large ship, but then you have this circular cutout of the wall with a yellow curtain on the other side. So the yellow curtains end up being a motif that's like the connective tissue between these different realms because the 360 videos have that curtain, but also the outside of the staging, the theatrical staging also has that. So you're kind of using that as a way to go in between these different realms.
[00:19:37.566] Martina Mahlknecht: I couldn't say it better.
[00:19:41.889] Martin Prinoth: Also to talk about the process we did. So the first was the shooting with the protagonists. It was the shooting with classic 360 degrees.
[00:19:54.237] Kent Bye: So just before you get into the shooting, because you are shooting with, it looks like the yellow curtains are there. So it seems like you had to figure out some of the mechanics of the technology before you shot. Kind of doing a proof of concept of that as well. So, it seemed like you had figured that out, or you figured it out afterwards.
[00:20:12.531] Martina Mahlknecht: You're right, we had this concept from the beginning, let's say, or the idea was we were thinking about how to tell this story if we can't enter a real cruise ship. But we did not want to enter a cruise ship anyway, because this was not what we wanted to tell. We wanted... to create like more a setting with an atmosphere of this strange place that you can't even if we could have the possibility to go there via 360 video we could not have an idea of what people are living there and so we said we have to tell it in another way and the inspirational moment for this was they told us that every week on a cruise ship there is a cruise show happening This is where the crew members are performing for the guests. They are showing their talents. And then we said, oh, this could be a moment we could start it. And we want to show this rehearsal for the cruise show. And then we came up with this setting that maybe refers a little bit to Lynch or somehow it has this strange, surreal character. Maybe you are moving in a world. That seems maybe dreamlike, not real. These places you're looking at or these spaces can't really be positioned like this. But as there is always a porthole in these spaces, it's on the ship or it's located somewhere. But it's a dreamlike setting, I would say.
[00:21:37.628] Martin Prinoth: We chose this kind of crew evening because we like to show or to tell something about playing a role and so if you are a crew member but in this crew evening you are also like an actor so you are acting a role and therefore we love this kind of curtain so what is in front of the curtain and what is behind the curtain what can we see what can we not see And this is what we like to do, that the audience can enter and watch behind the scenes, behind the curtain, and discover something that is maybe unknown or unseen. And also to reflect the medium, the headset, the VR medium as what makes VR visible for us, but what is still in disguise, remains in disguise. So this is what we loved to play with these realities to say is everything around us real or is that real that is behind a wall or behind a curtain? Is the world real, more realistic, more real if we put on the headset? What is it? So this kind of playing with the medium itself and with the roles and with acting and playing their own reality. So documentary, fiction, VR, pass-through reality. So these kind of layers of reality, this is what we wanted to experience with it.
[00:23:03.874] Kent Bye: Yeah, there's quite a lot of innovative architectures of how this piece is put together because you're able to create, with a very limited space, a way of transversing and moving your body through space so it feels like you're going from location to location and then there's Another person who's also going along with me and so we're crossing paths and I don't know if that was more of a throughput issue of having the same set and having two people see it at the same time or if there was more of like a narrative purpose for having like all these collisions or trying to avoid them. in this little small pathway as we're walking around and so there's a conceit where you're walking around in circles around once you enter in the ship there's a curtain that's also in the circle and you open up the curtain well there's kind of like a whole mixed reality dimension of a curtain that opens up and reveals this whole other layer of the 360 video because like in physical reality there's actually no curtain there and there's also a curtain on the periphery and so there's a blending and blurring of reality where my mind when I just said the curtains pulled back is like my mind was interpreting my memory of that as if there was a curtain there but there actually wasn't a curtain there but it's like a lot of really subtle ways that you're using lighting effects to kind of build it. It feels like a big magic trick that you're doing in this piece that is kind of playing with how can you play with the mixed reality that's blended with 360 video, but also with the theatrical staging to be able to have an architecture that makes sense for allowing the audience to move their body through space, but also open up these curtains and get these little snippets of a story that you're telling. And it feels like there's a lot of moving parts in there, but For me, I was actually more paying attention to the magic trick of it all, of how the technology was working. It was more difficult for me if you were to ask me, okay, what was happening in the beats of the story? Because I'm kind of a voyeur. I don't know the exact context. I'm not sure who these people are. I'm kind of like... trying to puzzle together. But each scene, it's different characters and different contexts. And so it was hard for me to track the through line of what that story was. But that's why I'm kind of wondering, as you were putting together, how you put together the story elements of telling the story of the conditions of these workers.
[00:25:16.933] Martin Prinoth: In fact, the story, we thought about not to tell the story in a linear way. So there is no linearity where you can say, yeah, the story begins here and ends there. So if you did manage to get all the stuff they said, it's no problem. That's okay. And it's not like this movie play where you have to have every information about linear dramaturgy. But a very important moment was when we decided to integrate the narrator, like Manule, who is welcoming us when you take on the headsets. And he is leading us through this experience. This was a very important point. And he's explaining you and also introducing you some protagonists of the living on the ship. And he is the guy who is giving you this kind of information, what we think is very helpful to understand the whole concept about this story. But yeah, it's also important for us as a filmmaker and theater maker, it was really hard. to let the dramaturgy also a little bit flow and not to say so first is this and then cut and then it's the other one and the other scene and the next scene but to say okay let's have the audience the viewers experience themselves so if one is more excited about experiencing the technique or the space so it's okay you don't have to experience the dramaturgy step by step So this was at the beginning very hard for us to say let's keep this for the audience and not to have it like right on the line. And so maybe you want to add something to this?
[00:26:59.882] Martina Mahlknecht: Right, because if you're wandering around and you are just catching the feeling, you still have the voices in your ear. Maybe you hear that they are coming from another part of the room, but you still get it. And you can get maybe more this atmosphere about being in this dreamlike landscape below deck.
[00:27:21.709] Kent Bye: I think when I see projects, I'm often looking at different levels of the project, like the qualities of presence, the context, and then the character and the story. And so there's always a lot to put together. And I feel like in this piece, there's a unique mechanic that, as an audience member, I have to figure out, OK, what is the structure of this piece? OK, I have to walk in a circle. OK, now I have to find the next curtain opening to get the next narrative bit. And then that closes. And then I have to kind of do another circle, avoid both the other participant. And then you were also there filming. So I was sort of navigating around you filming. And I'm like, OK, how do I? I'm just trying to find. And then the next curtain opens. And there's another little narrative bit. But then I was like, how's that connected? So if there was narration, I wouldn't be able to tell you what was said. So if you were just kind of like to recap what happens in the narrative parts, just to kind of help refresh my memory, because there's different scenes I remember, like kind of a dressing room, there's a cake, there's a chef, there's like a party, there's a talent show that's happening. So there's little like these beats, but yeah, if you could just describe, and you also say it's in a nonlinear fashion. So I don't know if there is an arc, but maybe you could just kind of describe what happens in each of these different scenes.
[00:28:36.673] Martina Mahlknecht: Yeah, there is a dramaturgy and it's this idea of they are preparing the crew show together. So there is Maria who is the entertainment director and she's like preparing everything. There is Grace who is like a worker on the ship and she's really working in a mechanic machine room. But she's a very talented singer and she's preparing her song. and this scene. Then there is Manole, he's the pastry chef and he's preparing a huge cake for the captain's dinner and this is happening through the whole experience, you don't see it the whole time but it's happening. And there are some scenes in the wardrobe. The wardrobe is the place where they are, while there is this rehearsal on stage, in the wardrobe the people are talking to each other, they are also telling a bit more intimate memories that they had on board. So this is the moment where we are not on stage, so we are in a more private atmosphere. And then in the end, I don't want to spoil, but in the end there is a moment, also a very common moment, and we don't really know is it real or is it just a test. There's an emergency moment that we all live together. We have it in the physical space and they have it in the narrative story. And this emergency does create this communal moment where we all meet. They are missing Manu in this moment. Manu is the pastry chef. And we all are starting to look for him. Where is Manu? Did we lose him? Is he still alive? And then in the end, maybe we can leave that open if we find Manu or if we don't find Manu.
[00:30:11.780] Martin Prinoth: What I want to add is that these are these communal scenes on stage. There is a stage where they are rehearsing and I find it really funny. They are trying to rehearse a sketch, a funny sketch, but the sketch isn't funny because they are trying to imitate the tourists on board of a cruise ship. So they are the low wage worker rehearsing a sketch about the tourists on board and this is not working and it's not fun at all. But then there is this kind of moments on stage and as Martina said before there are these more intimate scenes behind the curtain, behind the stage where the crew members are talking about their experiences on board, their real experiences on board And there are, I would say, really intimate stories about homesickness, about problems being away from their families and also about living as a woman on a cruise ship where a lot of men living there without their wives and families. And so I would say it's a combination between very intimate stories behind the scenes and about more
[00:31:30.067] Martina Mahlknecht: let's say funny or more outgoing scenes on stage and maybe in the end we could add that it is ending with the preparation or with a final cake and in this moment we have maybe more layers it's like asking these questions the stories or the narrative layer is like disappearing and it's asking maybe this question or about what is real or what is Like making everything possible. Maybe what is making possible the storytelling? What is making possible the life on a cruise ship? And also which stories can be told and which are told and which are not.
[00:32:14.685] Martin Prinoth: and which kind of people are we seeing in films or in theater or in VR and which not is kind of questioning the end but I want to spoil it but yeah this is the question what remains at the end I would say.
[00:32:28.496] Kent Bye: Yeah, I'd say the very ending of it just definitely felt like a magic trick that kind of reveals this blending and blurring of reality where I kind of had to peek underneath my headset just to see, like, is that real or is it not real? Because there was a bit of, like... Because as we're talking about this, there's the outside of the installation that has a curtain. It seemed like you had some fans that were blowing. It had some, like, real dynamic. That's probably just the air conditioning or... It felt like it was alive and also clear enough for me to notice that it was part of the physical installation. And then there's the inner curtain that's the mixed reality, but then it opens up automatically. And then you have the 360 video. So you kind of peek your head into like 360 video and it's got this kind of You know, often with 360 video, it is like when you move around in a 360 video, you can't actually change your perspective, but there's at least a way that you shot it that you can put your head in and out. And there's like a layer of like the mixed reality or VR that kind of has a way where you have a doorway that you're peeking your head through that you can kind of see. into the 360 video so the 360 video you can look in and actually see like the full fullness of 360 video and then take a step back and you see the mixed reality curtains that were there and then within the video on the edges you have like the curtains that are shot there so you have this seamless way that you're able to embed a 360 video in the context of mixed reality in the context of a theatrical installation that is all coherent that is reinforcing this illusion that you're able to play with in a way that is deconstructed in a way that makes you question the nature of reality at the end. The thing that I didn't know is like, how does that tie back into the story? Which was like, I was already having a hard time tracking the story. And so when it ended like that, I was like, okay, what was this piece about? Because it was hard for me to like know like, Well, if this is the finale, then what was the other scenes that I was seeing? And I think the other thing just to note is that sometimes there's an adage in film. It's like show don't tell. And so sometimes when the narrative bits of a film is all through dialogue that happens and I don't see what's happening and it's like a context of like, oh, this is the aftermath of them talking about their memories or their experiences without showing them. Then my memory of that tends to be a little bit more fragmented or fragmented. more of a cognitive load to try to take the abstractions of what's being delivered through a dialogue and the more diegetic things that get lost throughout the spectacle of processing everything else that's happening in the experience. Maybe that's just my brain of trying to understand the medium and what's happening with the medium. And I'd have to watch it again to get more of the story, but that's at least part of the hardest parts for me were to kind of track to see how the story was tying back into all the other architecture that you had built.
[00:35:12.751] Martin Prinoth: What I really like from your description right now is that you as a viewer, you can go nearer to the story or take a step back and watch it from the distance. This is a very good description. I like this very much. Thanks for this. Because it's like having this, you as a viewer, you also question yourself. So how am I connected with these people behind the curtain? And am I connected? And in which way? Do I want to be more nearer to them or far away? Do I need more distance or more tightness to the people and to the story? So I like this very much. So you are kind of a voyeur watching or not watching. So is it okay that I can also question yourself? Is it okay to watch these people talking about their private lives and private stories? Or should I take a step back? I like it very much how you described this kind of your experience you had inside. I like it very much.
[00:36:13.758] Martina Mahlknecht: And yeah, me too. It's very interesting how you describe it, this multi-layered and how you are, if I understood you right, also the question what the end is referring to, as you maybe were also concerned with other things in installation. But maybe for the end, we could also add, there is this question in the end when everything is like disappearing and we are back in reality, let's say. And there's this question, but magic does not exist, does it? And that's really an open question because whoever knows. But maybe what we like about this question is one, it's on this topic layer that people are working and are making things possible. But they are hidden from our eyes. And this is the magic, how it works. Somebody is doing the hard work to make magic possible. This is how it is referring to the stories we have heard before. But there's also this other layer. It's like, but magic does not exist. Maybe also referring to the medium. It's like something is disappearing. It could be it's VR. It could also be a film. It's like it's also kind of a magic tool that we love to work with.
[00:37:28.736] Kent Bye: In terms of the narrative, was magic discussed as a topic within the story? How does magic appear earlier in the piece or does it?
[00:37:38.965] Martin Prinoth: I would say the first magic is when you are here filling in this space on the island here in Venice and you are watching through this box and you're seeing this circle entrance and then you put on the glasses and then from the real curtain protagonist is coming out so you have the feeling that the person is coming out from this box and talking to you so maybe this could be some magic in film you could cannot do this in in theater you can right you can do this but then at the end he started to open up a postcard and then you feel like into this postcard because around you this postcard gets unfold And so this maybe is the second magic. And also when you go into the box, there are a lot of people not knowing exactly, is this now a real curtain I'm entering or is it physical? That's really a very complicated word. Is it physical, the curtain, or is it VR? And so maybe this is the third time where magic became true.
[00:38:47.219] Kent Bye: yeah at the very beginning you have the the kind of circle cut out with a yellow curtain and then when i put on the vr headset you have it calibrated such that you have replaced now this yellow curtain with the video of the yellow curtain and then i didn't notice that until this actor that you filmed or the one of the documentary subjects sticks his head out and you get to see his face and at that point i was like okay that's not a person that's here and they just tricked me like just like they're able to build up the illusion of entering into a ship so that was a really slick moment and the other thing other moment i just wanted to call out was the the way that you were using the lighting and the how the lighting was reflecting off of the actual curtains but also like you were doing virtual lighting on the mixed reality curtains and so just a the way that you're using lighting to help build this sense of environmental plausibility to kind of really sell this, you know, because you actually have some lights that are there. So I guess bringing in more of those theatrical staging elements to have ways that you can blend these different worlds together and have them feed off of each other to help give this extra sense of actually being there.
[00:39:56.823] Martin Prinoth: I don't know, I can't understand how you did, you can also cut it out, how do you manage to know about this and to talk so clear about all the projects you've seen, because you've seen more than 60, right? So, yeah, it's fascinating to hear you talking about it. It's not clear for me how you managed to do this, but you are, yeah, it's fantastic.
[00:40:22.578] Kent Bye: Part of it is the experiential design framework that I have creates a memory palace for how I'm paying attention to different stuff. And so like sometimes I'm paying attention to a lot of these things, but not able to pick up on the details of a story. So like I think some people, if they go in, they may not be paying attention to the other stories. ways in which it was constructed but they'll just kind of be focusing on this story so I think that's it's a trade-off that I have sometimes when I have a piece because it's like I'm trying to pay attention to as much as I can there's there's always stuff that I miss or like the sound design or other parts but yeah so anyway that's kind of like I've got my kind of memory palace that I use to try to kind of code in my experiences and go through enough of them to experience it, try to remember it, put it in my body. And then throughout the course of the conversation, then there'll be things that just pop in my mind around like this moment or that moment in terms of things. So yeah. Anyway, I guess as we start to wrap up, I'd love to hear what each of you think the ultimate potential of virtual reality, augmented reality, mixed reality with this kind of immersive storytelling and theater fusion of all these different disciplines, what that ultimate potential might be and what it might be able to enable.
[00:41:29.966] Martina Mahlknecht: I would say we are very much looking forward what happens also to this pass-through mode because we do especially like that you keep a connection to your body that there would be the possibility to have more moments not just the individual moment but a shared moment and also to have I like also the idea of avatars but for me I also want to have a more connected experience So I'm looking forward to what's happening there and to experiment further also with the combined light installation and to work with the synchronized levels and layers.
[00:42:08.257] Martin Prinoth: Yeah, we are also thinking about to integrate VR technology in film and the other way around. So we are really looking forward for new projects. We are in the development right now. And hopefully we see us again here, Kent, on the island.
[00:42:29.146] Kent Bye: Nice. And is there anything else that's left unsaid? Any final thoughts you'd like to share with the rest of the immersive community?
[00:42:38.188] Martina Mahlknecht: Thank you very much, Kent, for the talk. I want to share this. And everybody for watching the Biennale College Cinema, the Biennale edition of this year. We're very proud to be part of it.
[00:42:49.991] Martin Prinoth: Yeah, thanks again to Michelle Relak and Lis Rosenthal for having us here. It is mind-blowing and a great honor to be here. And we will experience a lot of other experiences here. And now I would ask you, Kent, if we can make a photo with you together.
[00:43:07.975] Kent Bye: Yeah, yeah, for sure. I'll just have my final thought here. Just Martin and Martina, thanks so much for joining me here on the podcast to help break down what I think is real interesting interdisciplinary fusion with a lot of this theatrical staging that the lighting with the mixed reality 360 video. There's a lot of interesting ways that you're exploring how all these layers are kind of interfacing with each other and how you can have people navigate a space and have it feel like we're kind of exploring a space in a way that is able to, I don't know, use the medium in a way that I think is very evocative in terms of where you can take something like this in the future. So yeah, very much enjoyed kind of helping break down all the different components of Below Deck. So thank you so much for joining me to help tell your story. Yeah.
[00:43:52.885] Martina Mahlknecht: Thank you. Thank you very much. Thank you, Kent.
[00:43:56.055] Kent Bye: Thanks again for listening to these episodes from Venice Immersive 2024. And yeah, I am a crowdfunded independent journalist. And so if you enjoy this coverage and find it valuable, then please do consider joining my Patreon at patreon.com slash voices of VR. Thanks for listening.
