I interviewed Rencontres director Mathieu Pradat at Venice Immersive 2024. See more context in the rough transcript below.
Here’s their artist’s statement:
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.458] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So continuing my series of looking at different immersive stories from Venice Immersive 2024, today's episode is about a piece called Rencontre, which is encounters in French. So this is sort of like a social VR experiment. It's got a lot of sparse, poetic, architectural elements where you're kind of interacting with these different NPC characters and giant chickens and then these planes. It's trying to use a lot of symbolic logic to tell a story, but it ended up for me at least being a little bit more fragmented and kind of hard to decode and understand everything. And so... wanted to sit down with the director just to understand a little bit about the design intention and then all the different constraints he had in terms of ways that he wanted to have limited amounts of explicit instructions for what was happening from scene to scene. And then how that ties together to this larger narrative around climate change and this kind of solastalgia feelings that he has around that, but also trying to do these emergent social dynamic experiments, I suppose. It's kind of a fragmented experience overall. And I just wanted to get a little bit more of an insight into the design intention and then how I experienced it. And then, yeah, it's just some ways that you can kind of close the gap between what he's intending and then what the end result is. So that's what we're coming on today's episode of the Voices of Y'all podcast. So this interview with the director of Rencontre happened on Sunday, September 1st, 2024. So with that, let's go ahead and dive right in.
[00:01:52.422] Mathieu Pradat: Hello, my name is Mathieu Prada. I am a French author, immersive author and cinema director based in Marseille, south of France. I've been working in VR and immersive installation using VRXR since 2015, almost 10 years now.
[00:02:11.190] Kent Bye: Maybe you could give a bit more context as to your background and your journey into VR.
[00:02:15.615] Mathieu Pradat: I've been trained as an architect and I've worked as an architect for 15 years, more or less, doing films while I was working as an architect on architecture, landscape, urbanism, and also first ventures into experimental film and fiction, short fictions. And I rediscovered VR in 2014, I think at Leipzig with the documentary Polar Sea from Thomas Wallner. a German-Canadian author, and I was super fascinated. I had known VR as a tool for architects on flat screens, and to see this tool being able to be used in storytelling to me was like a major enthusiastic discovery.
[00:03:01.831] Kent Bye: When you say rediscovery in 2014, when was your first discovery of VR?
[00:03:05.592] Mathieu Pradat: My first discovery of VR was in the 90s as a student in an architecture school with VRML, with VRML 1 and 2, I think. I just loved it. I loved playing around with it and doing 3D models. I really loved the tool. Our teachers back then, they really hated it. They considered it was just a gadget, but I loved it and I used it quite a bit in this period.
[00:03:29.013] Kent Bye: So 2014 is also when I got into VR. On January 1, 2014, I bought my Oculus Rift DK1. And so anybody who got into around that time, it's been about a decade now that you've been involved in XR. So maybe catch me up a bit on what you've been up to in the realm of the XR industry for the past decade.
[00:03:45.982] Mathieu Pradat: I started by doing 360, 3DOF movies, VR movies. I really thought it was an extension of cinema at the beginning. There was like a kind of a like aesthetic maybe approach and also a marketing approach and making a confusion on total cinema. It could be the total cinema of Andre Bazin and Barjavel. And I was like, okay, yeah, so maybe that's a way to go directly, more directly to cinema, which I was trying to do at the time, do fitted film and long documentary, long-form documentary. And it took me a while, it took me like the first, I had to do my first works in VR to understand that it was a completely different medium. And actually, I think I understood this very clearly when I was selected here with Proxima, my first VR film in 2016. And I could discover works such as Separate Silences, Draw Me Close, Draw Me Close that I loved, Alice. And I, okay, so it's about space as well. It's really about space and being together in a simulation. And then, so this festival with this kind of aesthetic approach and also the authority that Michel Reyac and Liz Rosenthal have, the kind of moral authority to make a statement, okay, these are works of art. really helped me to grasp what it was about and made me even more enthusiastic.
[00:05:06.551] Kent Bye: Yeah, architects are some of my favorite discipline to talk to because I feel like it's one of the last disciplines that's like a truly interdisciplinary practice that is fusing in so many different types of design disciplines that you have to all weave together. So it seems like a natural extension for architects to start to think about how the XR medium can start to not just modulate space, but also architectures and contexts for people to connect and to think about the social dynamics. And so maybe you could speak a little bit about your lens of architecture and how that makes you see the medium and how you start to use some of your practices of architecture through sketches or blueprints or whatever it is to help bring all of those different design disciplines into what you're doing now in XR.
[00:05:49.749] Mathieu Pradat: One of the basic things that is expected from architects is to see in space to be able to visualize space and that's really an exercise you have to practice and that helped me a lot at the beginning shooting with small cameras with not a lot of money so I could visualize even when I had no monitor no visual feedback so that was like a concrete help but more than this I think to me what is super exciting and what I like about creating in VR is that Space can be bent to the user's desire. It's a totally dynamic relation to space. In architecture, real architecture, we are in a dynamic relation that is solely driven by us. We are moving around in this architecture promenade like theorized by Le Corbusier. We're moving around, unraveling the space, and this is dynamic. with VR then the space has agency so this dynamic relation becomes really truly dynamic because it works both ways in a dialectic in a kind of a dialogue dialogic way so this is I think a new a totally new thing that doesn't exist in any medium we can have like XR can be super close to immersive theater it can be close to video games can be close to literature but this is specific this kind of being in a simulation the simulation is also talking to you in a way, and I think that has so many possibilities to be explored.
[00:07:16.610] Kent Bye: Great, and so maybe it's worth giving a bit more context for the project that you're showing here this year at Venice Immersive 2024, and how Recontra... how do you say it? Rencontre. Rencontre. I say Recontra because I can't say... Encounters also. Oh, Encounters. I'll say Encounters. So maybe you could give a bit more context for how Encounters came about.
[00:07:37.882] Mathieu Pradat: Encounters came about late in 2019 meaning that it's quite a challenge to bring those productions multi-users LBE in situ to life and it was coming I think about like lucid dreams I had about a global feeling of solostalgia of indifference towards climate change about also helplessness like okay there is really nothing that can turn it back or turn it around And maybe it blends, you know, it's hard to speak about inspiration, about feeling that I'm more close to me in a way, not abstract, I mean like indifference. Indifference in society, like sometimes I have a feeling, you know, people around me or maybe when I was younger can be very indifferent and I could suffer from this. So I try to bring a personal part of me also in the work. And overall, this thematic of climate change, I wanted to bring it not bringing indifference as a result, but indifference bringing climate change. Being the driving engine of climate change, like the, okay, we don't care about each other, we don't listen. We don't care about others destiny and this is why this is going just like nowhere or maybe in the wrong direction, you know So that was the first inspiration then the first chapter called the crowd came like this and I had this like intuition of simulating worlds, very minimalistic worlds at the beginning but with users interaction they are going to be saturated and really like saturated to the point that sometimes was difficult to code and to have the hardware that could follow up with the experience because we have at one point in the experience at three points so many assets in the headset so many things going on and I developed this first chapter into two other chapters the second one is called the birds where you can feed chicken and you can bring them back to maybe an original dinosaur state. Kind of creepy and also funny, I hope. And the third chapter, I hope to be allowing people to unwind, to be also exhilarating about aerial plane transport. Because, of course, with climate change, we are all part of this. I'm taking planes and sometimes I feel very terrible. Okay, I take too much planes in my work and... I felt like, OK, we also in this work, Rencontre, we need a way to just unwind and to do large gestures, to destroy small planes, to have huge explosions and just to feel a bit liberated of all this kind of pressure and thoughts that are contradictions going through our minds.
[00:10:21.550] Kent Bye: Yeah, I feel like there's a number of different strands here to pick up on. One is the design aesthetic and how it looks and the inspirations there, but also there seems to be a number of social dynamics and interactions that are happening here in terms of how the six people that are in the experience at the same time are then also interacting with these in PCs that are the same exact look and feel. And so there's like a blending and blurring of who's a actual physical person and who's a virtual NPC. And then there's the kind of three chapters and the story element. And then there's the different rules of this world that there's not a lot of language, not a lot of speaking, a lot of gestures, a lot of communication with symbolic language, I'd say, in terms of how the narrative is being presented to the user, because it's a lot of symbols that are to be decoded that we can start to decode here. But as you're coming up with this arc and this journey of the overall experience, let's first get at what was the invitation for people to come into this space, and what did you want them to come out of at the end of it?
[00:11:20.071] Mathieu Pradat: The whole thematics of Rencontre is kind of tragic and dramatic, but if we all go for the drama, it's going to be painful and maybe it's not going to reach any goal. It's not going to bring any awareness or any change. So I also wanted people to feel good together and to be amused, to have fun, sometimes to feel helpless, but sometimes also just to have fun. And so there are some... There are a lot of interactions. It's a very, very interactive piece about everything. Every gesture, every move gives a possibility, an opportunity to interact and to find your own way in the story. But it's not necessarily an interaction that you know where it's heading to. Because I think when you know where the interaction is heading to, then it's closer to a game, to a gameplay. And so the point will be to have a good gameplay. And there is no true gameplay. Rencontres are just suspended moments that are going to lead to other moments. And sometimes you have to find a way to unfold the narrative. For example, in the first chapter, those NPCs are just talking together and leaving you on the side. And if you try to break the circle when they are talking together, then they will disappear and laugh at the audience. But this doesn't work in the same way. Every time you enter a circle, the circle goes bigger. But you need more people to enter the circle to trigger it. So there is a kind of something tragic about this. And you feel laughed at. You don't feel really comfortable in the experience at this point. But it is also fun to do it in a way. So I try to bring those two approaches, fun and also feeling of... importance at times. But I don't want to leave people with this because that's my own personal belief. I'm kind of in a way very optimistic because I think change is very possible. We can do change. We can really achieve this. even though it requires something in my view extremely hard is to pay attention to others and this is extremely hard because we are so concerned and so many of us have so many problems to deal with so many uncertainties in life so it's asking very much of us but it's really within our reach it's possible in the human destiny i think to to achieve this collective awareness and to change our destiny in a way So, I wanted to lead the narrative to this on the last chapter, and it's called Rencontre for a good reason. So, paying attention means really meeting people. When you start really listening, you're actually meeting people. I'm not saying making friends every time, but true encounters, true like, okay, I know how this person feels. I know how this person, what is concerning this person. So, I wanted to have this kind of symbolic conversation. approach to this in the last chapter with um with a dialogue a simple gesture dialogue with a like imagine you're trying to talk to somebody and you have absolutely no possibility of using language we don't talk the same language how do we do so maybe we'll start with language and maybe this would lead to uh paying attention maybe this would lead to tenderness in a way so there is a after this first scene in the fourth chapter there is a virtual hug where you can actually hug yourself in a way and do yourself good but you can reach out symbolically to this npc and i think that's a very strong moment i i talked to it was good yesterday i met people who talked to me about this and told me how they felt about this in the experience they felt it was very Emotional so I was really happy and this is leading to a last part that is still in construction because here is our premiere and this last part is a restitution of recordings of users from all ages gender race countries Making a short statement in one breath like a Japanese haiku a few words So in this part, there is just their voice and gestures that are being recorded and this will be replayed in the near future and the next time we do the experience we'll start to replay some of those recordings that we will have curated beforehand because we we now have quite a few so we need to curate and use the one that seem not the most interesting necessarily but the most relevant also in the context of the work and start to replay these recordings and also build more and more recordings and make this experience very rich. And we'll see what we do with them. Maybe we'll only use them within the experience. Also, maybe at some point we, I don't know, how we will like maybe do a podcast or present, have a possibility to listen to other recordings after the experience. We'll see what we do, but we're really super excited to be starting this in Venice.
[00:16:14.952] Kent Bye: Well, the piece, it feels like the structure and architecture is very non-linear in the sense that it's got these sequences of moments and encounters that, you know, it's a very sparse architecture in the sense that it's like a black void, but with characters that are there that are NPCs. So there isn't a lot of context to understand, like, what are the proper social dynamics or behaviors that you should be doing. So it's a little bit like a liminal space context where it's like, okay, what's happening? This is something I'm out of any normal... interactions and there's these NPCs that are walking around doing weird things and it feels like a little bit like an open-ended open-world sandbox where you can kind of like interact and play but there's there's not a lot of direction as to what you should be doing in any moment it's more of like you have a group of people that you may or may not know in social VR experience and it's In a lot of ways, experiences like this, the experience of them can get dictated by the other people and what they're doing and how that plays out into these emergent social dynamics that are playing off of what's already there with the NPCs. And so it feels like the early beginnings of something where you could start to cultivate certain types of interactions and group behaviors, depending on what that invitation is to do. And so with these encounters, as you describe all the different intentions, because it's an open-ended sandbox and there's no discussion, there's no dialogue, there's no instructions, there's very minimal guidance to that, then sometimes those intentions get lost in terms of the non-specificity of what's going to happen. So I'll be kind of elaborating on that more, but I wanted to first ask, as you were starting to develop this experience, Was there a lot of user testing, or how did you iterate to get to where you're at now to try to slowly build up the scaffolding for each of these context-less NPCs that are doing these weird social dynamics?
[00:18:04.266] Mathieu Pradat: Yeah, there was a lot of testing with users and we actually created this with the public, being there with the public and working with the public. So we have maintained this and made many workshops in Ecuador, in France, many workshops in France, in a place called Le Grenier à Selle in Avignon that we love. in Hamburg, virtually it was COVID, but in Hamburg, in Vram, in Taipei, in Kaohsiung. So it was a very iterative approach. Yes, I really wanted this to be, as it is in a way, an abstract piece. I didn't want to have to tell anybody anything about what's going on and what's to be expected. And yes, to keep this kind of quest, people trying to Yeah, trying to understand. And for this, I think what I... My approach was to limit the numbers of things that are being represented. Everything plays a role, basically. There are no, like, I don't know what the saying is exactly, but, you know, when you show a gun on the screen, then it needs to be a part of the story. Script writers say this. This is really something we use on Rencontre. Like, anything you'll see, plays a role in the story and is an element you can interact with and yes you'll have to be looking for it so some people might be very comfortable with it some people might be not so comfortable but it's i will say we tried with this testing to find the the right combination when it's interactive it's not going to be in my mind a hundred percent who gets it Because if you do it for 100%, then it's going to be too easy and it might be a bit boring. It's got to be like 90% or 85%. I don't know exactly what will be the percentage. But yeah, we try to make it for every people, every age. But we try also to keep some... Some stuff you need to investigate a bit and I think that's a way also to keep people engaged and connected and a driving force for them to engage socially because you can talk in this experience, you can talk to your neighbor and a lot of people that's the first thing they try. Okay, we're together in the simulation and we have this kind of hostile NPCs with us. What do we do? I mean, are we like... having a bit of solidarity here to try to figure out what we should do. And yes, that's the way it goes usually. To be honest, when I started working on this and I was working on other pieces and I could see that sometimes because we did a lot of Q&A, so we did a lot of questionnaires and stuff. And I could see that some people were just looking. They weren't interacting. I was like, wow, okay, this is bad. This is not working. And so we started making interviews. We started to ask those people to fill the questionnaires. And they were actually sometimes, not all the times, but in many cases, the most engaged. those people watching the other people interact were the most engaged emotionally in the story so that was very reassuring and it's true like here also in Venice sometimes I see people they don't do anything they stay in a corner they watch what's going on and they do this for maybe 20 minutes and I'm like always like a bit worried and in most cases they are super engaged so this is also something that I find super I love this about this new work, this new medium, the amount of liberty you have in those works. That's amazing. You can interact, you cannot interact, you can sit, you can walk, you can look, you can look at NPCs or people. There is a very important amount of freedom and I think that's cool, that's good. I feel good with this.
[00:21:53.800] Kent Bye: Yeah, I feel like there's the onboarding, the actual experience. So there's different specific interactions at certain points that you want people to record a message. And there's no instructions, and it kind of happens symbolically. And so the interaction design of that was such that I didn't end up recording something. I wanted to record something because I In the onboarding process, I was actually told by two people at two separate times, OK, you're going to be in this experience. There's going to be this moment where this ball drops down. What you're going to need to do is to grab onto the ball and then put your hand over here. And then you do this. So it felt like some of the interaction design, because you weren't being as explicit in the actual experience, the kind of emotional labor of the instructions came into the onboarding where it was like, I was overwhelmed with all this cognitive load. I have no idea what the context is. I have no idea what this experience is. I'm being told about something that's going to happen at the end of the experience about I'm going to have to do this, this and this. And then you came in and then gave kind of a similar speech around like, OK, here's what you're going to need to know. Here's what we're doing here. And then so there was kind of like by that point, I was like, oh, my God, I think I missed half of what he just said, because I'm just trying to like be in the experience and not feel like I have all these instructions for what I'm supposed to do and then I get into the experience and there's like literally no like interaction design instructions and it's just kind of like an open world sandbox and our group they didn't do any sort of collaborative like trying to figure it out and we were all kind of in our own little worlds parallel playing in a sense without trying to decode the logic of what was happening but I feel like there's a little bit of a trade-off of like if you're not going to be as explicit in the design then then some of that will get offloaded into the onboarding process that then kind of overwhelms me before I even step foot into the experience.
[00:23:35.327] Mathieu Pradat: Yeah, you're absolutely right. About this final part, clearly this is not the final version. We need to do a bit of work on this. I really want to keep this poetic and limit the amount of instructions. So we want to cut all these instructions in onboarding, because that's overwhelming people, as you said. I totally agree with this. And it's simply something that is too much unveiling the experience. This is like a slow build-up. We want to lead there. We don't want to have to say that in the first place. So we're not yet there visually. We basically need to do more visual and sound design in this specific sequence. It was so hard to code, actually. It was such a challenge, a multi-user challenge, to have this kind of answer response and also to give everybody his own pace and to be able to limit the action to just what you see so we had to cut everything that wasn't your concern the user's concern so yes we did solve many issues on the sequence but there are still work to be done on this clearly and it's also a sequence right now that is just too long
[00:24:50.404] Kent Bye: Yeah, I was thinking a lot about this afterwards, because I was told by two people how to record a message. I was in the experience, and I failed to record the message. And then afterwards, you said, well, did you record a message? And I was like, well, no. Could I even record a message? And you're like, yeah, you're supposed to speak right after this point. And I'm like, well, that wasn't clear. And so I was like, well, first of all, there's a bunch of rules that you have, but sometimes you violate those rules. So there's these kind of gestures you're matching, and then there's audio cues that are like, OK, you got it. Beep. And then there's the message that just plays. So there is speaking in the experience, but it's like the message of the previous person who's recorded a message. And so then I started to think, well, what are the existing affordances of when you record a message? It's like, well, when you leave a message on the answering machine, it's like, please leave a message at the beep. And you could please leave a message to the future participants of this recording in three seconds. Three, two, one, beep. And then you record. or something like that, which is something that is all audio, or at least if it's not that instruction, then some sort of recording light that's blinking. And if you're going to use no language, then have someone actually walk through each of the steps that they do and then have them record it and have it play back just so that you can then mirror it. So yeah, I was just kind of figuring out If there's certain constraints that you have put into the experience that you're not going to be explicit in the instructions, but yet you still want people to do things, then how do you close that gap between like guiding and directing them step by step to what kind of interactions you want them to have?
[00:26:22.219] Mathieu Pradat: Yeah, let's see how this clearly needs additional work. Let's see if we can avoid direct prompting invitation. That would be nice. It's quite a challenge. One thing also that is hard for now for the public is that we are actually not replaying any previous recordings. so there are no examples there is not a sequence that you can have like 10 to 15 recordings that are expressing several very different views and suddenly okay this is your turn maybe you can contribute and we don't have those recordings and we don't have the pose also allowing the user to uh okay i've just listened to this how did i feel what is coming to my mind so I think this will come into place. It's clearly this sequence unlike the others. We haven't been iterating in workshops as in the first three chapters simply because it was so challenging. Right now it's working doing this multi-user recording live being transferred to the router. There's gesture dialog. All the tools are here and we have to fine-tune it a bit.
[00:27:38.553] Kent Bye: Yeah, and so as you're building out this world, there seems to be some very specific rules that you've set up, like no speaking and one-half gestures. Can you lay out some of the different rules that you've set out in this world that you've created?
[00:27:51.184] Mathieu Pradat: Maybe I can talk about rules and also just things you pass by that are not really interactive but really like are maybe having an effect. So the first thing is you go through a cascade and you just hear, you just hear the sound on your coat, on a virtual coat that you're not actually wearing and when you're turning around there is a cascade. To progress in the story afterwards, there is a small drop of water. This is the first time you see water and if people touch it with their hands, which they really do in most cases, it will trigger the appearance of a very, very big drop. This big drop is interactive. The more you pull away from it, the faster it goes. Usually people intuitively just gather around and kind of maintain this drop at their level, at their head's level, and they just enter. And when they enter this big drop, they have this sub-aquatic feeling. And the water is going to be the environment of this experience. The water is going to rise. Some elements are going to disappear under the water. And also there are two LBE institutes. versions of this work that we've already done. One is the one we're seeing here in Venice is a dry scenography. The other one is a wet with a pond scenography with 10 meters per five. So people will walk actually barefoot. And there are within the virtual environment of water and also within a physical environment of water, which triggers, I think, the personal memories of childhood and something very personal that engage people in the story. After this big drop, some weird human figures kind of take birth within us and they move away. And that's the only thing we're going to be able to do is kind of chase them, chase them around. And they are going to be very hostile. They're going to look at us, talk in some kind of lingo, make gestures like, don't follow me. And until... we're finally able to stop them. And when we are finally able to kind of stop them and reach out to them, they disappear above us and start laughing at us. And this is going to be the mechanic of the first chapters. They will gather in circle, and each time we will want to get inside and kind of be part of this, they will disappear and be more numerous. So that's for the interactive part, and this is ending in kind of total collapse. Each one of those chapters has an interlude. It's an interlude of water. That's one of the reasons we wanted to go to work in Taiwan. I love Taiwanese cinema. I love rain sequences in Taiwanese cinema. And the first interlude is a tropical rain. Just by moving around, users trigger rain. and trigger more, and usually people, they like this, so they move around and trigger more and more rain until it basically suddenly stops. So this is also an interactive sequence. If you don't do much, then there is not much rain, but still working. So there are all kind of behaviors there. We're moving on to the second chapter, the birds, where there's only one interaction. Grains appear in your hand, and you can feed the birds. And the birds are growing bigger and bigger. And you can walk around in the birds. Well, there are maybe other behaviors that are more subtle, but they're part of the experience. If you move towards the birds, they move away. If you move away from the birds, they move towards you. So they're always kind of pretty close, not too far. This is leading to the second interlude, the tempest. Basically, it's the same rule as the rain, tropical rains. Moving around will kind of make waves, sea waves, grow in a fantastic, very vertical, like stalagmite waves. So the more people move, the more those waves are around us, and you're basically inside the waves. So this works pretty well, I think. The people are really... being imaginative with this sequence. And the third chapter that I call the Colossus, where you become a giant and planes seem like mosquitoes, has a very simple interaction. It's a game of pickleball with planes with your heads and hands. You can box, you can play pickleball with the planes and if you're able to do this then they acquire speed. The more you hit them, the faster they go and the more they multiply and in the end it's impossible to avoid their trajectory and they will hit people's body and explode and this will bring this third world to also a collapse. There is another interlude after this called the fog and the fog rises and what users can do is erase the fog and return to a clear environment and simply by moving around and also using their hands so they have a kind of a fog resembling a big ice block in a way. So those interludes are on the same interactive principle. leading to the fourth chapter of the epilogue, where we will only see bubbles on the surface of the water. And if we step on those bubbles, then those NPC, the intruders, return. And when you move around them, they stand up. And suddenly the rules of the game change. They will not flee. They will not go away. They are simply sticking with you in a kind of a tango. And whatever you do, they will follow you until they're other npcs disappear we were inspired our developer luig vigor was inspired actually we were trying to figure this out how would we focus users intention to one npc one intruder and where he thought about west side story uh when the lovers see each other in in the ballroom and basically on the screen we exclude anybody else they're just he and she and we we did this in this kind of a very direct dialogue so we slowly have other intruders fade and the intruder starts making gestures to you and it makes two gestures with a pause that users can replicate and after that there is a gesture engaging the user to okay it's your turn you can now maybe propose gestures that will try to imitate and actually the this npc this intruder is we're quite proud to achieve this with the code is able to imitate anything you do. So right now, and we're probably going to do this in most cases, it's only the hands because they aren't detected by the headsets. But we've also done trackers and bracelet tests and fingers adds a lot of traffic on the router. So we've limited it. But yes, we can have this very direct engaging dialogue phase that I think in the end will work well, even though it's a bit rough for now. leading to a kind of pose where the intruder just spreads out its arm and makes gestures like okay let's go for an embrace let's go for a hug so I think people feel very differently about this some people don't do it some people do it directly. Some people find it very sensual. It depends on how it's felt. And after this hug, there is a release where we want to add now the recordings of the user, the participatory part of this work. So this is where we'll have Chinese, English, Spanish, teenagers voice, elderly persons, we have all kind of different voices, different also gestures, because the gesture of a youth is not the same as somebody who's older, so I think that can be quite emotional. And that leads us to this very difficult part where we want to, without adding too many instructions, have somebody making a contribution in the work and leave the right time, the right amount of time to think and the right time for the recording. So this is still a work in progress. And that leads to a replay sequence. So then you see, okay, you replay this recording and you can either, okay, discard it and redo our recording or validate this and this will be added to the experience ultimately. So that's for the epilogue, and we have a really final sequence that I've always liked, and I'm pretty happy with the way it comes out. It's a kind of a celebration inspired, I guess, by Italy, in a way, by Saint Antoine de Padoue, I don't know how you say this in English, Saint Anthony, maybe, you know, talking to the birds and having the birds reaching people's hands in a kind of a total harmony between humans and animals and nature. So birds are coming back from under the sea as like original animals like back in a very ancient evolutionary time and they will circle around users until maybe someone either finds the idea or imitates an intruder. and reaches out for one of the birds that would actually land on the hand. And then you can either keep it, pass it on to another participant, pass it on to your other hands, have it fly again, and that's going to be the end of the journey in the encounters.
[00:37:07.112] Kent Bye: That's a really comprehensive recap of as you're going through all that, there's a series of interactions and a pattern language of interactions that you have that are going from one sequence to the next. One question I have is there's my individual interactions that I'm doing, but then sometimes there's collective interactions. When there's the boundaries of the wall of the water, if there's multiple people close to each other, then it'll put down more of the water so that you kind of have this sense of rather than being in a bubble by yourself and under the water you're now with other people so there was this process of like finding the other people but and all the other interactions though it didn't seem like there were very many social dynamics that were happening or like if people were doing something together, is there any instance where if people collaborate and do things together, then you might have different results? Or is it all just like single player kind of interaction or it's very localized to an individual's individual actions?
[00:38:05.615] Mathieu Pradat: Yeah, the social aspect of interactions are in the interludes. So you're really going to drive more rain or more heavy waves if you interact together. So that's clearly together, it makes a difference. And in the first chapter and the big drop of water, clearly like if everybody spreads apart, then it goes super fast and then everybody goes close, then it's almost not moving, stationary. So the interludes are kind of collective interactions. The other chapters are not, except for the first one where you will not change the narrative, but you will really definitely accelerate it if you're joint forces.
[00:38:44.167] Kent Bye: Yeah, I guess that'd be one piece of feedback is that if there were more collective interactions or ways that you could see what's going to happen as a result, because there's some of the sequences where I was like, OK, I feed the birds and then the birds increase in size like four or five or six times. And but the fundamental interaction didn't seem to be developing or evolving. It just kind of like I was like, OK, I've done this, but I don't see how this is developing as a story or anything. It just kind of felt like. lingering on for a little bit too long and so I think that's for the main chapters that was a consistent experience was that that single player nature of those interactions didn't encourage me to collaborate or to work with the other people or for us to figure out how to do something different if we work together and even if you did implement that then how would you communicate that oh if you did this together then you would get a different interaction so some of it was clear with like the the water interactions like the other water interactions I don't recall if there's anything that we were doing as a group that was changing it. But as you think about like telling the story, what's through essentially a series of these different atomized interactions, how do you see that those interactions then build up to the larger story that you're trying to tell?
[00:39:54.036] Mathieu Pradat: as interactions.
[00:39:55.657] Kent Bye: Yeah, it seems like that's kind of the language of the piece, or these micro-interactions with some of the characters that are there, but the characters go from water to some of these random NPC characters, to some chickens that get to giant chickens, into the airplanes, and then it's pretty random in terms of how those would be connected to each other, so it didn't seem like it was building up to anything other than a bunch of kind of random things that we're doing for a while.
[00:40:21.792] Mathieu Pradat: Well, it actually, yes, it does in people, spectators. For the first chapters, I think there's like no rules given ahead of time. So there's a total discovery. And basically what spectators discover is that when they interact with this world, they bring it to saturation. They bring it to a very saturated state. Like we're just filling the environment, the earth, the space with those characters. And this is just inevitable. So this is installed in a way as the narrative rule, so whatever is going to happen afterwards in other worlds will have the same result. It will be maybe not having exactly the same meaning, like when with the birds there is a kind of maybe an idea of going backwards in time, not simply of indifference and maybe it's like we're not taking care of how we feed ourselves. It's not exactly the same meaning, but the mechanics, the way we are interacting and the way we saturate those worlds is very similar. So I think that, yes, I think people understand this very well, that those are chapters and that it's kind of written in the same logic, at least for the third chapter, and I hope for... with this to create a kind of, all right, we know now what's going on. It's going to be different, but we're finding a way to saturate these worlds and to bring them to collapse, leading through the fourth chapter where we are reversing the logic. Maybe also have people waiting for this, you know, try to be confronted to something that has more a brighter side, you know, so okay, we've been like feeling kind of disparate, especially in the first chapter, but okay, there's now we can reverse it and we can bring it down like it was just building up and we can build it down.
[00:42:14.744] Kent Bye: So at the end of each of the chapters there's kind of like the NPC characters are like drowning into the water and then the chickens when they're eating all the stuff they get giant but they're also puking and then they disappear and then there's the planes and then eventually they kind of go into the water so...
[00:42:32.163] Mathieu Pradat: Yeah, we go in the water with the planes.
[00:42:35.508] Kent Bye: And then, you know, at the end, you have a little bit more of, like, you're kind of mimicking and mirroring someone else, and then you have this kind of recording of this message. And then so how do you see that the... all these previous interactions are kind of building up to this. It feels like the climactic point is like you're going to have something to say to the other people who go through this. And so how do you imagine that these kind of series of different interactions are going to kind of build up a context to put people into like a certain state of mind to communicate something that could be very profound from something that's very random and dark and weird and kind of abstract and not clear into what's something to be kind of really deeply meaningful.
[00:43:18.065] Mathieu Pradat: Well, that's an interesting and hard question in a way. In my opinion, we've already been confronted like I expect, or I think the public has already been thinking of being confronted to this idea of we're saturating the planet with a human presence. We're eating millions of industrial chicken. We are saturating the skies with... planes. So this is this paroxysmal situations, worlds I'm bringing with Rencontre are not new to spectators. They have their own opinion on this. And it's a long piece. It's a 50 minute piece. So for those three chapters, it's about like, I don't know exactly, 35 minutes. So they really have the time to process this. And it's also repetitive. everything we do okay more saturation second chapter more saturation in in a different way so i think there is a something that okay we're really into this they have time to process and make their own opinion within the work so yeah maybe you know it's i don't know what's right you know sometimes when we we see a piece i see a piece and i think about it two days afterwards and i I don't have really the same opinion about it. And right here we're like harvesting very fresh kind of direct participation. So it's very spontaneous. There is a part of unknown with this. Let's see, you know, how it's not fully done. Maybe it will not be exactly what I expected. Maybe it's going to be very successful, depending also on people, context, where it's happening. Is it a kind of a not too loud space? Yeah, a part of this is an experiment. But I think there is sufficient time to engage with this and to give a deep contribution because it's not coming out of nowhere. It's coming out of people's life. Same, you know, like when I try to engage people with a sensation such as water, I think it brings back something very deep and ancient within them and they have it. I really trust the public in a way.
[00:45:49.366] Kent Bye: Yeah, when I read in the description or so in the video that there is a version of this where you do put water down on the bottom, I was kind of hoping to experience that part of it, but it wasn't in this installation here. And so are there any safety concerns with people falling or what's the experience? How's it changed when you put this layer of water for people to be on as they watch the experience?
[00:46:09.505] Mathieu Pradat: Yeah, there is no water on the Lazaredo Vacuo, so there is no possibility to do this with water in Venice. That's a bit paradoxical, but that's the way it goes. Water changes many things from the tests we've made so far. First, the pace. We don't even have to. We're telling people in dry or wet scenography, just don't run because we don't want them to fall with the headset. But we don't even have to tell people to walk slowly in the water. They just do it. You know, they are listening to their own steps and they're really like listening to what's around behind and what's real. They really pay attention to, OK, what is real? What is not real? Because the NPC, for instance, they don't produce wet step sounds. So all the sounds in the water are real. So there is a kind of more silence, I would say, from what we've seen and tested so far. And I think they, yeah, it's a unique sensation to step on the water. It's really something of a joy, you know. It's a mystical joy. And it's just very enjoyable.
[00:47:24.909] Kent Bye: Yeah, it was a big part of me that's sad that I didn't get to see it in its intended form, because I feel like it definitely would have changed my experience of the piece in terms of just my embodied interactions and to see what would have emerged. So yeah, but I guess there's no running water here on the island, so you couldn't do it?
[00:47:40.881] Mathieu Pradat: No, there is no running water. And I think in those conditions, when the small island being linked to the Lido, it would have been kind of overwhelming to do this anyway.
[00:47:53.550] Kent Bye: Great. And I guess as we start to wrap up, I'd love to hear what you think the ultimate potential of these types of forms of immersive storytelling might be and what they might be able to enable.
[00:48:04.716] Mathieu Pradat: Well, I really believe there can be a in situ, on location form that will be kind of the most, maybe with the best technology, the best venues with really top notch gears and sound and immersive venues that could be what if we pull apart the cinema from the science of you know the images the language of moving images has been codified in cinema but cinema is an experience and moving images can be found elsewhere on social networks and tv and so on theater on stage I think for immersion, this is like that, but even more. So we're trying to figure out what is immersion, what is the language, what is the place of the public, the collective place of the public in immersion. It's still an ongoing process. But we can see that these simulations can spread in so many different directions. It can be for live shows, theater, dance, games, social, education, health. So I hope, I really hope that there is going to be a way, a high class form that can be also contributing to building up the language in terms of aesthetics and really propose to the public a way to have really high quality shows and also to be in there on site because I think being in the simulation together It makes a huge difference to being in the simulation remotely. I'm advocating for this. I would like, I want this to happen, I think, being together and having a social, really social on-site experience. So I hope it's going to get in that direction. Not all. I mean, I think it can be different. There can be many ways to do it. I believe also in diversity, you know, like not one, just not one way to do it. But that's my predilection for me, the thing where I would like to continue to work.
[00:50:12.146] Kent Bye: One quick follow-up on the scenes with the NPC, because the NPCs are the same avatars as all the other characters in the piece. They're maybe slightly different, but for my rough take, they looked pretty similar in like... There's enough of this social dynamics where you're almost encouraged to walk through them or there's not really much you can do if they kind of surround you. I was kind of walking through the NPCs and at some point I kind of bumped into one of the other people that were in the room. And so there's this kind of like, you have to be aware of the protocols of not just like completely running through everybody, but to...
[00:50:52.282] Mathieu Pradat: No, in a rencontre you cannot walk through an SPC. It will always either step back or follow you. So you cannot walk in a bird, you cannot step accidentally in a plane. It will always be interactive and stay at the distance of the interaction. Except at specific points when like, okay, I have to trigger and to break this conversation that's just within those... Those characters, yeah. We actually were... I'm very interested with this similarity. I love a book called L'Autre Comme Moi, The Other One Like Me, by José Saramago, that was also adapted to film, that, yeah, this person we're not paying attention to is quite like me.
[00:51:35.157] Kent Bye: There were some times where I was walking through avatars when they were in a circle, because they're just standing there.
[00:51:40.301] Mathieu Pradat: That's the one point that you can go through them and it will trigger the next. But it's kind of an event, a specific event that we want people to trespass and to break those connections within them. But otherwise, there's no possibility to go through an avatar.
[00:51:59.065] Kent Bye: Yeah, and that's what I meant when I was in that scene, I was walking through the avatars and I bumped into somebody else and just like, because there were just like so many of them. But anyway, it was kind of like, if I were doing this virtually, then I wouldn't have bumped into somebody else. And so there's something nice to be co-located with other people doing an experience like this.
[00:52:17.549] Mathieu Pradat: Yeah, I think it's nice to be together and like we even added like in the introduction when we were testing we observed a group of students that were just having so much fun just being there as avatars and reconsidering maybe their presence and their relationships, suddenly they were all looking the same. And they're starting to have so much fun that, yes, we let some space at times just people for being together with those apparently like undistinguishable avatars.
[00:52:53.604] Kent Bye: Great. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:52:58.425] Mathieu Pradat: No, it's good. I'm really happy to be here in Venice. I'm really fond of this little community. I hope it will really build up. It's a very interesting research on many levels. I'm just looking forward to being able to kind of broaden the reach of this community. It's super, super interesting and yeah, I hope now those works have their life and I've seen so many very exciting works, very beautiful works here. I just would like those works to reach out to the public and I think the public is ready. whenever we're invited to a museum and we'll be in boston on november at the museum of science whenever we've been in a venue it was full house basically everybody is curious it's working well not only the young people okay i agree the young people they're really embracing this new language of simulation xr but we have lots of people above 70 60 and you can see in the making of they're here and they like it they embrace it so yeah please the venues now we're ready we're ready let's go
[00:54:13.095] Kent Bye: Awesome. Well, thanks so much for joining me today to help break down a little bit of Rencontre. How do you say it again?
[00:54:18.742] Mathieu Pradat: Rencontre. It's horrible. You know, it's got the R. Rencontre.
[00:54:23.287] Kent Bye: Rencontre. I'll say encounters. So, yeah, it's got a lot of these interactions that I think that you've done a lot of user testing. And I feel like as I... look at location-based entertainment, there's this trend of needing to have throughput of people, a number of people per hour. And so like looking at the lens of LBE, there's probably some stuff you could cut out of this and trim it down from 55 minutes.
[00:54:46.050] Mathieu Pradat: We will definitely. Yeah, it's, let's see, at least three, four minutes long, I think for now. Yeah. Yeah.
[00:54:53.338] Kent Bye: Well, in terms of like, yeah, just getting a number of people through hour and, you know, so there's all these calculations, but yeah, just LBE having people, these different types of group situations and, seeing more installations and social dynamics there. But in terms of like the format of location based entertainment and these types of experiences, the types of stuff you can do with creating emergent social dynamics and also what's the line between being explicit versus being abstract and then how much your intended message can get more diluted or lost if it's like too abstract or too poetic and how do you close that gap by either be more explicit or have things kind of have a pattern language that feels intuitive for people to understand and kind of decode the symbols and the logic of the piece and I feel like that's a it was difficult for me to kind of piece together each of the different component parts and how they were speaking or sing a language so I know in architecture there's pattern languages of designing four affordances and different things so I feel like there's a still an emerging pattern language for interactions in this kind of poetic symbolic language to be able to communicate so Yeah, it feels like at the very beginnings of where we're at now and certainly as we go forward in the future, kind of figuring out more and more of this stuff, especially as people like yourself are testing these things and getting more and more feedback on them. So, yeah, but thanks again for joining me to help break it all down.
[00:56:13.026] Mathieu Pradat: Thank you, Kent. I'm really looking forward to all the podcasts you've made here in Venice.
[00:56:17.907] Kent Bye: Thanks again for listening to these episodes from Venice Immersive 2024. And yeah, I am a crowdfunded independent journalist. And so if you enjoy this coverage and find it valuable, then please do consider joining my Patreon at patreon.com slash voices of VR. Thanks for listening.
