All Unsaved Progress Will Be Lost is the type of experience that is better the less you know about it, but I really, really loved this piece as it efficiently packs in a lot of emotional punch in such a short time. The primary mode of storytelling in All Unsaved Progress Will Be Lost is via motion graphic text as you float through nature scenes constrained underneath a bridge. The full context of protagonist’s rebellion & resistance is held until the very end, and so using the modality of text encourages the audience to use their imagination to fill in the gaps for in what’s missing for what’s happening and why. There’s an undertone of psychological terror that director Mélanie Courtinat was going for as she’s inspired by Jordan’s Peele’s work like Get Out as well as Annihilation, Midsommar, and It Follows. She users color shifts and moving from constrained spaces under bridges to much more open spaces to help emphasize the moments of peak emotion, and other contrasts that we dig into more detail in the context of our conversation. There’s also a lot of subtle direction of embodied postures that likely work on an unconscious level, and I really appreciate the depth of design consideration Courtinat has put into her piece. I’m looking forward to seeing where she goes next in her explorations of immersive storytelling.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that's looking at the structures and forms of immersive storytelling and the future of spatial computing. You can support me on Patreon at patreon.com slash voices of VR. So in today's episode, I unpack the experience called All Unsaved Progress Will Be Lost by Melanie Courtenoy. So this is a piece that I really recommend you try to see before listening too much about it. I think the less you know about a piece like this, the better. However, if you did get a chance to see it, or if you want to know a little bit more about using text, as an immersive experience. This is a piece that is using a lot of contrasts in very interesting ways. And so talk a little bit about Melanie's process of creating this piece, because you're basically creating this as a solar developer, doing all the different variety of aspects of this piece. But because you are reading a lot of the different text and you kind of get into this certain Mode of expectations of trying to understand what is happening in this piece So there's a dimension of psychological terror at this this hidden monster That's in this piece and there's a bit of a twist at the end where you learn more information and context about this. So That's what we're covering on today's episode of a severe podcast. So this interview with Melanie happened on Sunday September 4th 2022 so with that let's go ahead and dive right in
[00:01:34.095] Melanie Courtinat: My name is Mélanie Courtina. I'm born and based in Paris. I'm an immersive artist, a creator of immersive experiences. I use video game engines because my background, in terms of interest, are video games and contemporary art. So I'm here in Venice today to show my latest VR piece, All Unsafe Progress Will Be Lost, for its premiere.
[00:02:02.109] Kent Bye: So maybe you could give a bit more context as to your background and your journey into doing immersive art.
[00:02:06.993] Melanie Courtinat: Yes, with pleasure. When I was younger, I was studying fine arts. It was a long time ago. Teachers kind of pushed me to try drawings and sculpting, etc. And this type of traditional ways of creating didn't quite fit me. This wasn't really what I wanted to do. I was playing a lot of games and games were like the creations I was most comfortable with, the one I was most enjoying, because they feel so complete to me. And so I learned about a formation... School. a bachelor called Media Interaction Design. It was in Switzerland and I saw it, I was like, this is what I want to do. I did it for four years and when I finished it, I created for my bachelor project a VR garden using at the time HTC Vive trackers that I mounted on bracelets because I really wanted to have some kind of sensitive, sensorial experience without the users having to touch the cold plastic of the controllers, something really like this. So I finished school with this project, and one festival came to me and said, do you want to exhibit this here? It was 2017. And I was like, yeah, OK, why not? And then another one, and then another one, and another one. So, of course, this was before COVID. I got lucky enough to be able to travel a little bit around the world. For example, I went to the Tokyo Game Show, which was my childhood dream. And at the end, LVMH, the luxury brand, rented it for one of their events and I understood that there was something that could be done between immersive creation and these fashion luxury brands as well. So right now, I have both activities. I create immersive experiences for brands because they gave me a lot of field of action, a lot of means of creations as well. And in my free time, I create little video games or immersive experiences using AR, VR, just for fun and for myself, basically.
[00:04:26.361] Kent Bye: Yeah, maybe you could just give a bit more context as to how this project came about.
[00:04:30.325] Melanie Courtinat: Yeah, of course. So, basically, it's not the story that came first for this project. The story came way after. I knew about the themes I wanted to think about, to reflect on for this project. I knew the ambience I wanted. I like to work on themes like melancholy. My name comes from the word, actually. On loneliness and on violence. And I thought about making a project with a hidden monster, some kind of Lovecraftian horror. And at some point while writing, I realized that the monster, the horror I was wishing to talk about, existed. So without spoiling the experience, I took testimonies from an existing historical event, a catastrophe, and then the story came to me. And then I wrote the final one and created it. That's how it came to life. So basically it tells the story of a narrator, for me it's an old woman but it's never stated who they are, who testify about what they experienced during a time where soldiers came to her little village and said they had to evacuate everyone for a reason she basically does not understand. She does not understand why they say that the water is poison, the air is poison, And she's so sure that it's not because the threat is invisible. And so she decides not to go. And then things happen.
[00:06:09.914] Kent Bye: So as you're designing the actual experience of this piece, what I experienced was that you're kind of under a bridge, lots of plants and flowers, and you're scrolling through the space. But the primary mode of which I'm experiencing the story is through this text that's going by. And so I'm reading a lot. So I'm immersed within an environment, but I'm mostly just digesting the story as it's coming through the series of the text. And that changes and evolves over to a certain point. It opens up and then there's a lot of text that comes through. So I'd love to hear a bit about your process of deciding to have the audience member read the text rather than do a sort of a voiceover for them speaking it.
[00:06:48.838] Melanie Courtinat: There are two sides to this question. At first it was because I did the project by myself without any funding or any help, except for the music that was made by a friend of mine and except for the distribution done fully by Diversion Cinéma. Else, I was all alone creating this and you can only go as far as what one person is able to do. I can program, I can 3D model, I can write, but I didn't want it to make an experience where I try to go too far and fail at it. It's a single player experience without too much interaction because I knew this was what I was able to do at the time with my given time. And so I did not want it to have a character, a human inside the project, a 3D model of a human, because I knew I wasn't going to be able to make it as good as I wanted. So I was like, OK, so what can I do? I could add voices, but voices would give too much of a clue on who this is. You can sort of hear if it's a male, a woman speaking, more or less, old or young. So I thought about text as an answer. to this problem of not doing something I wasn't able to do good and not giving too much clue on who's the narrator.
[00:08:14.307] Kent Bye: So it's a bit of a, it's a bit of a mystery that you have building up here. I'll talk a little bit about my experience of this piece as I was going through. So I felt like there's a way that you're able to create this mystery that is unfolding. And so as it's unfolding, it's getting a little clues as I go along, but the spatial architecture, I think is interesting to talk about how you're creating this somewhat enclosed space and then you open it up. And so maybe you could talk about the environmental design that you have throughout the course of this piece.
[00:08:43.014] Melanie Courtinat: Yes, with pleasure. In the beginning of the experience, you're under a bridge. You can see open spaces that kind of lead to faraway places. There is a horizon you can see, but cannot go. And you feel a little bit constrained. And the more the story unfolds, the more open it becomes and also there is a little play that the user can't quite comprehend perhaps but at the end of the experience there's a moment in the story I can't spoil where the tone of the project becomes even more serious and you read testimonies that mix together because there are so many and at this moment you're still following the text with your head so your head goes down. So physically the user is tilting his head down like you do in sign of respect in real life when you do a minute of silence or when you go to church you bow your head down. It's not first degree like you're now tilting your head down because you're respecting the people who died but it's more something unconscious that interests me as well. I think I would like to go through Not necessarily the body in VR. This is something that has been done a lot. But I think finding ways to constrain the user body, to make the user do movements to force him in different positions. This is something I want to go further. Like crouching or kneeling or something like this. I think it's quite interesting.
[00:10:21.098] Kent Bye: I didn't actually notice that while I was watching it, but now that you say it, it's kind of interesting that you're looking down in VR, but you're kind of recreating an embodied experience of a bowing motion. So that's interesting that you're kind of using VR as a medium to invoke within the user a certain position that may have a larger meaning. So I think that might be the first time I've seen anyone do that. It's really interesting. wondering if you could elaborate on your process of creating contrasts between constrained and open and then you sort of have single lines of text and then there's a lot of text and then there's multiple colors and there's one color so just love to hear about different ways that you think about building and releasing tension through the spatial medium
[00:11:03.650] Melanie Courtinat: So with this project, it's my first time that I'm telling a story with text. Usually when I create small experiences, there's little to no text, no dialogues. There's a story, but you have to literally find visual clues and then invent it yourself. This is more literal. You have the text, you have words of a meaning in which you can understand a story that unfolds. There are several parts to the story as well. First part, the... narrator knows she has to evacuate then decides to stay and in the end something terrible happens that I've made quite visceral I believe by using sound and using lights and using everything that audiovisual medium of VR allows me to do, without making it too epileptic perhaps. And there is a, in French we say embrasement, fire starting. At this point everything becomes bright, everything changes color, and you start floating. And the more you float, the more Testify, you can read and you're being surrounded by those Testify and all the audio space is occupied as well with this kind of beep beep beep that I made all around. and there's a saturation feeling that's kind of too much for everyone. And as it becomes quite long, I believe that the user will tend to just breathe, look all around. He's floating now on top of everything. And at this exact moment where everything seems more open and doomed at the same time, he reads what it's based on and at this moment realizes that it wasn't just a science fiction story with a big bad monster or an alien or something like this, but this actually happened to real life people. and I believe it makes the moment quite strong. At least that was what I intended to do. While I was showing it these past days, I've seen people take off the headset and cry, and VR isn't quite optimized for crying. But I believe their tears and their emotion, it's not a reward, but it works, at least.
[00:13:31.847] Kent Bye: Yeah, I mean it does have quite a punch to it. I felt the emotion of it and because you're you're starting with kind of in this mental mindset of reading the text and digest and understand what's happening but you're in a pretty mundane environment and then when it shifts then it has all these you know like you said the sound and changing the color and quite an intense turn in the story that's kind of surprising and shocking and then Yeah, you're kind of in like, whoa, what's happening? Why? Then the environment starts to change. And I think that was going from a mundane environment into a real surrealistic, morphing, changing, like, I don't know. How would you describe that shift from the normal environment into this extraordinary or surreal environment?
[00:14:15.674] Melanie Courtinat: I think the normal environment you're referring to, for me, looks like a place I would like to be in. Designing VR and games allows me to create places I would like to be in, imaginary places, and so I designed the normal place, the beginning place, as some kind of personal safe heaven. I like virtual flowers and sunsets and fog and very heavy lights and concrete and all of this. Design the space I would myself like to be in. There are places in virtual environments or sometimes in movie sets, like when I see Welcome to Gataca. I think in English it's just Gattaca. Or some movies that represent spaces where that doesn't exist. I so strongly want to be there. I want to be in this place and look at this architecture and feeling surrounded like this. So I designed this kind of place where I would feel safe and where I feel good. and then I destroy it. When the reality of the narrator becomes quite tangible, quite heavy, I destroy her safe place I created.
[00:15:30.254] Kent Bye: One thing that's interesting in VR is that this experience feels like a dream. You're using symbols and dream logic and it turns from normal to surreal and so sometimes there's imagery within the piece that has Universal archetypes of a meaning of what they mean or their personal or they just don't mean anything specific They're just there within the context of the environment And so when I watch a piece like this then it's like trying to decode all the symbols and all the different things in there So is there any? Symbology or meaning that you have within the type of animals that you're putting within this landscape
[00:16:05.595] Melanie Courtinat: Yes, the inspiration for the animal was a picture I saw of a quite famous park based not so far from the catastrophe. There is a big tag of a family of deers and this picture quite struck me. Everything was left abandoned but this drawing stayed, this kind of proud family of deers. And when I read a little bit more testimonies of people who experienced this kind of catastrophe but stayed in the place, you can see that forest animals, such as deers, remain and they even prosper because they're not being hunted. So they stayed, everybody left except for them. And so I wrote the narrator in terms of, I believe they would felt some kind of connection to the only living creatures that remain. Friendship, perhaps family even. And of course, the doe is also a symbol of innocence, if we can be really true with this. So I added some of them in the project. to give a tone as well. You can see quite a frightened doe in the beginning. You can see a deer with mutant antennas in the middle. You can see an escaping doe at the end, etc. And then some kind of symbolic scene in the middle with dead deers and the bridge coming up and then you're going to a second part. So yeah, there is symbolism inside of this.
[00:17:38.775] Kent Bye: In the environment as it's morphing and shifting, it almost looked like radio or speakers or it was sort of like buildings and I didn't quite understand what was happening. The space is morphing into a surrealistic realm. How do you describe or design intentions shifting the space from the normal to the surreal? How are you thinking about that surreal space?
[00:17:57.810] Melanie Courtinat: Within the first part, the buildings are very square. I use Breton architecture, both because I really enjoy the way they make me feel. I feel safe in this kind of environment with very strict angles and everything being super straight. So we start with something quite strict and then it's just the light that I added that makes them look a different way in the end. It's only light, I didn't use any shaders or effects on the camera, I just add a big blast of light and this changes the environment.
[00:18:35.262] Kent Bye: As you are looking down at the end with the text, things are coming in so quickly that it's hard to digest everything in real time. I don't think I've ever experienced this kind of like overwhelm of not being able to read everything, but then over time I'm able to look down and read messages that are coming through. It's quite a contrast between reading one at a time and then looking down and seeing a flood of all these different messages that are coming through. So, you know, reading text in VR sometimes with being able to have the resolution to see it and sometimes I noticed you use kind of like a pixelated font so that you're not getting too distracted by the anti-aliasing that you might see in the normal text. So it fits within the screen door effect of the VR. So as it goes at a distance, it's still easy to kind of read what's being said. the legibility of the text still works with the VR headsets, but because of the screen door effect, I haven't seen anybody ever even attempt to try to put that much text on the screen at the same time, and so I'd love to hear a bit about your process of deciding to go that route of having lots of text there at the ending.
[00:19:37.652] Melanie Courtinat: You used the perfect word, I believe, to describe the text in the end. I believe that by asking friends and family to watch my project and just tell me if I was going too fast with the text or too slow, I got some kind of balance that works well in which you can grasp everything without being too bored with nothing happening. But in the end, there is literally a flood of text. As you said, there are too much of them. and you cannot possibly read everything even if you try. But I like the fact that you're tempted to try in the beginning, but you just can't. There is just too much because there were too many victims. And I wanted this to give the spectator a feeling of everything is too much at this time. The light is too much, the sounds are too much, there are too many texts. You will only grasp a few words, a few testimonies about personal experiences but the mass of all these personal experiences makes you understand that this is something with quite a large scale and then at this exact moment you read what it's based on and you get it.
[00:20:43.458] Kent Bye: Yeah, as I as I watch this piece, it gives me the feeling of like hearing a really powerful poem, you know, like how a poem just like hits you with a spatial metaphor or just lands in a particular way. So I'm curious if you have been inspired or influenced by poetry, or how do you think of the genre of immersive art that you're creating?
[00:21:02.986] Melanie Courtinat: I never read poetry. Besides my classes, at the beginning of my studies, I took a philosophy and literature class, but I ended up hating it and going toward arts, because I was needing to do more than just learn. I wanted to create as well. And since then, yeah, I don't think I wrote or read poetry. So no, it wasn't my influence at all. So I wouldn't say this text is like poetry, I would say there is a poetic side to it, I think. Because I was based on pre-existing testifying text, of course. But then adding my own layer to the story, my process was just write everything down. You don't care if it's good or bad, no one is going to judge you. No one is looking over your shoulder being like, that's a poor choice of words. You're not trying to impress anyone. except for yourself and at the end you remain with pages and pages of words of intentions and you just cut a lot of stuff out and what remains sort of tells you something but as you're in an immersive space there's one more thing you have to do you cannot just as if you were writing a book You cannot just put the words in the right order and then you turn pages, etc. You have to put them in a space so that it makes sense. As I was saying, I put them in the space so that they would bow their head with respect at the end. Sometimes too much text makes you understand that the narrator feels like... in front of a wall, like when an animal is hurt and is stuck into a corner. I wanted to describe this and make the user feel by the play on the text that appears all in the same time, everywhere. And then the text sort of guides the eyes of the user as well, so then he can look where I want them to look at the right time with the actions unfolding, the animals appearing, etc. The text was also a way for me to guide the eyes inside the space of the user.
[00:23:07.099] Kent Bye: And when you were designing this project, did you write the whole thing and then build the environments? Or did you build the environments and then write it? Or what was the process of going back and forth between the text of the piece and the world that you're building?
[00:23:23.348] Melanie Courtinat: Everything at the same time. Yeah, literally just taking breaks from creating a second camera layer with an extra strong anti-aliasing so that the text can be super readable and make the text not interactive with fog. So you could really twist ease because you don't want to make your user having to twist their eyes to read something. You have to make it as comfortable as possible. So yeah, I would design the space at the same time and do technical stuff to take a break from writing because it does not ask you the same energy at all doing technical settings and telling a story about Resisting and and being alone and melancholy and grief and doesn't quite ask you the same So were you actually being in VR and seeing the world and then coming out? Oh yeah, yeah, yeah. As I don't come from a cinema background, I don't think in terms of frames. I'm not sure I know really well how to frame. I'm not sure I know that, oh, top right corner, you can have something big, architectural, and bottom right, you have your character. That's never how I think. I always create in terms of environments and words because my work has always been games or immersive experiences. So I always start by creating the words like you would do in real life a model You're gonna shape the terrain and then you're gonna paint the colors of the terrain and then you're gonna add your little flowers and the hills and The trees and then the architecture etc and you're able to see this being up literally like a model and then You give yourself a free camera, you make it VR interactive, and you're able to fly in this world and visit it. It's one of my favorite moments. And only at this moment, when I'm okay with the light, I'm okay with the world building, actually, the fog, etc. Only at this moment I will travel inside it and find my frame, find where I'm gonna put the camera. So yeah, I always design the environment, then put the camera, and never the other way around, and never things around the camera. So there are hidden places inside the project that I'm the only one I saw, the point of view that are really, really beautiful. Once, for a magazine called Immersion in France, a magazine about video games, I made an interesting experience. of coming back to my files of a game and using a free camera to take pictures inside the experience of out-of-bound views of things that players could never see. And I had interesting images of it. That's something I think I will do with this again. What cannot be seen.
[00:26:12.715] Kent Bye: Wow, interesting. Yeah, I keep coming back to such a kind of a poetic experience and really powerful and just appreciate having you elaborate on your process of creating it because it works really well as a piece and yeah, the whole environmental design and the sound and yeah, just the whole story and everything. You had mentioned earlier that you were starting with themes of melancholy, grief, and also kind of a hidden Lovecraftian monster. Maybe you could talk a bit about if there are any specific references that you looked to in terms of like if it was a horror genre or Lovecraft or other things that you feel like are influencing your art style and aesthetic and this piece in particular.
[00:26:52.599] Melanie Courtinat: I'm not sure you will see the exact reference, but I can tell you that I find this kind of new psychological horror cinema scene really interesting, that you can see with Jordan Peele and in movies such as, of course, Get Out, but others like Midsommar, Hereditary, Annihilation, Men, all these kind of movies that don't quite use jumpscares as a way to frighten the user really easily and I mean for myself creating using VR it would be so easy to add jumpscares it's actually a very popular themes on steam and on stores for VR because you're so immersed in this environment it can only work super well, to add a quick sound or a quick monster that appears that will sort of make you jump and make you feel afraid. But I believe there are other ways to make you feel uneasy or to tell a horrific story without having to rely on these mechanisms. I have nothing against it, but I think I'm more inspired by If I would have to say games, horror games, it would be more of a Silent Hill vibe than a Resident Evil, for example.
[00:28:09.083] Kent Bye: Have you seen It Follows?
[00:28:11.404] Melanie Courtinat: Oh, yeah. Oh, yeah, yeah, of course. I love it. Everything about it. Oh, there was a story about... I think it was a series. Not so good. I didn't like so much, but something where the fog has monsters in it. And I think it's an inspiration for the piece as well. In terms of Lovecraftian horror, there's also The Lighthouse that was really, really, really good, I believe. So yeah, a lot of movies. And if I want to go up in horror, everything that Junji Ito writes or draws is like the most horrific thing I've ever read. It's maybe a bit too much for me as well. So I wouldn't go that far.
[00:28:56.405] Kent Bye: Yeah, it's interesting Get Out and Us and Nope, all these Jordan Peele films, and this move towards using the genre of horror to address some of these issues that are horrific in a lot of ways, but to do it in a way that is using the affordances of that genre to delve into issues in a way that creates this subtle psychological terror. So yeah, I think it's a real effective translation into VR.
[00:29:20.143] Melanie Courtinat: Yeah, and also movies like Annihilation or Midsommar has really gorgeous imagery, surrealistic and things never seen before. It's not just your horror movie where there's a setting and then some frames of it. There's really imaginative ideas, new kind of monster or horrific silhouettes that appear that I find also in terms of, purely in terms of aesthetics and visuals, super interesting.
[00:29:49.793] Kent Bye: Awesome. And finally, what do you think the ultimate potential of virtual reality and immersive storytelling and immersive art might be and what it might be able to enable?
[00:30:01.220] Melanie Courtinat: For me, I believe that VR will merge with AR to create MR, XR, whatever you want to call it. I truly believe that people are still very attached to reality and things are going toward mixing the two technologies, so to have really augmented reality. I think that's what's next. It's exciting, but I'm also happy with today's technology. I'm really glad that VR is developing and more and more things are possible, but I truly believe that with today's technology, if you understand the way it works and you don't try to make it too much and then blame the technology for it, you really can create powerful, immersive experiences.
[00:30:55.488] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:31:00.909] Melanie Courtinat: No, I don't think so. I just believe that for my project and what I want to do, I'm really happy to be selected in these festivals. I'm really, really beyond happy that it's going to go to other places, other countries after this one as well. But I believe that in the end, I would like to put this project online so that people that come from the gaming public can watch it as well, because I truly believe in games being super inspiring and being the future of the immersive creations I want to do. I do love this public and that's them I want to touch, so in the end I will put it on Steam for them. Hope they like it!
[00:31:40.003] Kent Bye: Yeah, well, congratulations on completing the piece and having it here. There's so many different types of pieces that are here and they're really focusing on different things. And for me, as I go through these different pieces and I come out of it, I really pay attention to the way it makes me feel and the emotional impact. And I think this is a piece that really hits home. So thanks a lot for sitting down and helping unpack it a little bit here on the podcast. So thank you.
[00:32:02.131] Melanie Courtinat: My pleasure. I'm really glad it moved you. And I'm super happy and excited and proud to be here. So thank you.
[00:32:09.846] Kent Bye: So that was Melanie Cortina, and the piece that she had there at Venice Immersive 2022 is called All Unsaved Progress Will Be Lost. So, if you want more context for the wrap-ups, then I'd recommend checking out the episode 1121, where I talk about all the 30 pieces in competition. And in episode 1144, there's an immersive panel that I did at Venice with some other immersive critics, talking about the art of reviewing immersive art and immersive entertainment. I recommend checking that out in order to dig into a little bit of my own process of what I'm trying to do with these larger series and trying to unpack and discuss the art and science of immersive storytelling with a lot of these different pieces that we're showing at Venice Immersive 2022. So that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voices of VR. Thanks for listening.
