Spaced Out is an aquatic virtual reality piece that premiered at the swimming pool at Sundance’s Festival Headquarters at the Sheraton Park City Hotel. It uses Ballast VR’s DIVR system that includes a waterproof VR headset, a snorkel,as well as a flotation belt, tether, and anchor that enables the feeling that you’re endlessly swimming in any direction.
Spaced Out is created by lead artist Pierre “Pyaré” Friquet, and it was inspired by Georges Méliès’ A Trip to the Moon. I had a chance to unpack his experiential design process along with XR artist Sutu, and sound designer & composer Mourad Bennacer (who created a sound design that could be propagated through the medium of water.) I also speak with Ballast VR co-founders Stephen Greenwood (CEO) and Ando Shah (CTO) about their journey in pioneering the aquatic VR space.
Overall, seeing a virtual reality piece in water is one of most visceral and immersive experiences I’ve had in VR, especially because there are things that happened to my body while in VR that I did not expect. Greenwood talks about how water inhibits our normal motion sickness triggers, which makes it so that some VR content is easier for people to experience. They also talk about the mammalian dive effect of being in water, which causes the heartrate to slow down, blood moving away from the periphery, and mind chatter going down. All of these are unique enough effects to merit further investigation in collaboration with Jeremy Bailenson’s Virtual Human Interaction Lab from Stanford, who will be studying the perception effects of underwater VR. There could a number of different therapeutic applications of aquatic VR, but there’s also a wide range of entertainment experiences that are now possible that Ballast VR has been exploring with a range of different water parks and other swimming pools around the world.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
https://twitter.com/BallastVR/status/1221222847151923200
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Update March 9, 2020, 7:54pm: A previous version stated that Jeremy Bailenson secured some funding from the National Science Foundation to study underwater perception, but Bailenson clarified to me that the NSF grant is “about studying the response to environmental VR that is about ocean content but not about underwater in particular. We are studying the perception effects of underwater VR, but the grant is not about it.”
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So continuing on in my series of looking at some of the experiences from Syndance 2020, looking at both the immersive storytelling innovations as well as some of the broader experiential design process and technical innovations. So today's episode is going to be looking at spaced out, which was actually happening at a swimming pool at Sundance, where you would actually go into the water and put on this VR headset from ballast VR and have this whole trippy, it's called spaced out. So you're actually going to the moon inspired by George Manley's trip to the moon. So this was an experience where you kind of imagine like just for a moment, imagine what it would be like to be in water and to have a VR experience. And then it's like slightly different than what you would expect because there's something weird it's happening to your body. So we talk a bit about my own experience of this and what they're finding. Actually, they got a big national science foundations grant to be able to investigate all the specific things that are happening to your body when you're in water and watching virtual reality. Okay. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Stephen Pierre-Ando Sutu-Emraud happened on Saturday, January 25th, 2020 at the Sundance Film Festival in Park City, Utah. So with that, let's go ahead and dive right in.
[00:01:32.317] Stephen Greenwood: My name is Steven Greenwood. I'm the CEO and co-founder of Ballast Technologies, and Ballast Technologies is the first company to specialize in aquatic virtual reality. We put virtual reality on water slides, in swimming pools, and we are currently in 10 countries around the world at different water parks and resorts all over the place.
[00:01:52.560] Pierre Pyaré Friquet: My artist name is Piare and my real name is Pierre Friquet. I'm an XR creator, a digital artist. I've been fascinated with VR since the age of seven when I tried a VR headset in 93 for the first time at Sega Center in London. Made in a space house, it's an aquatic VR experience selected at Sundance.
[00:02:13.248] Ando Shah: Hi, my name is Amdo Shah. I'm the Chief Technology Officer and Co-Founder of Ballast Technologies. I'm a creative technologist and I want to make the world a better place through wise use of technology.
[00:02:26.237] Sutu: Hey, I'm Sutu. I'm an artist. I also run iJack, an AR company, and my own art practice, Sutu Eats Flies. And I'm the VR artist on Space Dab.
[00:02:39.985] Mourad Bennacer: Hi, so my name is Mourad Benacer. I worked on this project as a sound designer and sound composer. I mostly specialize on immersive media in most of my creations.
[00:02:51.727] Kent Bye: So yeah, maybe you could give me a bit more context as to your background and your journey into VR.
[00:02:56.232] Stephen Greenwood: Yeah. I started out in media. So I studied film originally at UCLA and discovered that I didn't want to work in Hollywood. This sent me on a track of doing documentary films. And I did projects in Africa, Asia, and eventually wound up working for AOL as a travel blogger. This kind of put me down a path of being in the online media world. So I worked at Vox Media for several years and helped build the video studio that serves that company. Then I was at Reddit and eventually Discovery Channel where I was doing creative development of virtual reality content. And that's where I had the idea to combine water and virtual reality. So it was originally inspired for me when I was watching Netflix Stranger Things and there's an episode where Eleven gets put into a sensory deprivation tank and has this space helmet on in the water and for me it triggered this idea of I want to feel like an astronaut in space floating in water. And that same night I had dinner with a very good friend who is the CEO of a drone racing company and he was the one that pushed me to actually build a business out of it and start figuring out how we could actually turn this into a business. And that's when I was lucky enough to find my co-founders in San Francisco. And a few months later, we had struck our first partnership with a German water slide manufacturer that wanted VR on their water slides. And that's what's helped build this company into what it is today.
[00:04:23.497] Kent Bye: Oh, wow. I remember in the early days of VR, maybe two or three years ago, before this had actually been a product, it might have been up to four years ago, that people were like, oh, yeah, we should have VR and water in the swimming pool. And it was kind of like a joke. People were just like, oh, yeah, that's such a ridiculous idea. That would never work. And then, I don't know, at some point, that was sort of making the rounds. And they were like, no, no, actually, I'm building that. And then I was like, yeah, actually, I think that's going to be a really good idea. After having an experience of it, I feel like, That same type of sensory deprivation or the haptics you get from being able to move your body around I feel like it's like gives you this extra level of immersion now It's still like a three degree of freedom experience So you don't like have like your body being tracked or anything like that, but I still feel like it was a super immersive experience So for you, when was the first time that you were able to actually? Get to the point where it was just working
[00:05:14.945] Stephen Greenwood: Yeah, so almost immediately after the initial idea was conceived, we built a prototype and right away we saw that there was a lot of merit in the fact that when you're below the surface of water, your inner ear essentially switches off. So there's no motion sickness when you're doing VR in water. And just this initial evidence of this merit to this idea was enough to push it further and say, okay, there's something here that we need to explore. And I think, you know, it wasn't until almost a year and a half later, we'd gone off to do the water slide, because that's how we were funding the business, and we came back and regrouped as a team to build the more polished version of this headset that is used today. And it wasn't until the first time that I got to do our own content designed for this experience in the pool with our own headset that I had this really powerful moment of, okay, we're really onto something here. And the only way I can describe it is the sense of freedom that I had in it. You know, I'd done a lot of VR and spent a lot of time thinking about content development for VR. And being able to freely swim in a pool and actually feel like you're moving through this floating world was just kind of a eureka moment of, OK, we've really got something here. And it's panned out as a business as well. Like all of the installations that we've done at these water parks, it captures people's imaginations. A lot of them have never used a snorkel, so they see it as a lesson in getting to use a snorkel. And they come out completely transformed because they've done these experiences that they never thought they would be able to do, like swimming with whales or floating through space or skydiving with a wingsuit. You know, it's these types of experiences combined with that physical element of immersion that really makes this powerful.
[00:07:04.154] Kent Bye: So technically, you have water that goes in between your eyes and the screen. Maybe you could talk a bit about the reason why you did it that way. And like, what does that give you?
[00:07:15.585] Stephen Greenwood: Yeah. So in all honesty, we initially tried to seal off that section of the headset. And we realized two things. A, it was going to be kind of hard to actually keep that entire part free of water. But B, it would actually introduce a pocket of air that would pull the head up as you go under the surface. And so we chose to just let it flood because it honestly was easier at the time. And then we realized, wow, actually this has a lot of benefits to it because you don't have that air pocket that's pulling your head up. And if we had that air pocket in there, we would have to weight the headset down. And we know that there are other companies working on stuff like this that have done that route where they've weighted the headset down and it puts it too far below the surface to use a snorkel. So it makes the whole operation a lot more complicated. They have to use compressed air. It just makes the whole thing much more complicated than just using snorkels at the surface of water.
[00:08:09.651] Kent Bye: So maybe you could give me a bit more context as to this project and how it all came about.
[00:08:15.173] Pierre Pyaré Friquet: All right. It came about about the discovery of my longtime collaborator, Ando, who has created this startup, Ballast. And I've always been fascinated about an experience of levitation and transcendence, you know, defying the law of nature through technology. And at the same time I started to become involved with an NGO specialized in victims of sexual violence. It's called Parly in France. And then a lot of stories that I heard about survivors were about dissociation, which is a symptom known for people suffering from PTSD. So it can be other traumas, you know. And then the idea of Space Out came, where there's a feeling and emotion of being disconnected to your body, where a lot of victims feel depersonalized, derealized, or there's a sense of derealization of reality. And I felt that the moon was a kind of symbol, the relation of the moon and the earth, the relation of the mind and the body, the relation of water and space. I felt that there was a kind of material to explore. So basically the piece is about recreating the body.
[00:09:22.704] Kent Bye: Maybe you can also give a little bit more context as to how you came to Ballast and to work on this project.
[00:09:29.008] Ando Shah: In 2013, I was a washed-up engineer in Silicon Valley and sick of making more money for companies and not knowing why I was doing what I was doing. So I left and I worked in East Africa for a while doing solar projects and working in conservation. And that was very interesting for me because I got to spend an immense amount of time in the ocean and really wild parts of the ocean in Mozambique. studying whale sharks, manta rays, humpback whales and turtles. And I was building underwater camera equipment while I was doing this. I grew up with a crippling fear of water, the ocean. My grandfather died in a swimming accident. My family's reaction to it was, don't go near water, you'll be fine. I actually had to learn how to scuba dive to get over my fear of water in order to be able to swim. So going into the wilds of Mozambique and diving on a daily basis brought all those fears out and I was able to resolve them at a deep level. And something very interesting happened there, which was there's humpback whale season in southern Mozambique, where pods of humpback whales come by. And it's usually where the mother teaches the calf, who's usually a few months old, how to live, how to have fun, how to communicate, how to breathe properly, how to dive, how to hunt. And during this time, you're basically, if you go in the water, you're completely enveloped with whales and whale sounds. and during one of these dives I go down and immediately I've lost my group and that's the first rules of diving is don't lose your buddy and we're in like a thick milky soup because there's so much plankton around there's no visibility there's swell there's current I'm freaking out all my old fears come back and all these whales are singing around me And of course you can't see them and I can just hear them. There's no localization because your ears don't really work very well underwater. And I start freaking out. And as I'm freaking out, these whales are just getting closer and closer. And at some point, the whale song sort of wails up from within me because they're so close that my ribcage starts vibrating at whale song. And at this moment, I just let go and it was so transcendental for me. I had this moment of connection with these animals and nature. And then I come up and I'm thinking, you know, I'm so incredibly lucky to have this experience and be able to connect with nature in such a way. And I thought, what if I could bring this to other people and allow them to make these connections, which is the seed of the idea for using water in VR, which Steven and I had in parallel in different parts of the world. And many years later, we ended up working on a project for Reddit, and the producer on that project was like, wait, I've heard you guys both talk about this, you should do this. And so, funnily enough, we put our heads together and serendipity happened. And in the mix of all of this, in the middle of this, Pierre and I, we worked together making VR films. We did Jet Lag and Vibrations. Vibrations, we were in Nepal during the earthquakes in 2015. And it really gave us this sort of insight into human nature and how to bring these stories to people and allow them to connect. Somehow all of this wove itself back much later in life. And one of the things that we also realized, one of the interesting things with water in VR is motion sickness. And the other one is the mammalian dive effect, which happens to any human being. If you immerse yourself in a body of water, whether you're snorkeling or just swimming, a few things happen. The first thing that happens is your heart rate slows down. The blood in your body goes away from the periphery and goes towards its core. And then the chatter in your brain, in your mind, calms down. So it naturally becomes a more meditative, more mindful experience. So we piggyback on all of these effects to craft the narrative, the story, and the technology to make it all sort of work in symbiosis with all of this. Allow you to let go and be in your body. And so I think all of these came together very well with Pierre's vision and his idea of having this somatic experience where you are in your body and not out of body.
[00:13:38.882] Kent Bye: And maybe you could help set a bit more context as to this specific experience that's showing here at Sundance and at the point that you started to bring in other artists and collaborators on this piece.
[00:13:49.042] Pierre Pyaré Friquet: So it came about two years ago where I was selected at World VR Forum in Switzerland with another project that we co-made with Andre and I called Patents. And I had a conversation with Shari, Shari Frillo, a programmer from Sundance, where she asked me, OK, what's next? And then I said, I don't know, I was telling her stories, I mean, I was showing the fact that my collaborator has created a startup. And then she said, OK, go, go for it. And I applied six months later. And then it took me a month to kind of organically come up with something where I felt comfortable to explore. And then about six months, nine months later, I did a residency in the same festival in Switzerland, where we brought SUTU. And in Switzerland, in the mountains, we explored first a different path, which was more close to the idea of rediscovering the first sci-fi film called A Trip to the Moon by Georges Méliès, 1902. where, for me, he's really the father of cinema, inventor of VFX, before he was a magician, so I felt that VR needs this kind of exploration of the medium, using another one, basically, to influence it. And then it became a process where Sweetie and I, we explored different aspects, we were in a swimming pool for many days, and a lot of experience of getting feedback from users and we realized that the pre and post experience is very important and we felt that a lot of questions and a lot of concerns and doubts came about the water itself. How I'm going to be comfortable wearing swimsuits in front of strangers, for example. How I'm going to be breathing in the water. For people who don't have experience of snorkeling, sometimes it can be challenging. People think that you're so caught up with VR in terms of your attention is so absorbed that you might stop breathing. So there's a lot of concerns about body relation. And then Shari was actually the mentor of this residency. So there was a year gap after this first initial talk. I mean, then I felt that, you know, the key is the body relation with water and the sense of the primal experience that we have, you know, as a life form actually. Somewhere connected through our DNA, you know, that we came from ocean, from water, and even for nine months as a human being right now, this form of life, we have been for nine months, you know, living in water and in zero gravity almost, you know, in different gravities as Earth. So the process of the residency helped a lot and then I went on and then in fact at some point there was a connection with collaborating with really great legendary French filmmaker called Marc Caro, C-A-R-O, who co-directed Delicatessen, The City of Lost Children with Jean-Pierre Jeunet, who later on went to make Alien 4, Amélie. So Marc and I, we designed this exhibition, immersive exhibition, where I did one piece, I did another piece, you know. And then, this project is on hold, I mean, it's another story, let's say. And then, six months ago, I felt like a fever to make this project. I felt like sick that this project is not making it, and I wanted to get out, to make this project out of my system, in a way. And then I felt, OK, let me just stop seeking funds and let's just focus on actually making it. And I just locked myself in a room and made this project. And there was many stages where I felt I could make an art contemporary piece. So I did a video mapping experience, all of these actually assets. And then came the responsibility of the agreement of the World VR Forum residency, where I had to show something after that. So I did that in September, so technically it's like a premiere. And Shari came also, she was a member of the jury, so there was a lot of, you know, she has been the gold model of the project. And then, it was a different installation, which was in complete darkness, so it felt surreal, you know, and people felt more intimate. It was more in a sense of safety. Yeah, then I got a selection for Sundance, then worked again until very recently, let's say, on this piece.
[00:17:57.095] Kent Bye: Cool. And so, Sutu, maybe you could give a little more context as to your background and your journey into going into the Swiss Alps and starting to prototype what it means to create virtual experiences that are meant to be experienced while you're underwater.
[00:18:10.476] Sutu: So Pierre and I had collaborated on some projects in the past, and then Pierre sort of mentioned the project to me. The original idea being this adaptation of George Melies' Trip to the Moon, which I absolutely love. I'm a big fan of sci-fi, but also I love this kind of era of filmmaking. It's very theatrical. They made these gigantic props, and the actors are sort of maneuvering through the props whilst doing their performances. Which is perfect, like theatre is such a perfect translation for virtual reality because it's a spatial storytelling medium. We have this ability to move through these props. So that interested me to begin with and also the aesthetic of that era of those films is so rich. These giant mushrooms and The colour palette that was painted afterwards is also interesting. They painted onto the film afterwards to make the colour palette. So there was all these kind of aspects. And then generally, I mean, my whole career has been sort of exploring new ways to tell stories using technology. So as soon as he mentioned it's going to be underwater as well, I was like, hand up. Okay, sign me up, I'm coming. That was a really exciting week up in the Swiss Alps. We had this abandoned hotel to ourselves with a swimming pool and it was an exciting creative lab where I was spending like half the day at the computer and in virtual reality creating the artwork and then the other half of the day in the pool. and for the first week I was really just trying to create lots of different aesthetics and colors and space and shape and then animating the camera through these different environments and then I'll chuck that onto the headset and then I'll jump back in the pool and get a feeling what does it feel like to move through these colors and these shapes at this scale at this speed just to try to get a sense of what I like because I'm trying to inform myself of a new medium how much can we move through a virtual environment when we don't have the friction when we have this weightlessness of the water which was such a huge awakening because obviously we all know from VR this false sense of locomotion can be very nauseating so as soon as you remove the friction you've liberated and it opens up a whole new territory for storytelling and for specifically like VR storytelling And then I think the other thing that I'm super curious about, sort of from a psychological point of view, is spatial storytelling and how it plays on the brain. So a lot of my work and the themes of my work in the past has been a perception of reality, so how we can use storytelling and explore perceptions of reality. And so here we had another medium with this, again, weightlessness of being underwater. and moving through space. The combination of all of these sensory experiences and I really feel like this will open up a new territory for like academic research because very quickly it changes the way you're feeling mentally and this is again what Arndo was touching base on like with your heart rate adjusting quickly and Pierre was talking also about it with potential treatment in the field of post-traumatic stress and things like this. I've worked a lot with young people where we're doing virtual reality painting and physically moving in the environment. Like previously we'll be drawing our artwork on our computers or whatever but now we're moving our bodies through the artwork as we're creating it. And I've worked with young people who would be considered to have ADD But then when I put them in VR and they start working, they're moving their arms around and stuff, the physical activity of moving is really activating their minds. And one kid who, I couldn't get him to stay at a computer for longer than two minutes, suddenly in VR he's staying there for like two hours. super focused and so taking those kind of lessons that we learn about moving in space and being immersed in these other environments and how it can engage our minds in new ways I think like throwing the underwater scenario into that space it's just like another field that we don't have a clue like where that's going to take us but that's kind of what's super exciting about it. So for me this week at Sundance is going to be exciting just to talk to people after they've experienced it of all ages and just kind of start to sort of report back on how people feel once they pass through to the other side.
[00:22:39.525] Kent Bye: Yeah, having, I didn't realize that when you're underwater that it sort of shuts off your, the parts of your inner ear that causes a lot of the motion sickness. And so I did have this feeling of being able to locomote through and it just, yeah, just this very immersive experience. And it's hard to kind of describe how to put words to it. I mean, it is like very much an embodied experience that has different cuts, but yeah, I just, I very much enjoyed the overall experience. And for me, as I was going through the experience, I really loved you know, some people as I'm watching them, they're completely like zoning out, like they're in a float tank and not moving at all. But I like to just really kind of like move around and like feel like I was like navigating the space and you have like this body that's there. And as the body is there, then I'm trying to like mimic the motion. So I'm trying to like project myself into like the, that's me as I'm like seeing this body that's there and that's representation. And I'm like moving in the same motions. And so, And it's sort of, you know, it's hard to kind of invoke an out of body experience in that way. But as I was going through this, I was pleasantly surprised at how much of a trip it was, you know, and how being able to look around as well. But the other thing that was really surprising was the sound, because I hear the music and then there's a moment I was like, wait, how am I hearing like what's happening? And so like, as a sound designer, maybe you could just give a bit more context as to your journey into immersive space and then coming onto this project and the challenges of trying to do the sound design for something that is going to be experienced while someone's underwater.
[00:24:10.744] Mourad Bennacer: There's one thing that's definitely true with this project is that even to this point it raised more questions than actual answers about the technique in one way but how do you redefine the immersive experience in this context and at the same time it's also a context that is weirdly familiar to experience. Being in the water despite all the motion that's involved that's not totally new and being in this environment Sound-wise, you're always used to work with the space around you because simply you work in air, in an aerobic environment where the sound is propagated in a way that's precisely the thing we're trying to mimic whether in VR experiences, in multi-channel, multi-speaker experiences for the audience. What you try to look for is to multiply the channels to make the sound as rich as possible around you. So physically obviously in the water doesn't work that way at all because you could actually be more immersive just anything becomes mono in there. So you're surrounded by sound but at the same time what happens is it's like having sound coming from your head. in a way and that's like a weird paradox that I had to think away for a bit knowing that of course you won't have the same quality that you would have in the air but that's why when I jumped into the project actually Pierre invited me to work on this because he's actually tried some of my music some of the works I'm doing and actually find some kind of connection in there that he wanted us to explore more that's how we ended up doing doing this And it turned out to be a really interesting way for me to have my music heard a bit differently. Also, there was this whole connection with the moon landing and the recordings that Pierre intended to use in there. And I thought that was really interesting because the voice itself is one of the things you hear better underwater. So, it's hard to actually write a full experience for that and have any control over anything. But what you can do is have a feeling of motion, have a sound that kind of surrounds you without actually surrounding you. Have voices that remind you that something is happening because you can get really static in this kind of experience. And at the same time we ended up having this idea of basically designing the sound for the space where the experience happens. to actually have more of the sound experience actually comes from the user and the way he perceives the environment rather than the sound content in itself. That's one of the source of the multiple questions we're gonna have to solve in future projects, I would say.
[00:26:55.971] Kent Bye: Yeah, one of the aspects of this experience that is interesting and different, I don't quite know how to describe the experience because there's a difference between three degrees of freedom and six degrees of freedom. Obviously, you are having this headset that's not being tracked in a six degree of freedom way, it's a three DOF headset, but yet you're underwater and you still kind of feels like a six DOF experience in a way that I don't understand what exactly was going on perceptually there that maybe there's these signals from my body that was getting information like my proprioception was giving some input. But the sensor fusion was that, you know, I've seen a lot of VR and so I know that there's a certain aspects of it. So maybe you could just talk about the fact that it's 3DOF and is it even possible to eventually do 6DOF tracking underwater?
[00:27:44.470] Ando Shah: Yeah, when we started, the first thing we wanted to do was build a tracking system underwater. And I come from a background of building tracking systems for large-scale warehouse stuff. So I was like, yeah, this is going to be fun. Let's do it underwater. As soon as I tried to do it, it's one of the hardest things you can do. Most mechanisms will not work. Optical is thrown off by bubbles. Sound is really, really hard. Infrared is absorbed immediately in water, like five centimeters. So you can't use anything. Only magnetic sort of worked for us. But as soon as we started building it and in parallel trying these experiences, we realized that we don't really need it and maybe we can just get by in the beginning without having a tracked experience. And so at the end of that, we were like, OK, let's do these experiences and let's put the tracking on hold and let's see how it works out. And the feedback we got was so amazing that there was one guy who kept swimming, kept swimming. So because of him, we added the tether. We put the weights at the bottom, and then he went back in again. He kept swimming, and he refused to believe that he wasn't being yanked around, that he thought he kept moving. So it gave us this insight that you can sort of fool the brain into believing all of this is happening. So we don't really know what the mechanism is, and one of our advisors is Professor Jeremy Bailenson at Stanford, and he is studying this with an NSF grant. Part of it is how this mechanism is actually going about. So the short answer is nobody knows, but it works, so let's just make use of it.
[00:29:18.146] Kent Bye: Yeah, that's sort of my experience because like I didn't realize I was tethered and so I was like imagining that I was Not moving and then once I realized I was tethered then I started to try to like almost swim as fast as I could and I couldn't feel any resistance which was it was sort of an interesting way of Keeping people in one location and giving them this sense of being able to move around when I was in space in the spaced out experience I was really kind of like twisting around and moving and every time I was moving around I wanted to like act like I was like swimming through this like etheric vacuum so there's nothing actually there but with the water kind of imagining that I was getting this feedback. So what is it about that experience where I don't feel the tether pulling back on me?
[00:29:59.536] Ando Shah: Yeah, so this took a bit of design thinking and a lot of iterations. We started out with just ropes and just to see how it would work, we tried so many variations of elasticity to get to that right middle point where if you're too elastic you end up swimming and then you get pulled back later. So you actually do end up feeling that reverse pull whereas your body's not doing it. And if you're completely, if there's non-elastic, then you do end up feeling the tension and you feel it a lot in wherever you're tied. So we introduced this idea of this flotation belt, and then we tied the tether to it. And the flotation belt is really important because it allows you to float without having to swim or keep your legs up. Different people have different buoyancy of their legs. Some people's legs naturally float, others naturally sink. So the hip belt allows you to just relax if you want to, or swim if you want to, and it allows you to be tethered. So that elasticity is actually an important feature that doesn't pull too much, doesn't let you go too far either.
[00:30:57.958] Kent Bye: Well, I just wanted to start to wrap up here and just kind of unpack the experience itself a little bit more because you're a director, you have a vision for what you want, and you're collaborating with artists and also the sound design. What was the creative process in an experience like this? And how do you do that feedback loop of even articulating what you want in something that is so abstract? And what the communication process is of this iteration of getting to the point that where it is now, but this collaborative process, like maybe you could kind of unpack that a little bit more.
[00:31:30.227] Pierre Pyaré Friquet: Sure, a lot of begging. Please, please. No, I mean, I'm very happy to hear all this feed, to hear, you know, dear collaborators and colleagues and fellow humans, you know, I mean, to talk about this project. And I'm actually amazed, you know, how articulate they are, how they got into, you know, that vibration vision. And for me, basically, it's all about giving up control, giving up this idea of individual artists, you know, giving up this idea of owning a piece, I mean, just for all of convenience of language, not to use the term lead artist. For me, I just feel like I'm channelizing energy, I feel inspired by something, and then people share it, you know. I mean, people around me right now have expressed things that I didn't think about it, but it makes so much sense, you know, and I'm so happy about that. Souto and I, being in the Swiss Alps, in this strange place, thinking about the moon, we came up with so many ideas that inspired and helped. You can see that they put a lot in themselves, the personal self, and the music as well influences them a lot. So to answer your question, I mean, I did this project half asleep, you know, I was, I mean, half awake and that's when I wanted to get into it because I wanted to make VR experiences that talk directly to the reptilian brain in a sense that it's going beyond language, beyond rationalization. When you listen to music, it's a very visceral experience. You like it or you don't like it. It's primal. When you watch a film or read a book, you always put your mind... I like the cinematography, I like this acting, I like that scene. You always put these nuances. And I feel, as a society, as human beings, we need to go back to something and get a new experience. As I said, I mentioned the work, the therapeutic work with victims of sexual violence. I was talking about earlier the dissociation aspect. It comes through when the trauma happens, when the amount of violence coming from outside generated, you know, inside, you know, the experience that this first c-duration, you know, so your body frees, you know, and a part of your brain suddenly decides to stop receiving any information because such amount of stress that it can't handle anymore. So almost is a kind of interesting link between neuropsychology, biology, you know, emotions, and it's very, very interesting to have this loop. And why do we freeze? Because it's a reptilian part of our brain that basically plays dead. It's a survival mechanism that reptilians do. Snakes, lizards, they play dead. And all of that, you know, this sense of insecurity and different reactions to the body and disgust, hypersensibility, shame, you know, a lot of that. So I wanted to make a poetic work because I'm not a scientist, I'm not a doctor, I'm not a therapist, you know, I'm an artist. I wanted to play my role. And this process of the water, and why it's linked to basically, okay, we didn't talk about the project, it's basically a trip to the moon, you know, from Earth, you know, we see a character, a moon man, which is a mirror of ourself in the water, and I took inspiration from the Baron of Mtchaozen, the book, but also, of course, adapted by Thierry Grilham. and a man with the head of a moon and we go to the moon but then we go inside the moon and we go to the heart of the moon you know and then slowly slowly we lose you know a sense of reality and figurativeness you know we don't know where we are and then so we come back and for me the experience of zero gravity is important where we I mean you mentioned technically speaking how it works you know and the real deal for me is afterwards when you remove the headset you come out of the water And then, I don't know how you felt, but a lot of people, and that's what I want people to feel, is that they appreciate Earth gravity again. They enjoy, suddenly, the energy that we cannot consider it anymore unless we go to the Moon, I mean, we go outside, you know, the Earth gravity. And a lot of astronauts will actually tell you that, you know, what a joy to be on Earth, you know, what a joy to feel this gravity, you know, and... And so when people come out of this experience, you know, yes, they experience weightlessness, zero gravity, but then they experience gravity again. And that's where the experience comes outside the technology, outside, you know, the recipient of this information, of this data, of the video file. You know, it's basically designing as a creator, a process where there's a before, anticipation, doubts, how I'm going to be inside the water, how I'm going to float, you know, rest of breathing. And then the experience, of course, of the path going to the moon as a symbol or so, you know. and then to reconnect to your body, and then you come out of the experience and then you want to settle in and you feel a bit grounded. I hope that's what people feel.
[00:36:16.510] Kent Bye: I know that there's the concept of qualia, which is the actual experience that you're having. You know, people who've done a lot of VR, they can kind of like imagine what it would be like to be in a swimming pool and to be in VR. And the thing that I like to say about experiences like this is like imagine what that would be and it's like slightly different. It's like you do it and there's like things that are surprising that we still don't fully know or have figured out yet. But yeah, I kind of feel all those kind of like float tank and meditation. swimming in this like womb environment and going through the space and the sound, you know, just like it's a whole new experience that I think is really unique. So, but yeah, just to wrap things up here, I'm just curious to hear from each of you what you each think the ultimate potential of virtual reality might be and what it might be able to enable.
[00:37:04.492] Stephen Greenwood: Yeah, I think we're just scratching the surface of the type of transportation that we can actually deliver to people in terms of taking them to a realm that they aren't used to experiencing. So when you get in the water, your whole environment is changed. You're not used to being neutrally buoyant. You're not used to having your senses stimulated in this way. And there's so much more that we can do with haptics and vibration and sound and bubbles and jets. We've just released a product that is bringing the first few of those into production and it's a really convincing experience. People are even more excited about it than they are with this existing version. You know, I would love to see us working on things that involve therapy and wellness and meditation exercises, multiplayer experiences, gamified experiences that encourage people to repeat their experience over and over. So there's a lot for us to do. We've got a lot ahead of us and it's just been amazing to see the reception of it so far and the adoption of it in some of these water parks and resorts. And we're looking forward to solving more hard problems and figuring things out along the way.
[00:38:13.451] Sutu: I definitely want my studio underwater in the future. I think that week in the Alps, spending half the time in the normal familiar setting and then the other half as my aquatic self in the water. I was totally down with that, man. I could do that. I'm ready to migrate to that setup.
[00:38:32.608] Mourad Bennacer: Well, if you may, I can join you to your setup whenever you have it. No, but I won't pretend to have anything about VR especially, but I would say this thing, working underwater with this kind of experience, I don't see how we can get more immersive than that. But apart from that, we're in a bad country, who knows.
[00:38:52.804] Pierre Pyaré Friquet: Alright, so I think VR is really humanity's oldest dream, you know, in the sense of creating abstractness out of this reality. Like for example, sharing stories around the fire, you know, and cave paintings, you know, and the sense of being somewhere else, maybe and someone else, somewhere else and someone else. I think it's humanity's oldest dream. And we use technology to sort of defy the laws of nature. So this, you know, it's really really tied up to the evolution of humanity and I think at some point, you know, we want to instead of being creatures, you know, we want to be creators, you know, and like in the film Blade Runner, you know, you want to be, maybe the wrong reference, but I'm trying to say that we eventually want to become gods, you know, but in a sense of, in a non-religious way, in terms of mastering reality, in terms of embodying, you know, manifesting what's inside our head, you know, in terms of, biologically speaking. So, I'm very interested in something called dynamic reality or tangible media. So, there's one department at the MIT Media Lab, you know, we talk about that. If you watch the film Transcendence, the bad film Transcendence, we've joined up, or recently Black Panther, there is, you know, tangible holograms, you know, if you remember. Imagine pixels were replaced by sands, you know, moving sands, and you could actually touch. So, for example, 3D printing is actually... Imagine 3D printing which prints 24 frames per second with textures and everything, right? That would be the ultimate VR, I don't know if that answers your question. And I forgot, so I just want to wrap up that, you know, I'm developing what's next to the project and how I can use aquatic VR that is basically developing the proof of concept I did last year, the MIT VR cartoon. based on mindfulness meditation through an interactive piece where we recorded a breathing sound of snorkel is very very loud so we can pick up this information and then feedback the app where we reward people visually musically if they are calm you know and so that's we want to work with different victims of traumas to kind of feel connected again so It's just technology, but it's there to amplify your human experience. So we shouldn't be too caught up with this, but just go back to what's important for us.
[00:41:09.349] Ando Shah: I would echo what Pierre said and I think it's a question of unlocking human potential and allowing people to be their fullest selves and how we can enable all of this and it's reconciling this ancient paradox of us always wanting to escape who we are, what we are, where we are and at the same time facing the reality of what we are, who we are, where we are.
[00:41:34.345] Kent Bye: That sounds like a good place to stop. So thank you so much, all, for joining me today on the podcast. So thank you.
[00:41:39.188] Pierre Pyaré Friquet: Thank you, Kent. Thank you, Kent. Amazing. Thank you so much.
[00:41:42.890] Sutu: Thanks for having us. Bye-bye, Kent. Bye. Thank you very much.
[00:41:48.060] Kent Bye: So that was Steven Greenwood, the CEO and co-founder of BallastVR, the artist Piare, also known as Pierre Forcut, he's the lead artist on the project, Endo Shaw, he's the CTO and co-founder of BallastVR, Sutu, he's the VR artist on the project and also the founder of iJackAR company, and Sutu Eats Flies, as well as Mourad Benassar, he's the sound designer and sound composer. So I've had a number of different takeaways about this interview is that first of all, well, there are things that were happening to my body that I've never experienced in any other VR experience. It was certainly extremely immersive to be in the water, moving around. They were talking about afterwards, how there's a couple of different strategies that people take when they watch this experience. One is that you just completely zone out, don't move at all. And you just like take it all in. And so it's like this float tank type of experience where you're getting this stimulation. I did not take that approach. I actually took the approach of trying to swim around, and it was amazing that I could feel like I was swimming, but it took me a while to notice that I actually had this tether on because, you know, I was expecting that. I was like, how are they keeping me from running into the wall? But because the way that they designed this belt to go around your body, I didn't actually feel the resistance of the tension. It actually did a great job of allowing me to feel like I had this unending ability to just swim in any direction, which I really appreciated. So the other thing about this experience was that they said that when you're in the water It actually like kind of shuts off for your normal motion sickness triggers You're able to see different types of experiences that if you were outside of VR may give you motion sickness just because the disconnect between your vestibular system and what you're seeing and that you know something about being in water is somehow miraculously turns that off. So you're able to actually do different types of experiences within VR in water that maybe would give you motion sickness if you were to see it in any other context. And so they use that to be able to fly through this world and highly inspired by George Mele's Trip to the Moon, which is a film that was available. I actually went and watched it from beginning to end. I've never really actually watched the full experience. I've only just seen clips of the rocket going into the moon, but There's this whole scene where the people who are in the rocket go inside of the moon and they see all these mushrooms. And so in the, the spaced out experience, they have all these mushrooms that you're flying by. So taking a lot of the iconography and imagery from George Mele's is a trip to the moon and trying to embed different aspects into this experience of spaced out. The other thing that was really quite astonishing and amazing to learn about was the sound design that they have, which they're actually somehow radiating the sound waves out and you're able to hear them. not with any directionality, but to be able to have a sound design that's actually going through the medium of water, which I've never experienced before. As I was doing the experience, there was this moment where I was like, wait a minute, how am I hearing this so clearly? I don't have headphones on, but I'm just hearing it ambiently through the water. So they're able to transmit audio waves through water somehow. So that was amazing to see how they kind of figured that out. And then this whole concept of the mammalian dive effect, where Your heart rate slows down, the blood goes away from your periphery, your mind chatter goes down. And so I think folks that are in the float tank community where you're actually like floating in the water, there's lots of different research that's been done and floats and the physiological benefits that you get from that. I don't know how comparable this is because you are in water and you're just kind of floating there with a snorkel rather than lying on your back, like typically you would have with a float tank. So you have a snorkel that you're breathing, and you're able to still get, I think, a lot of that effect of having this real calming down. Like I said, I was moving around a lot, so I didn't necessarily have the direct experience of that more meditative aspect of it. The other thing that was quite interesting was to see how water would actually come in between what you're seeing and the screen. And for me, because I wear glasses and there was a little bit of blurriness that happened when I saw it, just because, you know, I'm looking at this and I'm seeing not exactly clear images and somewhat blurry. I still had a good experience, but it wasn't like going into a VR experience and having unincluded view on the world. And so. I think that probably depends on how the water diffuses through this whole mechanism and whether or not you get a clear view or not. So it seemed to be a little bit more of a hit or miss in terms of how clear an image that you're actually seeing. But because of the experience itself was pretty abstract, I was still able to get the gist of the entire experience. They actually put me through another experience that they've created, which was flying through a mountain. You're like doing this gliding with a wingsuit. They asked me what I thought afterwards, and you know, there's this thing called Birdly, where Birdly, you're actually like flying through the air and there's wind that is blowing against you. And at the Geneva International Film Festival, I saw a version of Birdly where you actually are floating through water. So Birdly is something that seems like a highly optimized for flying through the air, but then Birdly tried to be underwater. And this is the aquatic VR who's trying to fly through the air. So there's a little bit of an elemental mismatch there in terms of like when I'm flying through the air, I expect to have a lot of wind haptics in my body. And when I'm underwater, I expect to feel the water. So trying to figure out what types of experiences would be really good for ballast VR, trying to explore out this whole aquatic VR underwater experiences would also be interesting just because the water makes more sense with certain experiences. So just the final thought that I have is that this seems like a very unique, interesting distribution opportunity here. There's lots of pools all over the place. The fact that they're like doing specific things with water parks, I didn't get a chance to talk to them much about that or to experience all of them, obviously, because I was at Sundance, not a water park. But how they're starting to integrate these immersive technologies into these water park rides rides that, you know, are already probably very immersive in their own, right. But if people have been there enough times, then being able to add these immersive experiences on top of that would potentially allow them to do all sorts of different ways of being able to modify these experiences that are already there and just allow people more incentive to be able to come back to them. I know that there was some different theme parks around the world that were starting to do that with the gear VR and looking at roller coaster rides, like very old roller coaster rides where they weren't necessarily like bringing in a lot of new folks, but been there for decades and decades. But if they start to add something like the virtual reality technologies, then does that make it feel like a completely new ride? So I imagine that something like this, once they get more research, like I said, at the very top, the national science foundation has been investing different grants. I think they're working with Jeremy balance and at Stanford at his virtual human interaction lab, to be able to study what's actually happening when you do virtual reality in water and to see if there might be some additional therapeutic effects. I imagine that if there's going to be a route towards this, that yeah, there's a lot of different hotels that are out there that have swimming pools, but there's still have to hire people to be able to manage them. And I think that the overhead of that versus the desire for people to be aware that this is available and then wouldn't want to go see it and do it and be willing to go out into the water. You know, as I'm at Sundance, you know, it was a little bit of like, okay, now I have to specifically take this time and I know I'm going to be getting into water. I know I have to kind of dry out. It's cold. There's snow outside. You know, this is sort of like. not exactly the type of experiences that is necessarily easy for some people to be able to go and see at a place like Sundance. But it ended up being worth the trip for sure to be able to actually experience it. But it can be an inconvenience for folks. And I don't necessarily sure if this is going to be taking off within the consumer market. But I do see that if there's specific therapeutic applications that they see something like this, with aquatic VR then start to expect to see this get out more. I was really struck by Sutu's comment that he said you know he could really adapt his whole workflow to making art and then actually spending a lot of time in the pool looking at his art. That to me was really striking that you have an artist who's been involved in VR to see what it felt like to be able to start to create these aquatic VR type of experiences. So I don't know where this is going to go from here where it's going to spread or if it's going to be like a whole movement like the float tank movement is a whole thing where they have places all around the world where you can go into float tanks and do this whole sensory deprivation experience so whether or not they're going to be able to tap into that market not quite sure but it is certainly compelling and it is worth definitely trying to try out for yourself especially if you've done a lot of VR because there are things that are happening to my body that I couldn't quite explain. It was different. And lo and behold, turns out that I'm not the only one who had that type of experience and that there's all sorts of additional things that I think there's a lot more study that needs to be done as to what the future of this aquatic VR is. I think it actually has the potential to be able to actually have very compelling therapeutic applications. Being able to actually track your body, I think would be amazing, but it sounds like, again, that's one of the most difficult things to do. But if you're able to track your body and be in water, that would be absolutely incredible and amazing. As well as if they're able to figure out some sort of way to tie in an experience where you're actually like swimming or doing exercise, you know, something that people would already want to do, but you know, something like Beat Saber is an example where it may encourage people to start to do the type of exercise that they wouldn't ordinarily do. And if they're able to do it on a daily or consistent basis, I'm not quite sure if experiences that they've created yet would be compelling enough for me to want to do it each and every day. But having that as some sort of like goal to see, okay, what type of experiences could this be like either daily experience or, you know, finding a way to get the word out there in the different distribution outlets. This is something that the curators of Sundance Milo is talking about. Like there's so many different hotels around the world that have a swing pool, but again, it's the logistics of being able to actually have the docents there and the marketing and just getting the word out. I think it's probably a little early for them to be doing that, but I'll be very curious to see how they. continue to grow out. And if they have different agreements with different places around the world and where this goes in the future, but there is something super compelling to aquatic VR. And I'm excited to see what happens with the research that continues to be done on it and where it goes in the future. And if you have a chance to try it out, I definitely recommend to try it out just because you can start to like, imagine what it is. And then you're going to be able to compare what your experience actually is an experience like this, and then start to calibrate, like, what's it mean for this whole future of aquatic VR. So that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a list of supported podcasts, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
