The immersive story of Notes on Blindness: Into the Darkness (2016) VR experience (Quest, Rift) was awarded with a Legacy Peabody Award for Digital & Interactive Storytelling along with 15 other projects on Thursday, March 24th. This experience has come up again and again, and in a Voices of VR podcast episode that feels long overdue, I had a chance to speak with Producer, Creative Directory, & co-founder of Atlas V Arnaud Colinart about the evolution of this piece from the original New York Times Op-Doc/Short Film version that played at Sundance 2014, and then to the feature-length film and VR version that both premiered at Sundance 2016 (and later released on Quest on 2019). It’s a brilliant use of immersive storytelling and spatialized audio, and I get a lot more details on the evolution of this historic piece from Colinart as it is rewarded with a Legacy Peabody Award for the new category of Digital & Interactive Storytelling.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So, continuing on on my little mini-series of featuring some of the winners of the new categories of the Peabody Awards, today's episode is with Arnaud Colinard. He's the producer and creative director of Immersive Experiences and was a part of creating the Notes on Blindness. Notes on Blindness started as a short film that was a New York Times op-doc. John Hull had these audio recordings that he was recording of himself as he was going blind. It was translated into a 2D film that was also premiered at Sundance in 2016. It was really focusing on more of a biopic arc of the story of John Hull from a third-person perspective and his story as an individual. The Notes on Blindness XR experience was much more of a first-person perspective that debuted at Sundance 2016. It was much more trying to put you into the experience and use different forms of spatial audio. This ended up being a pretty catalytic experience on a number of different fronts, in terms of inspiring a number of different people to get in the VR industry, to really start to show the power of immersive storytelling. I actually started Atlas V, which a lot of the different people ended up starting a whole company to be able to start to distributed and put this together and put it out into the world. It's an amazing story. If you haven't had a chance to go see it, watch it again. I found out that I hadn't actually seen the entire experience back in 2016. There's actually a couple more chapters that have come out since then, when it's released later in the summer of 2016. So definitely check out Notes on Blindness. You can download it on the Oculus for the Quest or the Rift, actually. And yeah, that's a pretty historic piece, and I'm glad that I finally had a chance to catch up with Arnaud to be able to get a little bit more of the story and the evolution of this piece and to kind of reflect a little bit on the significance of what this piece has been able to bring about with the larger industry. So it's one of the recipients of the new categories of a Peabody Award from Digital and Interactive Storytelling. And yeah, it's an XR experience that's just being awarded, but one of the 16 Legacy Awards has this new category and new sets of winners that are being announced today. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Arnaud happened on Tuesday, March 22nd, 2022. So with that, let's go ahead and dive right in.
[00:02:30.356] Arnaud Colinart: My name is Arnaud Collinard and I'm producer and sometime creative director mainly of immersive experience and animation film.
[00:02:41.740] Kent Bye: Maybe you could give a bit more context as to your background and your journey into working on virtual reality pieces.
[00:02:48.743] Arnaud Colinart: So I started working in film around 2006, after I failed the test to be an history teacher three years ago, before I was doing a PhD in contemporary history around semiology and symbolism in political images. And I failed the test to be a history teacher and continue to do a PhD. And then I was really interested by documentary. And at the same time, it was the first internet bubble. And in fact, I did another master around digital creation. It was Paris 8, where they were mixing programmers, visual artists, and people like me coming from humanities, philosophy, artistry that didn't know exactly what they wanted to do. And in fact, I started to work with team to do interactive content online. CD-ROM, mainly focused on culture. And connected to that, I started working in a production company named Agatfilm in Paris. It's a company who did a lot of feature film and a lot of TV magazine. And they were looking for someone to propose web content for a TV magazine. They were just started at the time, the Metropolis for the cultural network, Arte. And this is how I started to work in digital content in the film and audiovisual company. And then I basically follow the different trends, web 2.0, mobile, mobile app. And in 2014, I produced a video game called Typewriter. It was a video game around typography. It had been a pretty big success. It had been selected by Apple App of the Week. And then for the first time, one of my projects had an international recognition. It was a game made by Cosmographic and three other programmers who are also, some of them working in VR right now. And with this project, Typewriter, I discovered Game Engine. And I continued my collaboration with the European cultural network, ARTE. And one day, Marianne Lévy-Leblond, who is head of content at Arte, asked us to meet two British directors, James Pinney and Peter Middleton, who were looking for a co-producer to co-produce a feature film called Not Some Blindness. And Arte, who is always trying to bring experience online in addition of linear content, tell us to meet this director and their producer and maybe to imagine something in addition of the Fetcher film. So with my colleague and co-producer, Agathe Film, David Cujart, we met Peter and James to co-produce the Fetcher film, which has been released on Netflix and on Arte. And at the time our proposal was really to do an interactive audio experience. It was in 2014, beginning of the year, I was really into serial, the podcast. And I was like, okay, we should imagine based on the testimony of John Earle and all this audio material, we should imagine an interactive experience, audio based around John Earle testimony in addition of the film. And we start working on that. And in fact, the first prototype we did on Nodes of Blindness were 100% an audio experience without any visual. And after a few user tests, because it was on mobile using a gyroscopy and touch screen, in fact, we saw that all the users were really captivated and focused by the screen of their phone, even if the phone was black. So at this point, we are like, okay, we are totally missing the point. We are working on the device with a screen and we have to use the screen. And then we start working in a 360 magic window mode. And this is how we start working around this topic of how do you represent blindness. And we did a second prototype in a 360 version mode on iOS. And at the time, my line producer on the video game typewriter, Damien Henry, was working on the project. And it was a Google Cardboard in which you can put your phone and have a virtual reality experience. And he showed me that and I was like, wow, okay, that's super cool. And this is how, in fact, I had been introduced to VR. I knew I already had the Oculus DK1, but because I was always working for a TV and especially public service company, the question of reaching an audience was always super important. So, in fact, I discarded VR pretty quickly because I didn't know how to reach an audience with a VR headset. And then Damien showed me the cardboard he was starting to develop. and that he ended to develop with Google. And then he had been hired by Google to develop the Cardboard program at Google Arts and Culture in Paris. I was like, okay, in fact, this is what this experience needs because in this 360 mode, there is all this world around that is basically eating my attention. And I need the user to focus their attention on the screen. And then when I'm in this box, it's like a small movie theater. and we should probably start prototyping on that. So with the dev team we were working with at the time called Audio Gaming, the team I work with on Typewriter, who now became NoveLab, we start prototyping the project from the 360 mode to a VR mode. And we decided to work on Samsung Gear because also it was more convenient. And in fact, this is how little by little, with a lot of iteration focus on what the the story needed and what the user could need, we end having this VR experience around Nodes on Linus.
[00:08:34.998] Kent Bye: So yeah, it sounds like there's a lot of catalysts there for this project from Google Cardboard to be able to see what's possible with VR, this audio reactive piece that started on the phone. There was also a separate film component. So maybe we could take a step back. And when you were coming onto this project, I haven't seen the film yet, but was there also a completed film that was made of Notes on Blindness that was taking these audio recordings of John Hall, who was recording himself as he was in the process of going blind in like 1983, had all these tape recordings that then the basis for the VR experience of Notes on Blindness, but was there also an accompanying film? And at what point did you come in on making the interactive part? Were these other pieces already in the process? And if there was an exchange back and forth, they were informing each other. Yeah. Maybe just give a bit more context on that.
[00:09:26.180] Arnaud Colinart: So when we started Not Some Blindness, Peter Middleton and James Pinney, the director of the Fletcher film, already did a short film for New York Times Op-Doc produced by the British company Archers Mark and their producer Mike Brett and Steve Jameson. And they produced a short film and they won an Emmy Award for this short film. And then they had so much content from the John Earle tapes that they were looking for partners to finance a feature film. And this is when we met them, when I was still working at Agathe Filmmakes in Yellow with my colleague David Cujart to co-produce the long form film and also to imagine something around on a digital format.
[00:10:09.988] Kent Bye: Okay. So Archer's Mark was involved in the process of both the piece that you have with Notes on Blindness, but also just more recently, The Morning You Wake to the End of the World, which this piece of Notes on Blindness was actually an inspiration and catalyst for this other piece that was just having its world premiere of episode two and three at South by Southwest, which ended up winning the jury prize for best XR experience. So congratulations on that. That's actually also coming out on Thursday, the 24th. So it's going back and looking at the origins of this and Archer's Mark. And so I don't even know at what point did Atlas 5 come about, if it was out of this project or if it was already created or if you helped co-found that as well. And maybe you could just give a bit more context as to the players of Archer Mark and Atlas 5 and this collaboration that started on Notes on Blindness.
[00:10:58.936] Arnaud Colinart: Yeah, so I think the funny thing with Not Some Blindness, with this project, in fact, a lot of people who have been involved in Not Some Blindness started after to work in VR much more deeply. So, Mike and Steve, who are also directors, were the producers of the Fetcher film, Not Some Lines. And on our side, we co-produced the Fetcher film with them, and we created with Peter James and Amory Labjurt from Novelab, we really created the VR experience. So, during the process of Not Some Lines, We read all the transcription that Peter and James had from the tape. We watched several times the short film. We participate as co-producer of the Fetcher, of the lecture around the Fetcher film. So we wrote the VR at the same time the Fetcher film was written. And the idea was to try to have something different that's in the Fetcher film. The Fetcher film is more like a biopic at the third person. So we decide in the VR experience to be at the first person. and to bring something that the film can't bring, which is bring you a sense of what is blindness through interactivity. So even if the Samsung gear was quite limited at the time, we tried to use every interactive component, especially the tap controller, and to find interactions such as creating the wind to reveal the environment through the sound, etc. So our idea was really, we even have a diagram of that to have the film and the VR to basically complete each other and to tell the biggest picture around journal testimony. So this is how we work with Peter and James, who just released also a film on Chaplin, using the same techniques around audio interview, always produced by Mike and Steve from Archon's Mark. And after Not Some Lines, we had been surprised by the way he had been received and the success of the project. And then I continued to be in touch with Mike and Steve. And after the success of Not Some Lines, I met a lot of people in the VR community at this time. It was a kind of golden age where all this community were in Cannes, Oculus Connect 3, a lot of creators involved. And at this point, I was like, wow. I was basically thinking, OK, this is what happened when the first iPhone arrived, where it was a revolution in terms of use, of how you use digital content. It was a creative revolution. And I was like, okay, I want to be part of that. And in my former company, we were eight partners and I was the youngest. The company wasn't focused on really on innovation and digital. And after several encounters with some other French producers, Antoine Quirol and Pierre Zandrovitch, We are basically in the same situation as I was. We said, okay, in fact, we have to quit our company and to start something. And at South By five years ago, Antoine Carole, Fred Volluer, who was helping me to distribute Notes on Blindness in the US, Pierre wasn't there at the time, but we decided, okay, we do the jump. We quit our company and we start a VR company. And this is how Atlas VR5 started. And because I kept a stronger relation with Mike and Steve and a real admiration for their work as producer and director, we always have been in touch, trying to work together. And one day we finally found our new project and it was on the morning we went to the end of the world.
[00:14:24.064] Kent Bye: Okay. So it sounds like that you'd work on notes and blindness. And then the people that were involved with it, it was actually after it had already been finished. Sundance 2016 was the first time I saw it. So at that point, Atlas V did not exist yet. And then it was later that you had founded it.
[00:14:39.127] Arnaud Colinart: Yeah, exactly. So on Notes on Blindness, I co-created the project with Peter and James, who are mainly focused on the story and the script. And Amaury Laburte and I were really working on interaction and creating an experience that could satisfy the user of VR. I did, in the past, video game animation. I produced several interactive documentaries. So for the first time, it was a perfect mix between documentary with a very strong story, with very strong characters and this real innovative format. And when I quit the company, I got Filmex Nihilo. In fact, in the company, no one was able to continue to distribute or exploit the project. And also because I created the piece, I was really involved in all the decisions. So at some point, when Oculus contacted us to do a Rift version and then a Quest version, this is Atlas V who handled the creative and technical part with the team from Novelab. And then we continue to distribute with a distribution agreement from Agathe Meixnello, Arthurs Mark and Arte the project. So this is how in fact we keep in the loop because you know there is a lot of company in the early year of VR who had been funded by VC for platform or technical tools. And in fact, with Pierre and Antoine, we really believe that if you want a device such as a VR headset to reach an audience, you need content. So we really think that building a catalog has a strong value. And all the experience Pierre, Antoine and I produced in the past, they produced iPhilip and Alteration. that were more 360 video and 360 fiction or not something like this. We tried to not leave this project in our former company and let them die. We really think these projects are still relevant and they are reaching their audience in a very long tail and several years. So we had been deeply involved also in the distribution of this project made in 2015 and 16, 17.
[00:16:44.553] Kent Bye: Yeah, I first saw it at Sundance 2016, and I just watched it again on the Quest. I'm not sure if I actually watched it since I first saw it at Sundance. I've certainly heard a lot of people talking about it over the years. And I guess the question I have is, is the version that's on the Quest now with the five or six chapters with an epilogue, was that pretty much the same version that was shown at Sundance 2016? Or was there more that was added Cause I know sometimes they'll have like a initial prototype of a piece, but then more stuff is added later and then released. But was it pretty fully formed the same structure and content that was shown in 2016 as it is now, or was there more stuff added?
[00:17:24.776] Arnaud Colinart: So for Sundance, uh, we present a four chapter on six. We added the kitchen scene. And we added the epilogue. But we managed for Sundance to still have a dramatic arc. The last chapter for Sundance 2016 was the core, when you are in the cathedral. And for the release, we added the scene where Jean-Earl describes how rain creates sound and helps him to understand the environment. And we added this epilogue.
[00:17:53.581] Kent Bye: Okay. Yeah. Cause it was like, there were scenes in there that I don't remember from when I first saw it. So that must be because it was added layer. So when was all six chapters in the epilogue, did that ever showed festivals or was that only come out when it was for the full release on Oculus?
[00:18:10.214] Arnaud Colinart: So the first and only presentation of the project was at Sundance at the same time of the Fetcher film screening at the Egyptian. and we released the entire project in June 2016 at the same time of the premiere of the Fletcher film. The film was distributed by Curzon, the British film distributor. And we had an amazing premiere in London and the premiere of the VR in another theater. And then after the release, we had several festivals. The main one was Sheffield and we had an award there. And maybe, you know, when I look back, the strange thing around This project had several dates around this project that had been pretty important. The day we received the answer from Sundance, that was a very important selection. It was the first time I had a project selected there. It was very important for the film. It was November 14, and it was in the evening. There was the attack of the Bataclan in Paris, and I was in Switzerland at this time. I was jury in a festival in Geneva. and I had to flee with my partner from Switzerland to France to go back to my kids because we thought they were going to close the border. And the day we received the prize at Sheffield for Not Some Blindness, the day after it was the vote for the Brexit. So I still have some dates that every year resonate with me, especially when, for example, we are waiting for Sundance election that happened this year. I'm always thinking of our first election and also this event that happened the same day on another time zone. But which I think is interesting with Not Some Lines is the first person I met when I arrived in Sundance after hours of flight was Kamal Sinclair. that introduced me to Jelena Rafiski from Oculus, and we collaborate on numerous projects with her. And honestly, we were in a bubble doing our stuff. And even during Sundance, you know, Sundance can be really overwhelming when you don't know Sundance. And we were happy of the experience, but we didn't know that so many people have seen the project. And in fact, it's in the week after that we realized that people really enjoy the project, we start receiving email from U.S. magazine. I remember having a call from Case Milk assistant asking if we could send the build and I'm receiving an email from Oculus saying that James Bond director who was working on the film around blindness wanted to see the project if we could send the build and all of that was like extremely weird. And then we had been invited at different talks. In fact, it made me realize that we achieved something with the project and also that a scene was really starting to emerge. This is how I met the Kaleidoscope community. This is how I met Eliza McNitt and how we produced Spheres with Jess Engel. This is how I met Nico Casavecchia and Martine Allais on Battlescar that we produced. This project has really been a fire starter for me and Without this project, I would never have started this new company for sure.
[00:21:14.703] Kent Bye: Yeah. And over the years I've done interviews with hundreds and hundreds of different people and notes on blindness comes up quite a bit as being a big catalyst for a number of different people really diving into the industry with a lot of seriousness. And so it certainly had an impact, which I think is reflected by the fact that it's being rewarded with one of these legacy awards for the Peabody's. And I just went back and rewatched the piece and I just wanted to unpack a little bit of why I think this piece works so well. I think for me, you're using spatialized audio in a way that is really quite sophisticated in the type of story that you're telling. You're telling a spatial story in the sense of you have these recordings from John Hull that he's documenting his loss of perception, but you're able to use this real minimalist point cloud aesthetic that is able to give you an impression that helps you to spatialize where these sounds are coming from. as you turn your head, the sounds are coming from those specific directions. And so maybe you could talk about how this piece originally started with a pure interactive audio piece. And if it was originally intended to be a spatialized audio piece where you were like navigating around, but then you saw that people were looking at the screen and then eventually moved into this fully immersive piece that as you look around, you're hearing things in a spatialized way. So maybe you could talk about that use of spatialized audio and how important that spatialized audio was in telling the story.
[00:22:38.377] Arnaud Colinart: Yeah. So what impressed me the most in digital content, mainly the one produced by the NFB was how they always found the right form for the story. And when we start to collaborate with Peter Middleton and James Kinney, the material from John was like, so incredible. That was like, okay, we have to do something around audio. This project should be pure audio. And when I produced this game, Type Rider, I worked with a French company named Audio Gaming. And Maurice Labeur was the CEO of Audio Gaming. focalize on interactive audio. So I called them immediately and I said, you work on the audio on this project typewriter. I have this incredible material. We have to imagine an interactive audio experience documentary. And then we start working on a mobile app using gyroscopy and using all the features you can have on the mobile phone to create a specialized audio experience. So it was really the beginning of the project was focused on specialized audio. And then we did several tests and we saw how the user were in fact focused on the screen, even if the screen was black. And then we kept the specialized audio approach, but we start to, we try to create a visual metaphor of blindness because what John is talking about is to bring sighted and unsighted audience together, but mainly to understand what is it to be blind. So this project is not, in fact, there is some accessibility aspect in Not Some Blindness, more on the feature film, but this project is really to raise awareness in terms of what it is to live without the sense of sight that leads 90% of our relation to the world. So because our senses as a sighted audience is deeply connected to our eyes, we bring back the screen to achieve this goal of explain or trying to explain what it's like to be blind based on general testimony.
[00:24:46.313] Kent Bye: Yeah. And as I watched through it again, there's some interactive components that I noticed. And I guess before we dive into those, did you have interactivity in the initial premiere at Sundance?
[00:24:57.802] Arnaud Colinart: Yeah, absolutely. In fact, putting aside the success of the project and being recognized as a project rewarded by a legacy Peabody award, I still think it's one of my best productions because we managed to find the right level of interactivity for the story, but also for the user. I don't think there is too much interaction. I don't think there is not enough interaction. And I think that the reason we succeeded to do that is because we had been, we tried to be so close to, when I said to the text, but so close to the transcription, so close to John Earle was saying, and we are always trying to think, okay, how the audience can receive John Earle words and interaction at the same time. So it was basically finding a rhythm between John Earle speaking and the environment asking you to interact with. So this part has always been important in the project. It was part of the Sundance presentation, and I think we had a pretty big shift with the Quest version. But we decided, in fact, to not add anything more than what we did. First, for production reasons, because at this time I was already working on Spheres and Battlescar, and I didn't have time to focus on that. And also because, for me, the piece was done, and it was part of this time, it was part of the early days of VR, targeting Samsung Gear, and I didn't want to do a 4K version. or 8K version as you can have for 90 movies and to change the interaction and be able to move around in free-roaming for a quest.
[00:26:37.645] Kent Bye: Yeah, so there's each of these scenes, it's cultivating this awareness to imagine what it would be like to not be sighted and to be blind and to slowly layer an entire ecosystem of sound, like one sound at a time in the first scene. You have these poetic metaphors that are illustrating these things so that as a viewer, you're able to connect to these things. And at some point you have everything just disappear. And I remember that moment from the first time that I saw it in 2016, that just how striking it was to build up this language and build up the sense of audio presence, and then to take it away to really emphasize what the experience of being blind would be like. And then these other scenes were the more interactive component, the second scene when you're following a bird. Love to hear you maybe explain a little bit more this idea of shooting out what seems like wind, blue feather-like particle effects that then illuminate different aspects of the environment so that as you're looking around, you're tracing this bird around, but it's also having this environmental awareness of listening to this world around you as you're on the porch. And what you saw that the interactivity was adding at that point in the experience, rather than just passively experience is something that could have potentially been a 360 video.
[00:27:51.220] Arnaud Colinart: Yeah. So I'm not some minus just to start with your first comment on the world building. So I'm not some minus the environment is absolutely a fundamental because how you perceive the environment is the core of the journal experience. So what we decide to do and what we push is that we code the project based on this baseline from John Earl, which is when there is no activity, there is no sound and the world dies. So everything in the project is coded to appear or be lighted if there is an activity. So our early prototype is a character walking totally in the dark. It's a 3D mesh walking. So it's basically black. And then we added a sound of their footstep. And because of the sound of the footstep, the sound is creating light on the 3D model. So everything, the code was based on the experience of John Hull. and the way he presented this world. And for me that changed everything because in a way we could have decided to do it the other way or to have the same effect without any connection with that. But the fact that we managed to have this baseline and this concept guide us in the entire experience and I think this is what gives to the experience is consistency. including the last scene where we are shifting everything and you are moving from a dark and blue environment to a colorful environment with a lot of white and colors and everything had been possible because we managed to find the key concept of the experience is in general testimony. And based on that, we had to build an experience. And because we wanted to use interaction, we decided to use what you can have in traditional video game or in interactive experience, which is like you need some cue and you did some things to guide the user. So in the wind scene, where Jon Ul explained that a windy day is for him the best day because he can feel the world living around him. We decided two things. First, we decided to bring much more poetry and that you as user, you will create wind. And to avoid that you don't know exactly what to do, we create this idea of your bird floating around that you're going to follow, sending this wind. So you don't know if it's leaf, you don't know if it's feather, you don't know if it's particle. And we decided to create a much more poetic visual experience to bring something more than just the audio testimony from John Hull. And I remember we had a lot of discussion with Jessica Briard, who also were releasing a film at the time, because in Nuts and Blindness, you are John Hull and at the same time, you are a kind of entity. You are you as user and you are not John Hull because John Hull is not able to blow wind from his mouth. And we had a lot of discussion with Jessica because I think she was not convinced by the way we decide to frame, in fact, the story. Like, who are we? Who are you in Not So Madness? Are you John Earle? Are you the user? Who are you? And it was a very good feedback. But I think we wanted this freedom, as in cinema, to have multiple point of view or to interrogate yourself in terms of who am I? Am I John Earle? Am I me? What is this world? Why I can blow wind? And in fact, The reason we did that is also because we deeply think that the experience of not some blindness, blindness is an inner experience, and there is a lot of poetry, John Earle was theologist and a pastor. So there is a lot of things that are much more than just a testimony about blindness. It's also thoughts on the world, on perception, on love, and there is an incredible quality in the book, Not Some Lines, that leads us to create an experience with probably much more poetry than when we expected at the beginning for a documentary project.
[00:31:52.606] Kent Bye: Yeah, just hearing those recordings from John Hull and hearing those episodic memories and then to have them recreated in VR, you know, I didn't never question as to whether or not I was John Hull or myself. I mean, I guess I was standing in and imagining the experiences that he had, but from my own embodied experience, I never really actually questioned if I was a part of the story because I was never addressed. I was sort of a ghost who is looking into the story, even though I'm embodied in the story. So yeah, maybe I've seen enough VR where I just don't think about that as much if I'm never addressed as a character. But there are moments when you're asked to engage with say, finding all the different objects in a room to again, have this experience of filling out a room, but also the scene where you're looking at the feet and you're walking out into the world and walking back into the world. I'd love to hear a little bit more about that, especially because locomotion within VR is something that sometimes gives folks motion sickness, but this was extremely comfortable because there's not a lot of vection and it's so sparse that it's a way to experiment with locomotion in an early version of a VR piece without having to deal with some of the negative consequences with locomoting through a piece. Maybe for the time it was radical in the sense where they didn't have a lot of motion because it did create a lot of emotions like this, especially back in 2016 in the early days of VR. But yeah, I'd love to hear a little bit about that scene of walking out into the world and then turning around and then coming back was something that was also interesting in the sense of just adding more tension as you're going out into the world and then turning 180 degrees and then walking backwards.
[00:33:29.542] Arnaud Colinart: It's fun that you mentioned this scene that I totally forgot. In fact, we prototyped this scene as an audio experience. The first prototype we did ever on the audio version was trying to simulate footstep in the snow at the same time you are working with your phone on the end. So because you are able to track vibration with the gyroscope, you put in your headphone, you add general voice that explains what it is to lose your sense of space because of snow. And the same time you were working physically on the room, you were hearing footsteps in the snow in the headset. And we love this scene. And we didn't manage to imagine it in the first VR version that we present in Sundance. And for the version at Sundance, we are very happy to what we present. And maybe we could have released like this. But my feeling was that the story didn't add enough contrast. You know, it was in the same beat. Right a few weeks before Sundance, we started to work on this scene and I discussed with the team and I said, we must find a way to do this scene because we need contrast. We need this moment of fear. We need fear because fear is part of journal experience where your senses change drastically. And it's a scene that we spend, honestly, I think months on it. First, to find the right art direction, to find the right beat in the scene, and as you said, to find the right locomotion and the right camera movement to not give you motion sickness, but also to create this sense of stress. So the most complicated part on this scene has been to create the beat and the awkwardness of being there and the fear, and then to coming back in the house at the end, that give you a sense of safe space and to make you understand how blind people can be trapped in their daily environment, because this is the only space you know enough to be able to live in.
[00:35:31.571] Kent Bye: Yeah. Yeah. I definitely can see that, you know, creating that sense of fear and tension. And as you turn around, you have like these red particle effects and other ways of trying to spatially communicate that sense of anxiety or fear or a sense of not knowing. And I wanted to talk about that last major scene, last chapter of the choir, because when I watched it today, I found it really evocative and beautiful just to be able to talk about the experience of the spatialized audio of the choir and how someone was telling John Hull about how they wished that they could have seen the faces and the light. And his reaction was like, actually, this was perfect for him to be able to have this choir experience. And I don't know, there's just something really poetic about that scene of how it over time actually also the forms of the people even started to blur out. So you couldn't even make out their distinguishing features that they just ended up being these colored blobs that were there that were representing where the sound was coming from. But I found that to be a really powerful scene to start to really wrap up this experience, especially as he's starting to reckon what's it mean to lost his sight and that he at the end says, okay, I think I know what it means now that I have become blind. But yeah, I'd love to hear any reflections on that scene of the music and I was thinking technically of like, wow, did they put microphones on these people or how you even recorded that? Cause it seemed like such a beautifully spatial reflection of all that music and to recreate that because it it's recorded originally as kind of a memory. And so it seems like a reconstruction of an event that happened that John was at, that you did a beautiful job of giving the viewer an experience of what he might've experienced there.
[00:37:15.073] Arnaud Colinart: Yeah, so this scene is really important because the experience that John Hull described is the moment where he's finally leaving the memories of being a sighted person. He's totally embracing and accepting this new way of perceiving the world. So that's why this scene is really, really important. And that was the last one we presented in Sundance 2016. The original text and also the scene in the film, because I think it's in the feature, doesn't happen in the church. It's still in a choir, but it's a much more casual environment, I think, in a party. But what Jean-Noël said is so powerful and so metaphysic, also in a way, that we decided to frame it in a church. Also, we thought it was more understandable for the user if we bring this choir in a church or a cathedral. So what we did is that we actually record a choir in France with Amaury and the sound engineer from Audio Gaming. And we build the scene based on that. And I think for us, we didn't add any hesitation because if there is one space that you can experience the nature of sound is in a church or a cathedral or this kind of incredible space, maybe in a museum. But I think as a kid growing in France, in church, you have two things. You have the smell of the incense and you have the sound. And for us, it was totally making sense to imagine this scene in this context. So we re-record the choir and we imagine the scene unfolding and the perception of John Earle shifting from these visual memories to a new way of perception, which is purely audio. And that's why the environment is shifting from 3D mesh to more vibrating and abstract shapes.
[00:39:04.870] Kent Bye: And on the recording, did you record each individual choir member with a microphone and then specialize it in a game engine? Or was this more of an ambisonic recording that was just put into a game engine that had that specialization built in with all the reflections and everything?
[00:39:19.608] Arnaud Colinart: Yeah, in fact, it was a pretty basic recording because we didn't have a lot of time and a lot of money to do that. So it was a really classical stereo recording that we simulate in the engine. Audio gaming is really specialized in audio for video games. So there are plenty of tools and software and plug-ins to do this kind of stuff. But something that is interesting is that a lot of people, right after the release of Not Somebody's Name, they came to us and said, you didn't specialize enough the sound. And we're like, what do you mean? It's like, you should have much more sound elements on the left and then on the right. So basically specialized audio for a lot of people, in fact, means the THX effect in an AMC theater. It's like you need to have the Star Wars fighter crushing your ear on the side and then on the other. And we said, look, in fact, what we are trying to do is something realistic and we are specialing the audio in a visual way and we are playing with sound but we don't want to do a tech demo for audio specialization also because if there is one thing that people are not perceiving well enough this is sound we all have a different perception of sound So we did some version of the audio mix with much more specialization, but didn't make any sense creatively or in the experience of John. So we decide everything is specialized, but in a pretty realistic and low key way, because what is specializing the audio is in fact the visual and the experience of being in the space.
[00:40:50.251] Kent Bye: Yeah, well, experiencing it, I noticed the reflections and it felt like an actual recording, but maybe it was just the specialization of putting those characters in those locations that my mind was adding extra additional layers of specialization that weren't actually there. So either way, the scene worked really well for me as a culminating point for the piece overall. So yeah, just, I guess it was, we start to wrap up and reflect on this being awarded a legacy Peabody award for not only the innovations that were done, but also the impact that it's had on the industry as a whole. And so I'd love to hear any other reflections in terms of, we talked about earlier in terms of all the different people that you met and leading to other projects. And I've certainly talked to a number of people who have cited this as a piece that really was a turning point for them to get into the industry. But as you stand here now and look back of the last number of years, since this first premiered in 2016, what do you assess as the impact of a piece like this in terms of pushing forward the forms of immersive storytelling, but also opening the minds and the possibilities for the industry at large for the future of immersive storytelling?
[00:41:57.330] Arnaud Colinart: So first, The thing I'm trying to explain when I'm doing keynotes on Not Some Blindness or I have questions on this project is that I'm not a futurist. I didn't add, you know, a strike of genius like, okay, let's do a VR project about blindness. No one will never finance a VR project about blindness if you pitch it this way, you know, it's so counterintuitive. So the reason we managed to make it work is because During eight years I worked in interactive content, in online interactive documentary, in video game, in animation, and I was working with programmers, visual artists, and at some point I think we had enough skills and we encountered the right team and the right story to do something meaningful. I really think that most of the success and most of the successful work that had been done are connected to work and not intuition. There is a part of intuition, but there is mainly work. And Not So Blindness was a lot of work from Peter and James, from Amory and the audio gaming team, and from myself. And after Not So Blindness, I continued to produce video games. I did some other VR piece. And in fact, little by little, I saw during conferences, I saw during an OC4 visual of Not Some Blindness behind Mark Zuckerberg, et cetera. And I was like, wow, OK, something is happening. We had an incredible post from Sheryl Sandberg, the CEO of Facebook, explaining that Not Some Blindness was the reason why they were invested in VR. And I was like, wow, OK, we touched something there. And as a creator and producer, At this time, I was more focused on, okay, I don't want to be the Not Sunblindness guy forever. And I want to move forward and I want to continue to do new things. And this is where I'd be really, really grateful to meet through René Pinel and Kaleidoscope Network, people like Eliza McNeat that did Spheres with Jess Engel, my co-producer, then Martin and Nico. And when we started Atlas V, it was the beginning of these two projects of Spheres and Battlescar. And I was really saying to myself, OK, if we were working on the chapter one of each project, we present both projects at Sundance. And we are like, OK, if we manage to go to Sundance with these two projects, and if we manage to achieve what we want to Sundance, I'm going to do something for myself. And I will take time to create from scratch an entire experience. Both projects have been selected to Sundance. And in fact, I never did anything from scratch as a creative, because in fact, projects are taking so much time. And in fact, I'm much more excited by stories that other creators have to tell. And then I have to say that because the industry is taking time to grow into a revenue and also taking a direction really strongly through video game, a lot of creators coming from a film background, such as Eliza or Martin and Nico, left right now the industry. And in my opinion, it's because as producers, we have some difficulty to bring them the budget they can have on commercial or feature film of TV series. And this is what also pushes us at Atlas V to develop in-house our own project. And that was tough during the pandemic because we were a little on our own with our team. We have great creators in our team. But during Sourcebuy this year, I have been so happy to reconnect with creator May Abdallah from Anagram, Vincent Morisset, Raki and all the team behind Minimum Mask. They are really people who have something to say and they are the way to say it. And what I want to do and what we want to do as a company at last is to be able to empower this voice and to bring their work to life. So I really hope that we will see. VR and immersive experience and potentially that include the metaverse going mainstream to bring these voices in our field. These voices were there. They left because in fact they didn't add any way to have a living to work on a VR project. Right now, most of the people who are invested in VR are coming from video game studio or are video game creator, which is great. but I hope that with the market growing we will have room for narrative experience or interactive experience that are not game and that will bring back these creators who are more diverse than in the video game sector.
[00:46:23.299] Kent Bye: Yeah, well, this news of Notes on Blindness being awarded a legacy award of a Peabody, it's coming out on the same day on Thursday, March 24th, as the release of On the Morning You Wake to the End of the World, which has been another project that in some part was catalyzed and inspired by Notes on Blindness because Some of the people that were involved in catalyzing the origins of that project saw that piece and then reached out to both Archer's Mark and yourself to bring that project out. So that's been another project that I guess is a creative child of Notes on Blindness. And so, yeah, I just wanted to see if you have any other comments on that, just because I feel like that's another example of a really powerful story that has similar point cloud aesthetic approach that, you know, the way that you told the story necessarily because of the pandemic, it was going to be a lot more immersive, but actually think it works quite well. And it's just a really powerfully told story. And yeah, I feel like it's part of the legacy of the notes on blindness as well. And if you have any thoughts or comments on that.
[00:47:21.521] Arnaud Colinart: Yeah, I didn't realize that the Peabody announcement would be the same day as the release, which is again, really symbolic, because of the project we start on the morning, that was called at the beginning 38 minutes, four years ago. So one year after we start our company Atlas V, and not so many years have been released in 2016. So first, for me, there is several connections between On The Morning You Wake To The End Of The World and Not Some Blindness. And first, this is all connected to Mike and Steve. I think we share something which is we want to tell stories. And for me, in fact, that's the most important element of what we are doing. There is still a lot of VR projects that in fact don't have enough resources or where the creators don't have enough skills yet to really empower the technology with their stories or empower the stories with technology. And I think, in fact, for me, the most important in both experiences is the quality of the story. And both projects are connected to non-fiction, are connected to audio testimony. So they are connected to truth, in a way, and to human experience. This is, for me, what connects both of these projects. And in fact, this is exactly what VR can bring as a medium, is to bring experiences and an intensity of experiences that you will not be able to live with another platform. On the visual side, there is definitely a connection that you can also have in the project we co-produced called Vestige, which is one of my favorite also. It's because I think, especially with my business partner, Pierre Zondrovic, who co-created the project with us, we are obsessed by memories, we are obsessed by time passing, we are obsessed by disparation, collapsing. And in fact, in point cloud aesthetic, Pierre is doing a lot of point cloud photography. For example, the aesthetic of point cloud is something that we really enjoy to play with. And also that reflect a lot of melancholia. And I think something that connects not some blindness or vestige or on the morning you wake to the end of the world is melancholia and the states of looking at the world that's collapsing and trying to have faith in the future. And there is also, these three projects are also connected with grief. So I think we always have been interested by this aesthetic. For Not Some Blindness or for On the Morning You Wake, we had a lot of visual references in mind that we didn't manage to achieve visually in the engine of the platform because of Samsung Gear for Notes on Linus. And it was at the time already an incredible technical achievement to run that on the Quest, this level of particle. And in fact, with the audio gaming team that became Novelab a few years after, we did it again on Quest because I don't think you have a lot of project with 45 minutes of volumetric capture with point cloud fully interactive in a Quest. It's a tremendous technical work that Novelab did. So I think as producer, it's easier for me to embrace the technical constraint and the impact of this technical constraint on visual style. But we had so much more in mind visually. We really wanted to have much more point clouds. We really wanted to make it run on the NVIDIA 1080 full particle all around. And in fact, it wasn't possible, but at some point we are against not there to do a tech demo. We are here to tell a story. And so at some point you embrace the constraint of the platform. And if these particles became a vertex, the small triangle you see in the experience to facilitate the optimization, you do it. So as creator, Mike, Steve, Pierre and I, we are really excited to continue to work together, but for creators who are not coming from the video game background, the fact that the industry moved from huge graphic card and PC to mobile and snap diagram processor, we really lose a sense of beauty and of precision we could have at the time, and we are really missing that. So we are trying to embrace free roaming and interaction and it's super interesting but we can't wait to have a powerful computer running on your headset because it's going to totally change how visually appealing the experience will be and it will really change also the connection between the user and the experience.
[00:51:51.630] Kent Bye: Yeah. Or, or even things like cloud rendering or things that can bump up the type of rendering that you get on a PC, but seeing it on the quest. So yeah, I'm, I'm sure that Atlas V will be on the frontiers of whatever the full extent possible and really pushing the limits. And for sure, you're already pushing the limits of what's possible on the piece, like on the morning you wake to the end of the world, what you've been able to achieve there with having that run on a quest. Yeah. Just not only an amazing story that you've been able to tell, but also just the technical achievements to be able to have, like you said, that much volumetric capture. But yeah, I guess, uh, as we start to wrap up and I'm just curious what you think the ultimate potential of virtual reality and immersive storytelling might be and what it might be able to enable.
[00:52:33.987] Arnaud Colinart: So the most powerful immersive experience I had. is definitely the Sleep No More experience. So when I did that a few years ago, I applied for a training at Punchdrunk and it was basically, okay, this is what I want to do. VR is just a transition for my students to do immersive experience. And I was obsessed by that for months. And then the pandemic strike, and of course, I think it challenged this business model. I'm going to see the new Punchdrunk piece in a few weeks in London, The Droning Man. My personal opinion and what I would love to continue to do is immersive experience without any headset or without any technology that is directly visible. We are working on the immersive installation with Kinect Azure to track movement. We are using different setup where we are trying to remove the headset until this headset or small glasses. We'll continue to do VR, of course, because there is a lot of things that you can't achieve without a VR headset. But my opinion is that it's just a training for something that would be a powerful trend that probably my kids are going to live, which is this immersive experience that you can describe in the game by David Fincher, for example. I think for me, this is part of the same area. And virtual reality will definitely merge with augmented reality and MR headset. And I think it's going to be used deeply in education, for sure. But in entertainment, it's going to be used also probably for shooters and esports. But we still have this form factor that is really uncomfortable, even in the best headset right now, which is for me the Quest. And what I'm interested in is, I still think that we are at the very beginning of virtual reality. We are just starting to understand what it is, the same way audience discover cinema in cinema. And then we add home video, et cetera. We are still maybe at the end of the cinema discovery. And I'm curious to see how immersive experience would be mainstream. And I think it would be a physical mapping, but I don't think we will have technology on us or it will be very light one. It will be through your phone, it will be through your glasses. After Sourcebuy, we went to the Infinite, the experience by the Fee Center at Houston. It's definitely the future of exhibition. It's absolutely brilliant. Your experience is really impacted by the weight of the headset, by the difficulty of tracking and the limit of tracking technology, but you can definitely see when you go there, where the entertainment is going. The same way we had this huge arcade video game. When you see a video game arcade, you can totally project that at some point you're going to play on a portable device. And for me, this experience was really the proof of that. The train is at full speed and now it's a question of technology and finding the right business model for devices and also for this kind of exhibition to be developed.
[00:55:42.124] Kent Bye: Yeah, just a couple of things to follow up on there. I saw Sleep No More in October of 2011 and similarly my mind was blown and I don't think I've had an immersive experience in VR that has quite matched that level of immersion and just being really transported into another world that I had in that experience of Sleep No More. I don't know if you had a chance to see Believe Your Eyes. That was a project that was a piece that was at a gallery at Venice in 2019 by the Phi Center. Did you get a chance to see Believe Your Eyes? Yeah. That was another VR experience that was using a gear VR experience, but the whole immersive theater actors that are surrounding that piece, this had a larger context. was another thing that kind of blurring the lines of reality. I just really appreciate everything that Punch Drunk is doing. So I'm very curious to see where that goes for you and the future of Atlas V in terms of immersive theater productions. And I'm looking forward to seeing The Infinite in Seattle when it opens there in May. I'm here in Portland, Oregon. I haven't had a chance to make it down to Houston for the Felix and Paul piece of The Infinite, and I think it originally showed with the Fly Center in Montreal. But yeah, just what Felix and Paul and this kind of being able to have a room scale, or maybe even larger than room scale, like just gallery space scale, walking around and to have the same spatial experience of the International Space Station. So I'm really really looking forward to where that goes, especially with what they just produced with Spacewalkers of putting a VR camera out in space, but adding those other additional elements of immersive theater in the physical reality. The other thing that comes to mind, of course, coming back from South by Southwest and the Phi Center is composition by Vincent Morissette and being able to really have that physicality of the projection mapped experience with other people. Yeah, just really pushing the forms in terms of the types of interactive and immersive experiences. I don't know if you have any thoughts or reflections on that piece of composition and what the Phi Center is doing in general there in Montreal.
[00:57:40.867] Arnaud Colinart: Yes, in fact, the first time I had an award for production was a project around the Chernobyl area named The Zone, and it was an online documentary. We had an award at Festival du Nouveau Cinéma in Montreal, and we shared the prize with BlaBla, one of the most awarded work from Vincent Morisset. And this is how I discovered his work, and I was amazed by his level of skills, animation and programming, and the poetry behind it, and the imagination. So when I went to South By on the XR floor and I saw Vincent Hadepies, I just ran directly through there and they were running some tests and I was like, okay, can I try it? So I had the chance to spend half an hour in composition. And in another life, I had a band and I toured and I was close to a totally different life. I'm really connected to music and I'm really trying to bring music to my kids. And I'm always looking at instruments, especially synths, that can be playful and that you can share with kids and how it can be playful without knowing the instrument. And I think what Vincent and the FI Center achieved there is something that is really a musical experience in which you can be free and at the same time guided and at the same time you have all this animation and visual element that create a world. And then I said to Myriam, Machar and Vincent, what he said is like, okay, you have to commercialize that. I will buy this to play with my kids. as an instrument. So there is a lot of tracking with a Kinect. So I'm not sure it's easy, but for me, that was definitely my favorite piece at South By. And the Fee Center, I mean, they are the forefront of contemporary art, especially on the digital side. And what they are doing in the immersive sector is really unique. They are really pushing and challenging the I think they are also pushing museum and challenging museum. I went with the Atlas V team to the NASA museum before going to the Infinite Exhibition. And what we thought is like, okay, the NASA gave to SpaceX the launcher. They should just close this museum and give them to the Fee Center and Felix and Paul, and they're going to disrupt what it's like to do a space museum in the 21st century. What I hope is that, especially in Europe, we could find business model to bring this experience in Europe. It seems still very complicated because the cost of production are high. I'm not sure that in France people are attracted this much by technology. It's coming, but I really hope we will see more and more exhibition and maybe smaller one with an easiest business model to bring this kind of exhibition in Paris or in London.
[01:00:22.132] Kent Bye: Great. Well, we, we covered a lot of ground today and I'm just wondering if there's, uh, anything else that's left and said that you'd like to say to the broader immersive community.
[01:00:30.335] Arnaud Colinart: So first I'm really grateful to have this opportunity because I think all the creator and producer in this field, we can feel very lonely. Sometime in our piece, not some blindness took us three years on the morning you wake took us four years. and my message will be don't give up and continue to explore and if you need a space and to come back to animation or your traditional media film, go but please come back because we need your voices, we need the producer, we need the creator and we don't want to have just again and again the same creators or company to present their work. We need to be challenged. We need to have challenger. We need to be beaten. We need to fail. And I think this is something that we have a lot of difficulty to find right now in the VR production is the right to fail and innovate because it's always risky and it's taking time and there is not always revenue attached. So yeah, my message for the immersive community will be don't give up and if you leave, please come back soon.
[01:01:35.100] Kent Bye: Hmm. Awesome. Well, thank you so much for joining me today on the podcast to be able to unpack not only your journey with notes on blindness, but to reflect on the wider industry. Like I said, congratulations again for receiving a legacy award from the Peabody. I think it's very well deserved. And as I talked to many people from the industry, it certainly had a huge impact. And as I talked to you today, I can just see how that was the seeds of so many other projects that you worked on. And I'm just glad that it's, had the opportunity to get the recognition, but also to, as on the same day happened to be celebrating the launch of On the Morning You Wake to the End of the World, which was the winner of the best XR experience at South by Southwest, an amazing piece as well. So hopefully folks will be able to go back and watch both of these pieces and just see the power of immersive storytelling. Cause I think, like you said, it's the power of the stories that are able to use the affordances of the medium to connect to the human experience. And I think with the pieces that we talked about today and all the other explorations and pioneering work that you've done that Atlas V have certainly been pushing the edges of the forms of immersive storytelling. And yeah, just thanks for joining me today to be able to help unpack it all.
[01:02:41.563] Arnaud Colinart: Thank you so much. I was really happy to share this conversation with you and I hope we're going to meet soon on some other festival or immersive theater experience.
[01:02:51.668] Kent Bye: So that was Arnaud Colinard, he's a producer and creative director of Immersive Experiences and one of the co-founders of Atlas V and worked on Notes on Blightness, both on the 2D film as well as the XR version. So I have a number of different takeaways about this interview is that first of all, well over the years as I've done lots of different interviews with different people, Nose-on-blindness is often referred to as something that was a real catalytic moment for a lot of people to really show the power of immersive storytelling and to get them to join into the industry and see what they could do in terms of pushing forward the medium. For me, it was really fascinating just to hear the evolution and all the different ways in which they really wanted to create this as just a spatial audio, but at that point, They found that people were just looking at their screen and they wanted to just put people completely immersed to be able to create effectively these different aspects of these metaphors of people moving around and then to slowly take them away to recreate these feelings of going blind. So just a lot of really interesting interactions within this piece as well as you look around and your wind is engaging with the environment and it's also kind of reflecting on how The wind was the way that John Hull would be able to understand the spatial environments that he was in, and then to walk through and then walk back to be able to get this feeling of unease and anxiousness as you're going away from this small area where you're completely familiar. And when John was going outside of that, and people who were blind, They have a little bit more anxiety. So just trying to create some of these different embodied experiences and just a really poetic ending and epilogue and just a really, really beautiful piece. And yeah, I'm really glad that it was honored with one of the legacy Peabody Awards. And I'm just also glad to be able to catch up with Arnaud and to reflect upon how this was a big part of his own career of starting Atlas V as well with his other co-founders. So, that's all I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a support podcast, and I do rely upon donations from people like yourself in order to continue bringing this coverage. So you could become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
