UTURN is a 360-degree video that seamlessly blends together two 180-degree hemisphere worlds containing parallel storylines of two different perspectives at a tech start up. One perspective is from a male founder’s point of view and the other perspective is from a woman coder who is crunching to finish a demo for investors. The crossfades from the quad binaural audio design by Shaun Farley helps to dynamically mix the conversations from different together in a way that really sells the experience of walking between two different worlds. Usually a 360-video doesn’t have reactive edits of the visuals, but UTURN proves that there is a lot of latitude in creating an immersive and interactive audio experience that feels reactive to your gaze.
UTURN explores themes of gender discrimination, and uses the 360-degree medium to capture group dynamics and unconscious bias within the workplace. I had a chance to catch up with NativeVR’s reative director and UTURN executive producer Nathalie Mathe to talk about the themes of sexism and gender discrimination covered in the piece, the technical storytelling innovations, the innovative sound design, the challenges of telling parallel stories, and some of the funding challenges involved in bootstrapping an independent production.
LISTEN TO THE VOICES OF VR PODCAST
UTURN is one of the more technically accomplished 360-videos that makes a number of storytelling innovations to create the feeling of an interactive experience of seamlessly turning between different worlds, and it addresses important themes of sexism within the tech industry VR in a new way. Mathe is looking for educational opportunities at colleges and corporations to share UTURN as a catalyst for facilitating group discussions about what people experienced while watching it. UTURN currently does not have any distribution yet, but I’ll update this post if it becomes available to check out as I think it has a lot of important lessons for the future of storytelling in VR.
Here’s a trailer for UTURN
This is a listener supported podcast, considering making a donation to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Support Voices of VR
- Subscribe on iTunes
- Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. So there has been a lot of news recently, both in Silicon Valley and Hollywood, about sexual harassment, sexual assault. The whole Me Too social media campaign has just brought a new level of awareness in terms of the pervasiveness of the experience of women when it comes to dealing with either sexual harassment or sexual assault just throughout the course of their lives. And, you know, there's a VR experience that I'm going to be talking about today. It's by Nathalie Maté, and it's called U-Turn. And she's talking about the issue of gender discrimination within the workforce. And there's some quite innovative VR technological techniques that she's using. She's actually splits the 360 video into two distinct hemispheres. So from one half of the 180 degrees, you see a woman's point of view, and she's kind of in the trenches working hard with some other coworkers. And then you turn around and the other 180 degree hemisphere, you see the boss's point of view. And so they have different interactions, but throughout the course of this experience, you kind of see the different levels of unconscious behaviors that happen towards women within the workplace. and uh... it's it's a story that's told in parallel there's these two distinct stories that are happening and the video is saying the same but as you turn your head around you're getting a different audio sound mix uh... they're using a quad binaural sound mix so that as you turn your head you're creating a reactive audio mix based upon where you're looking you're getting these two worlds are blending together and uh... it was such a powerful effect and uh... i think that storytelling mechanism, it's super powerful to be able to kind of dip in and out of these different worlds and to be looking at these two different perspectives. So we're talking about both the technical process of making this experience of some of the other larger issues of gender discrimination and sexism within the tech industry, as well as the funding issues of what it's like to be an independent creator within the VR community. And the level of creativity that it has to take in order to learn the technology, be able to innovate in terms of storytelling, as well as how to fund a project like this. So that's what we'll be covering on today's episode of the Voices of VR podcast. So this interview with Natalie happened at the Samsung Developers Conference that was happening in San Francisco, California on Thursday, October 19th, 2017. So with that, let's go ahead and dive right in.
[00:02:48.960] Nathalie Mathe: Okay, so my name is Nathalie Maté and I am a VR creator and filmmaker and I've been working in VR for the past four years. And before that, I was in film visual effects and animation for about 10 years. And even before that, I was more in computer research. So my background is both like technical and creative. That's what really excited me about VR when I started.
[00:03:16.365] Kent Bye: Yeah, and maybe you could tell me about the experience of U-Turn, because I had a chance to see that at Oculus Connect 4, and I was really impressed with the experience of being able to kind of be in two worlds at once, where you're looking one way, you have a 180-degree view of one perspective, and then you turn around, and you have the other perspective, but the thing that really sold the experience overall was the audio of having that kind of seamlessly mixed as I'm turning my head around. I felt like I was really kind of dipping into one reality and kind of slipping out of that reality and going into the other one as I turn my head around. It was quite a unique affordance of virtual reality that I think that you used it to great effect. So I'm just curious to hear how this project came about and then that mechanic and how that kind of drove the experience that you created.
[00:04:04.012] Nathalie Mathe: So originally, we wanted to make an experience to make people feel how it is to be a woman working in a male-dominated startup. But we also wanted it to be inclusive, so to focus more on how some tech startups can be really toxic environments, both for men and women, not just from the woman point of view. So earlier I partnered with other female storytellers and we brainstormed different ideas and we came up pretty fast with the idea that we wanted to do something very realistic, not space rescue missions, not some of those. And we wanted to do that with real actors, to have emotions and dialogue. And then I also talked to a man who became my co-creator, a guy with a background, Justin Chin, with a background in gaming and also filmmaking. And he told me early on, like, yeah that would be great to also have the male perspective and to alternate you know between the two or to be able because and he doesn't even remember saying that like but I remember he told me like no guy is going to want to be in a female's body for more than a few minutes so I started to think about You know, how can we create an experience that puts you in the shoes? Because I was very decided from the beginning that I wanted a POV experience with a real actor's body that, you know, you could embody and values different things. I thought about, you know, do we do it with a selection, gaze-based selection? Then you shift point of view. Do we do it as a director's cut? So we alternate between different perspectives. And none of that was really appealing to me. It didn't feel natural. I wanted the experience to really be fully immersive and not disruptive. Then, you know, I just came up with this idea of splitting the space in two and having the one story on each side and the female point of view and the male point of view. And when we started to work with some designers, that was like after the shooting, when we started to do the posts, we really worked hard with them to find the right level of how do you remind people that there is still this other world behind you when you are watching one side and not forget to turn, but at the same time not have it be disruptive so that it takes you away from what you are watching. So it was kind of this subtle mix between, yeah, reminding people and playing on the fear of missing out and not being disruptive.
[00:06:49.953] Kent Bye: Yeah, it took me a good maybe minute or two before I fully realized what was happening, which was that, you know, the sound was actually dynamically mixed down based upon where I was looking. It felt like pretty seamless and it actually kind of took The phone calls, I think, when people were kind of talking on a phone at the same time, when I realized that I could turn my head around and I would hear the voice coming through the video chat system that they're using. And then I turn my head around and then I hear it in high fidelity and I could kind of like turn my head in the middle and kind of hear like an equal mix of both worlds happening at the same time. But in one world, you kind of have the workers that are working around the clock, putting in overtime to get this demo to work. And then the other side, you turn around and you see the people who are the managers who are at dinner, kind of like having these locker room talks with their other male friends, and then also in the board meeting. And so you kind of switch between these two contexts of the people who are in power and the people who are kind of at the bottom of the power totem pole. and you kind of see that dynamic but also turns out that gender comes in in terms of the different sexes and how that kind of plays out so you get a full holistic experience of all that and so I guess as you were designing that is this from your own direct experiences or talk a bit about like how you were actually kind of constructing these different worlds because you know it is kind of a stark context switch on a number of different times when you're in kind of the work environment and then you kind of slip into this privileged conversation amongst men and then sort of like how you're able to kind of piece that story together in that way.
[00:08:25.886] Nathalie Mathe: So yes, I've been working in tech and media for 25 years, so definitely I've experienced a lot of the situations. But also, I had a female writer, Ryan Lynch, who wrote the script, and we discussed a lot together about what situations to portray. And she also had some input from her husband, who is a founder of his own company that actually got bought after we finished shooting by Adobe, so it was interesting. So yeah, I think she drew some of this experience with the VC founders and stuff because I'm not a founder. I mean, I've never pitched to VC founders or stuff like that. So it came from all of our personal experiences and, you know, what we have heard. I also did a lot of research on studies that have been made on women at work and what are the typical situations they encounter and the biases and all of those. I watch a lot of great spoof videos and that some of them like inverting the roles of men and women to kind of show the ridiculous of it. So it's a bunch of personal experience and existing studies out there but When women watch it, the first reaction is always the same. And the men's reactions are very diverse, but the women's reactions are always the same. It's like, oh my gosh, this is so much my life every day at work. So it's a bit depressing for women to watch it. I mean, it's fun, but we made it to show the status quo, not as an uplifting video to show how we could change things. It might come up in future episodes, if we get to make future episodes. And most men have a very diverse reaction. Some of them, they enjoy so much the VR mechanism that they cannot forget the story. They're like, oh my gosh, this is so great. I can edit my own version of the movie. And some others, they are really taken back by how those males are behaving. And they're like, oh, I wanted to punch them in the face. And some of them are like, yeah, why? So it's all over the spectrum. And this was made to really bring more awareness to those behaviors and how they are possible or accepted in the workplace, where they should not be accepted. So yeah, that's part of where it comes from.
[00:10:48.901] Kent Bye: Yeah, I had the experience of, like I was the editor of the experience where I was deciding what to pay attention to and what to look at. And, you know, I've done some experiments with trying to kind of recreate this feeling of a sleep no more type of environment where You have different action happening in different rooms and you kind of choose where to go and see the action. And my conclusion in my early experiments with that was that sometimes the narrative is just better to just show and you have control and you can make sure what the audience was able to actually see and perceive. And in this case, you're kind of in that line of having to kind of have two parallel stories going on at the same time. And sometimes there's just some boring parts where nothing's happening. So maybe that's an invitation to kind of look at what's happening. And so there's kind of different cues that you have in terms of like maybe it's time to go look and see what's happening in the other scene. How do you approach that? I mean, like, to try to put out the pieces of the information but at the same time it's hard to know as a creator whether or not they actually saw the things that you were trying to point out.
[00:11:59.847] Nathalie Mathe: Well, we tried to make sure, first of all, that each story on each side was standing on its own. So you could watch each one individually and still get a full story from beginning to end. The second thing is we also decided to have cuts at different moments. not to have synchronized cuts except for the first scene and the last scene that are the same length. The other scenes are different lengths, so when one scene ends, you know, it's kind of a suggestion to turn and watch what's happening on the other side. Yeah, there are some things that are silent. We noticed that the sound is really important. I mean, we learned, and we didn't plan everything, to be honest. You know, we just tried to make, like, a really good story and compelling characters and a good edit. and use the sound to suggest when to turn. But I noticed on the coder's side, sometimes there are scenes that are a little more quiet, and when people turn around and they don't hear anything within a few seconds, they just see them coding and we don't know something is going to happen, then they will turn back. So it's not always obvious. to control. It's like, oh gosh, they are going to miss this really important point, you know, when she makes like a secret agreement with the young guy and stuff. But it's okay because at the end, even if people miss half of it, they get the overall story. They get the overall arc and what's going on. So that was the main point. They might miss the subtle elements and this makes people want to watch it a second time. Many people have told us, I need to watch what I missed. So basically it's like playing a lot on the fear of missing out. I knew from the beginning this was kind of an experience that we were doing and maybe it would work, maybe it wouldn't. So we did the best we could and, you know, if we had funding, maybe we could have gone back and reshoot some scenes, but we didn't have these options. But it already works pretty well this way, so not everything was totally planned out. It's like focusing on each individual element, you know, and making it as good as possible. And I didn't want to control, like sometimes, At the beginning, before watching it, people ask me, so how do I know if I should turn? I'm like, it's up to you. You know, it's like in real life, you decide which side you want to follow, which side of the story you want to hear, and whatever you miss, you miss. You know, you have the consequences of your choice. So it's part of the design to be this way, and you miss part of it.
[00:14:34.215] Kent Bye: Yeah, it's interesting to hear that there was a secret pact, because that was one of the things that I had missed, apparently, in the story. But I'm curious to hear from your perspective some of the major points or beats that you were trying to make. Because I got at the end, and I guess at a risk for a spoiler alert for anybody that wants to have the experience, I suggest that you have the experience. and then sort of come back to this to kind of unpack it a little bit more. But I'm curious from your perspective, because what I really got at the end was that there was a moment when the team was thanked and attribution was given, and that in a lot of ways, the woman who did a lot of the work was kind of ignored and wasn't really given proper attribution for the work that she had contributed. And she kind of just walks off. And there's not a moment where the men who are being congratulated are able to really kind of stop and really distribute that attribution more fairly. But what were some of the other big beats or points that you were trying to make within this experience?
[00:15:32.652] Nathalie Mathe: So the underlying theme that we wanted to address in this one was really about when women try to contribute and they don't get heard or they contribute to the success and they don't get that knowledge at the end or their ideas taken by some others and also like some of the different treatment like whenever she suggests solutions, she's kind of stopped or denied up front, versus the younger guy, when he says, hey, I have some idea, can I try it out? The other guy is like, yeah, sure, dude, you know, just have fun. So those are part of the different treatments of women, that they are not trusted up front, usually, they have to prove themselves first, versus guys, they are given the benefits of the doubt, and they are trusted before they prove themselves. So, you know, small elements like that. We also wanted to show the stress, the effect of the stress on the male side. So the boss who is dealing with VCs and investors and he's really nervous. And then, you know, he's getting advice from his wife. So we are also touching on like the wife used to work and now she's staying at home, but she's still very supportive of her husband because she knows what it is to start her own company. She has done it before. and she very much wants to go back to work and so they have to deal with also personal and family issues. So we wanted to show the human side of each of the characters and the female coder, she's also kind of harsh. little very gifted coder and she might not react you know appropriately all the time and that's not the point to show like perfect character like she's pissed off at the end and she walks away and people have told me well why doesn't she stand up and tell them I look guys it's not okay you know it's like that's not fair it's like well this is to open a dialogue this is not to answer questions so If people ask question at the end, yeah, they should ask themselves. Why does she react this way? Why maybe it happens to her many times before maybe not maybe she is just you know That's the way she hasn't learned yet how to deal with the situation. Maybe we don't know so the idea that Each person has their own version of the story that they built as they are watching the experience. The purpose was always to then people who have watched it together could compare and say, hey, you know, I got that out of it. I understood that. How is it different? What did I miss? Or what did you understand differently? And being able to have this dialogue. And that was always the intention of the project. It's not just something that you watch individually, but you can discuss with other people afterwards.
[00:18:18.266] Kent Bye: Yeah, that makes sense and I see how something like Sleep No More where it's like literally impossible to see everything because there's like a hundred rooms and like 23 actors running around and so you can only capture a little glimmer of the stories even though it loops you're able to kind of see like two and a half of the loops. capture a bunch of the fragments that then you have to piece together at the end and it feels like a similar experience in that I could see how if you watch it with other people then you have a discussion then you're able to really kind of unpack it to the next level and maybe get those missed parts and this is I guess you know into the process of like even learning how to watch something like this in a way that feels satisfying to you because you know there's part of me that wants to go back and be a completionist and like Watch the full version from one side and then you know watch it again from the other side because the first time that I did see it it was kind of like this Mixing in the editing and kind of just using my intuition and my gut as to when to kind of turn I was turning quite a bit And I was able, I felt like I was able to get a lot of that story, but it's like this cultivation of like, this is not a good experience for people to see as their first experience because it feels like people kind of like knowing how to watch a 360 video. This is, I think, a little bit more advanced in terms of the storytelling language and grammar that you're using, kind of takes a certain level of sophistication for people to be able to watch. And I'm not sure if you've found that as well.
[00:19:41.797] Nathalie Mathe: Yeah, it's interesting. At the beginning, when we didn't have the right sound mix and everything, and people watched it, they would forget to watch what was in the back. And even people who were used to 360 video, because they would tell me, yeah, but there's never anything happening in the back anyway, so I... And then, yeah, there was another person, it was like her very first time and she was like, oh my gosh, I thought I broke something because there is like another story behind. So yeah, we need like a little tutorial at the beginning or we need to explain people before they watch it. It's different from your usual 360 video. You know, there are two stories happening at once and they are in parallel and they are synchronized and they even like communicate with each other. So yeah, it's a bit more complex than a simple 360 video.
[00:20:32.208] Kent Bye: Yeah, it was kind of a fun thing to discover because I didn't know anything about it and to just kind of like discover that that was what was happening. It was kind of a fun thing for me to find out.
[00:20:41.802] Nathalie Mathe: And people do enjoy the first scene where they figure it out also by themselves and who they are by seeing whatever in the Skype call, they can see their face. By turning around, you can see, you know, you speaking on the other computer and stuff like that. So it's, I ask people, like, do you want me to, like, explain more from the beginning? They were like, no, no, no, it was really great to figure it out on my own, you know. So, yeah, it really depends. The other thing I wanted to say was about the watching experience. We've done some user studies, like recording the sessions of how people watch it, and it's still being analyzed and stuff. We work with independent researchers, but it's fascinating to me to see, like, each person watch it totally differently. And it's more revealing to me about the person who watches it than it is about the film. It's like, I guess, every film. But yeah, some people, they don't want to miss anything and they shift all the time. Some people, they shift, you know, only like a dozen times. Some people, they prefer to watch one side of the story and then the next one a second time so that they don't miss anything or they feel more immersed. So it's really like totally personal and that's what I think is great about this experience is that it gives people the choice to follow their own preferences and yet at the end it also makes them want to see the other's perspective which is like the whole purpose of the experience that you are interested you know even if it's just out of FOMO you want to see the other side of the story most of the time in life we are not interested in hearing the other side of the story. So this mechanism kind of makes people want to see the other side and that's what I ended up really feeling like this was very successful at achieving that.
[00:22:39.218] Kent Bye: Yeah, it reminds me of the experiences that Rose Trochet has done at Sundance back in 2015 and 2016, Perspective Chapter 1 and Chapter 2. The second chapter, she tells a story of two policemen who have this encounter with two young black men, and one of the young black men ends up getting shot. she ended up having four different segments where you kind of see four different perspectives kind of cutting between them and then as you watch all four of the segments you get everybody's perspective of the whole experience, but you're kind of cutting in between them and what I found was really interesting was that depending on the order that you see them it sort of gives you a different flavor of the experience and so it's a little bit more I guess quantized in terms of like the controlling of that narrative and saying that you watch it, but you just have to watch the same thing like over and over and over again, and I guess it's like this idea of situated knowledges such that you know you have many different people who have different perspectives of where they were at in space and time but also their position of power and privilege and And in the perspective chapter two, there's like every character makes a mistake. They're not all flawless. And so it creates this weird moral dilemma where like depending on what your predispositions are in these types of situations, you may actually perceive the situation differently, especially if you only see certain parts that have parts cut out. And then I had the experience of seeing evidence in the last piece that I hadn't seen before that kind of changed my whole perspective of the whole thing. But it was kind of like doled out in a way that I could watch all of them and get a sense of the story. But that had the problem of like how many people were willing to watch the full half hour of all the perspectives to get that full experience. They may have just had one or two of those experiences and thought they got it. And I had the experience of having to see all of them to really get it. So it feels like there's this unique affordance of virtual realities to be able to take these different perspectives. With this experience, you have the challenge of like, if you turn too much and kind of make edits too often, then you lose a lot of the story that's unfolding. And if you never shift, then you miss half the story. So it's like really trying to find that balance to be able to most efficiently jump in between all these different perspectives to kind of get the nuances of what actually happened.
[00:25:01.118] Nathalie Mathe: I wish I had seen, because I've heard of it, the perspective films. I didn't know they had four versions. You could dig deeper and deeper, I guess. We tried to make a design that takes advantage of 360 video. The only thing that you can do in 360 video, and keep in mind, we started more than a year ago, so there was not even a controller at the time for 360 headsets. And so the only thing you can do is already turn around. That's what agency users have. It's not like in a Vive headset or an Oculus where they can move around and do stuff with controllers. So how can we use this only possible action from the user to use and give them two different perspectives? That was kind of the thinking behind the design. And the other piece of thinking is like the sphere has been used in the arts. you know, to integrate multiple perspectives and multiple views. Like, if you look at installations, video installations, where they use, like, four different video walls to integrate, or the dome installations also, where oftentimes they have, like, multiple perspectives. So, like, we didn't invent anything. There is a great tradition of storytellers or, you know, artists who have tried to integrate multiple perspectives into their stories and their work. So, yeah, this was one attempt to do it. differently by giving the viewer the entire editorial choice and not giving him a director's cut so that they make their own choice and they might miss something and that's okay but yeah hopefully in the end they got most out of what's needed out of the story and they get the idea and if they want to watch it a second time they might get to the more subtle points that you know they missed of why she doesn't get credited so
[00:26:56.818] Kent Bye: Yeah, it also reminds me of a piece I saw at Sundance back in 2016, where they did quite a lot of spatialized audio, and where they have, like, you're looking exactly in front of you, and maybe you have a conversation that you're able to hear, but there's also somebody off to the right, maybe 120 degrees away, and when you turn around, you have this whole other spatial audio experience. And so that, to me, showed the power of how much you can do with just audio to be able to do editing. because the video is pretty set. You're not making any sort of dynamic edits to the video. That's already kind of baked. It's that same idea that you implemented here, which was that you were able to have the video pretty much set, but you had an entirely different audio experience. And that audio was the thing that actually allowed you to have that dynamic experience with it.
[00:27:44.490] Nathalie Mathe: And definitely I want to add something about audio. And it's been said before that audio is like 50% of VR experience. But this is like totally true for our experience. Some people have told us, oh my gosh, it's like 80% of the experience. So yeah, definitely we used directional audio. We didn't use the ambisonic format. We used the quad binaural, which is a different format that allows us to have four different mixes in each of the cardinal directions. And those get mixed differently based on where you are looking, where the viewer is looking. And it allowed us to have the full creative control on how the sound gets mixed, depending on what direction you are looking at. We couldn't have done that with ambisonic format. So thanks to the amazing sound guys that we worked with, you know, they have worked at LucasArts and Soundworker studios, so they are like really good at what they do. Yeah, without the sound experience, it wouldn't work at all. And to the point where, you know, you have to think about it, like the sound was not even 10% of the budget of the entire video. And it's like more than 50% of the experience. And it's very powerful. And it's from a human, you know, user interface UX point of view, it's much more powerful than visuals because You don't have to click a button, you don't have to make a conscious action. The sound is more unconscious on the human brain, the way we process sound. So the fact that you can still hear what's going on in the back and decide to shift or not is much less disturbing than if we had done it with some visuals. So sound is really underutilized.
[00:29:33.505] Kent Bye: Yeah, that was really interesting to hear that, because I've seen there's a number of different companies that have done that. Was it, happened to be Dysonix that were doing the sound, or was it somebody else, who was doing the sound?
[00:29:41.997] Nathalie Mathe: This was Sean Farley, we work with them as freelance, Sean Farley and Kevin Bolland, who are both sound editors.
[00:29:50.299] Kent Bye: OK, so just kind of a technical question in terms of where were you putting the quad binaural microphones when people were speaking? Because sometimes when they're speaking, it sounds like they're really behind you. And so I would imagine that, well, maybe if it's just right in front of them, and then you would just use the opposite polarity of the 180 degrees from them as if the ears were facing the opposite direction of them to get that sense of it sounding behind you.
[00:30:16.699] Nathalie Mathe: So in terms of recording, this was all recorded with traditional means, like lavaliers and sound booms, you know, to record the room. Nothing was recorded with binaural microphones or 3D spatial microphone. This was all recreated in post. So they placed the sound, you know, behind or up front or to the right or to the left as they wished. I think they use Pro Tools and some plugins. I'm not a specialist on sound, but basically, yeah, it's all post-production.
[00:30:48.827] Kent Bye: Okay, okay, that makes sense that they were sort of like recreated that quad binaural.
[00:30:53.187] Nathalie Mathe: The same way that they were able to position the voice, you know, you are embodying an actor, so we needed the voice to come from, like, as if I'm speaking from my mouth. I hear it a different way than if the sound is coming from someone external to me. So yeah, we positioned all those sounds so that they felt real, basically.
[00:31:13.737] Kent Bye: Well, you still had to do a lot of long takes, it looked like. Were you actually having them talk in real time to each other while you were shooting the scene? Or did you have to fudge the sound of that later and have it play back and then have them ad lib and fill in the gaps? Because it's always difficult to have the exact timing that you need whenever you're having a conversation like that.
[00:31:34.944] Nathalie Mathe: So yes definitely doing synchronized stories and parts where they communicate really become very tricky. So first of all we tried with the script to write the dialogues to be synchronized you know kind of line by line in front of each other on the page to make sure like they are going to end up speaking at the same time. Then, of course, there is the real-time performance of the actor, which you don't fully control. And we shot one side with the actors on one day, and then the other side on another day. So what we did is, first of all, we had the actors, like, for example, the male actor that was on the male side, be there during the shoot to be in the dialogue. And we didn't record him, but he was talking back to the actors as the actors were being recorded on the coder's side. and vice versa. The second day, we used the recorded dialogues from the first day so that the rhythm of his dialogue would match what has been recorded. So we, you know, we tried to time it and to synchronize it this way. But then when we ended up doing the edit, of course, a lot of stuff didn't match. Or, you know, there were added lengths to the take. And yeah, those were long takes because each scene is at least like one minute. Some scenes are like two or three minutes, so very long. So definitely we had to do some creative edits. Part of the tricks was because it's POV, the main actors are not on screen when you are from their point of view. So you can kind of cut and edit their dialogues and we did some of that as long as it matches their gestures and their hands and their body movements. So we did some of that. And then if we needed a scene to be shorter, that was the tricky part, because we could only cut like the beginning or the end. So on one of the scenes, we had to be more creative, like the scene pitching to the VCs, the boardroom. We like individually editing each of the guys sitting for when they were not moving and to like being able to like scrape a few seconds or a few dozen seconds that we needed. So it was tricky. And then the second part that was tricky was like because it's about coding and computer screens and texting on phones and, you know, Skyping with other people. We had to reinsert all those screens in post. because, of course, we didn't have them when we were shooting, you know, we let the actors do their acting. So, yes, we had to also, like, recreate all those screens, match them to, like, when the actor was typing, when she's typing code on the screen, you know, you see the code going. Create fake windows, fake Skype calls, all of those, and add them back in post, and we worked. So we recreated all those. screenshots and then we worked with Tippett studio which is this great studio in Berkeley who you know they have done all the Jurassic Park and Star Wars movie and they were really interested in doing more 360 video. Phil Tippett had already done a stop-motion 360 short film with WeVR I think. So they did all the compositing of the screens back and rotoscoping some of the hands that got in front of the screen and putting back the reflections and all the compositing stuff. So yeah, way more work than we had anticipated basically. Yeah.
[00:35:06.141] Kent Bye: Yeah. And so for you, what kind of experiences do you want to have in VR?
[00:35:11.323] Nathalie Mathe: Um, so me personally, I'm really interested in creative experiences or experience that push the boundaries of storytelling in interesting ways. But yeah, I'm leaning towards like more interactive. I want to explore how to, have more interactivity, and how do you participate in the story, or how do you create your own world and, you know, invite others to participate. And those tools, it's really tricky because for gaming or for CG, you know, there is Unity, and most people are using it on Unreal. But for video, there are a few apps being developed to allow you to do interactive videos. Doing video in Unity is really not the best solution. So as filmmakers, you know, the tools are being developed as you are trying to do things. And it's almost like I need to invent and develop my own tools if I really want to do what I want to do. As I go along, I would love to have some toolkit. Like, I used to do some generative arts with processing. This is like this real-time coding that you do that creates visual in real times and that you can play with. It's in 2D. A lot of artists use it. There is TouchDesigner for real-time graphics as well that VG artists are using. But it's like, where is that in VR to be able to use that? Tilt Brush is great, but yeah, I'm looking around for tools that I could use to tell more interactive stories. And can I touch a little bit on the production funding side of the project? Do we have more time? Yeah, sure.
[00:36:46.242] Kent Bye: What do you want to say about the funding for projects like this?
[00:36:49.308] Nathalie Mathe: So yeah, many people tell us, wow, you know, it's really high production value. You guys did like an amazing job, the quality of the image and the sound and everything. And yeah, it's very professional. It was done by professional, but. I want to insist that this is an entirely self-funded project. Bootstrapped, you know, there was no funding. We couldn't get any funding when we looked for it. And we got some in-kind support for cameras, from Freedom 360, for locations, for office space, you know, and some... Everyone who contributed, we had lower day rates than normal. We paid all the crews and everyone. The three creatives, me, Justin and Ryan, didn't get paid at all. Sometimes for a year, sometimes for several months that each one of us worked on this project. And part of it, you know, I'm trying to understand like how do we get funding for indie projects and if you are not a well-known director, if you don't have a big actor, if it's not a well-known IP that the studios want to fund. Creating content and creating media is very different from starting a tech company. It's not about VC funding, you know, you don't get VC funding to create content unless you are developing some specific tech that goes with it. So it's more about the traditional film funding model, like how do indie filmmakers do it? How is the traditional media, Hollywood, you know, etc. doing it? And the grants for that are lagging behind. Most of the grants are not for VR. There are a few labs, festivals that are doing labs, helping VR filmmakers. There are very few of those. And it's also all about who you know and being in the right network to find the support that you need. So right now we are still short on funding to be able to distribute it online and looking for distribution deals and being able to put it out there. It's been through great festivals so far. We had the San Francisco Film Festival, the Random Film Festival in London, the FIFA in Toronto. We won an award and the New York Jump into VR Fest. So it's been really well received and it's been getting some traction, but it's been a lot of personal push and efforts to get it out there, to have people talk about it. So please, like our Facebook page, it's You Turn VR, and talk about us and support us any way you can.
[00:39:15.892] Kent Bye: Yeah, I just went to Rene Penel's, he does the Kaleidoscope VR and he had a first look market where he traveled around the world to 30 different cities and started meeting a lot of independent filmmakers and he brought together a bunch of them to try to do this matchmaking type of process with independent creators that he had met along the way, but also these funders who were interested in funding. And for me, I've been seeing the kind of the ecosystem of virtual reality is like three legs, but there's actually like a fourth leg. The first leg is the technology that people are able to actually create the different experiences. Then the content creators have to understand the technology enough to be able to actually create that content. And then eventually it gets to the audience who has to learn how to watch this content. But, you know, that fourth leg is the distribution and the funding to be able to fund it in the first place. And there are a number of really big players, whether it's Samsung and YouTube and Facebook. So they each have had their own strategy in terms of how they have the pipeline for funding the different types of content that's out there. But it is like difficult beyond that because there's not a lot of other, you know, overseas and other countries, they have some governmental grants that are coming through. but within the United States arts funding is very limited and anybody that is sort of providing funding is also looking at the economic return and then there's not necessarily a distribution platform that has a monetization strategy that is able to get that clear return on the investment. So I see this challenge of like the big major players that you know Samsung, YouTube and Facebook haven't necessarily created a monetization strategy to make that work. And there's other sort of platforms, whether it's Within or Weaver and Jaunt VR. They each have their own kind of way that they may be starting to do that, but I don't see the major players trying to figure out a way to create a marketplace for people to be able to buy the content, because they're still just trying to get the content out there. So that's kind of what I see. I don't know if you have thoughts about
[00:41:21.727] Nathalie Mathe: Monetization is one thing, but for now I'm not even expecting people to be willing to pay any money to watch those videos. So the other part of it is investment in terms of acquisition, being able to buy the content to put on their platforms. And so far, those are not traditional media companies. have the same practices as they are used to deal with like game developers or you know tech companies and Dealing with content creator is a bit different. You don't like sign the same contract, you know, you you sell usually territories and and limited duration and all those kind of stuff and the amount of money that they are willing to buy content for is very low right now. So it's the question of how do we make it so that content creators can survive those first few years, you know, until there is enough content out there that the public want to see that maybe even like pay for it on a monthly basis or something else. So yeah, it's like this bootstrapping is kind of difficult, I think. And the tech companies have to figure out how to work with the media and the content creators to make it like a sustainable environment for original content to be created, not just create content that we already know about.
[00:42:42.428] Kent Bye: Yeah, it's an issue that I've had other discussions with people in the VR industry, and I think it's probably one of the biggest challenges, especially for 360 filmmakers, because it's different when you can sell a volumetric experience on Steam, because then there's at least some alternative marketplaces that are out there, but just for storytelling in general, I feel like there's a lot that's holding back content creators to be able to dive full in, especially when they can't find a way to pay the bills and make a living doing it.
[00:43:12.533] Nathalie Mathe: So one thing we are looking at right now, we have been approached by different consultants who do like development training and learning to use our U-Turn experience as a basis to develop like workshops on diversity training and leadership training and to use that to change the culture of the companies that they work for. And another venue is like doing university screenings. We are going to do our first university screening with 120 students from different departments in March for Women's Day. But you know, those are small income sources. It's more like you sell the licensing rights of your film to those different partners. But this is great because this is like the audience that we are targeting in a way, you know, we want it out there to most people as possible, but university and tech companies are also the right audience too. So if we need partners to bring it out to those audience, it's not just putting it on a public platform, it's also bringing it to the right context that people are going to watch it and discuss it. So you have to be basically very creative when you do VR. Creative in terms of the technology, creative in terms of the storytelling, and creative in terms of the funding. And to really, yeah, you have to invent the model on every level. That's kind of my conclusion.
[00:44:35.008] Kent Bye: Wow. Yeah, definitely. And finally, what do you think is kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:44:50.241] Nathalie Mathe: That's a big question. For me, it's about being able to let other people experience life as someone else, in a different situation, in a different world, in someone else's shoes. Really, it's kind of an out-of-body experience, you know. It's like, yeah, being able to experience different point of views, different stories of something that you couldn't if you were just being yourself. And it can be like artistic experience, it can be documentary experiences, it can be like gaming experience, it can be something totally new that we haven't invented yet. I'm not so big on social VR personally, but, you know, who knows, yeah.
[00:45:36.362] Kent Bye: And is there anything else left unsaid that you'd like to say?
[00:45:39.946] Nathalie Mathe: I want to say it's been a great learning experience. I jumped into that almost two years ago and many times it's been feeling like jumping out of an airplane without a parachute and having to knit it as I'm falling along. But I did find great collaborators and great people to work with who shared the vision, even if ultimately I also understood that it's my own project and no one else is going to carry it for me. So I have to be there and do the work and not give up. Many times I was ready to give up. So I would really want to encourage other people, no matter their starting level of their skills, to jump into VR. and be inventive and creative and do what they can. And maybe they don't do a project as ambitious as what we did, and they start with a little project. But we need people like that. We need more people who come from very diverse backgrounds to create in VR and participate.
[00:46:44.402] Kent Bye: Awesome. Well, thank you so much.
[00:46:46.243] Nathalie Mathe: Thank you.
[00:46:47.757] Kent Bye: So that was Natalie Mette. She's the co-director and founder of the production company called Native VR and the experience was called U-Turn. And I first saw it at the Oculus Connect, and it was just an experience that really stuck with me. It was an experience that when I watched it, it took me maybe 30 to 40 seconds to really figure out what was happening with the audio mix. It was just so seamless. But it was something that was just so powerful to be able to be blending between these two audio scapes. And this is a concept that Betty Moeller actually talked about a little bit in terms of being able to share embodiments into two different physical spaces. Once you've been able to have that experience of feeling like you're actually walking between two different worlds with two different parallel storylines happening at the same time, it's quite a surreal experience and I really enjoyed it. I think it's a really powerful affordance of the virtual reality medium and I expect to see a lot more experiments with this type of storytelling mechanism. So it's kind of equivalent to some of the immersive theater experiences where there's parallel storylines that are happening at the same time. And you kind of have to live edit it by making choices and deciding what to pay attention to. And you're going to miss about half the story because there's things that are happening on the other side of the things you can't. necessarily really even paying attention to. So even as Natalie was going through and recounting some different plot points within the experience, there are things that I just simply missed. And so I think one of the powerful potential applications for an experience like this would be to watch it with a group of other people all at the same time. You come out and then you talk about it and you talk about what you saw, what you didn't see, and then maybe even you may have witnessed something but you may have not contextualized or picked up on it. And I think that's the really interesting thing is that Right now, dealing with unconscious bias is something that is probably one of the biggest open topics that is being discussed both in Hollywood as well as in the Silicon Valley and the tech industry, which is that there's certain things that men just can't see. And there's also things that women can't see. There's things that all of us can't see based upon our privilege and power and position of where we're at in the society. But in specific to men and women, I think this comes up a lot in technology where there's a lot of unconscious bias that is just kind of embedded within our worldviews and there's behavior that comes out that is difficult to really name and isolate. This is a narrative experience and so I didn't get the chance to see every different dimension of these stories because I was kind of hopping in and out fairly frequently so I was getting kind of a dose of the story and the side effect of that is that I kind of maybe missed some of the more nuanced dynamics that were happening between these different people. I think that's one of the things about virtual reality as a medium is that you start to really try to explore the group dynamics of a situation because you're able to kind of have that full immersive experience and to see where people are at relative in the space and you'll be able to actually look at different people's facial expressions and reactions as they're relating to each other. And so just like on Skype, you might be able to do a one-on-one conversation, but once you start to get into these group situations and group dynamics, it's really actually hard to have that type of group dynamic play out in a 2D medium. And given the spatial medium with spatialized audio, you start to pick up on these group dynamics a lot more. And so I think that the virtual reality medium is going to be able to start to explore these types of issues of unconscious bias within the workplace. And you can definitely see how this could be used as a teaching device if it was given the right instruction or if it was done within a group context and maybe watch it once or twice. If there's specific points that you're able to kind of pull out and then facilitate a group discussion. I just got back from the virtual reality strategy conference and this was basically all the enterprise people that are trying to figure out how to make money within virtual reality. One aspect of that is, you know, some of the people just are trying to find out the applications for VR within the enterprise. And one of those huge applications is in training. After going to the VR strategy, I'm totally convinced that there's enough of a corporate interest in the enterprise market that's going to kind of help bootstrap the entire VR industry. Not quite sure if it's going to come completely from games. I think it's going to come a lot from the business side, especially with the standalone headsets. There's just so many more opportunities when it comes from training using these standalone headsets than I think the gaming affordances aren't as compelling. They're nice and helpful, but the downgrade in the graphics, I just have questions in terms of whether or not the PC VR is just going to be a much more compelling gaming experience from some of the most hardcore gamers. And so Natalie had mentioned that she was thinking about going on tour with this type of experience to show it to people and then kind of facilitate these different group discussions. And I think that this is actually a great distribution idea as long as you start to get connected to those types of companies that would want to facilitate that type of experience. I think that it's going to be quite an interesting approach to try to do some of these diversity trainings or be able to train around discrimination and harassment and maybe, you know, give some of these immersive experiences and then provide an opportunity for people to talk about it. The other thing is just the funding issue, which is that for independent creators, it's kind of a dire situation for a number of people. I mean, there's just a handful of funders that are out there that are really funding this type of content and you really have to kind of be scrappy and be able to pull together the resources to be able to create these different types of projects and Natalie was amazingly able to come up with a budget herself and bootstrap this project and the way that it was executed the technical execution and the sound design I think it's a really powerful VR piece and I think it has a strong message and just the The sound design and that experience of kind of stepping in between two different worlds is just something that I think is going to be a huge innovation within the medium of virtual reality. So that's all I have for today. And I just wanted to thank you for listening to the Voices of VR podcast. And I am about to go to the PatreonCon. So I'm going to be talking to the creators of the Patreon website and kind of doing a two-day teach-in to kind of learn all the best practices for how to really run a great Patreon. So I'm hoping to learn a lot there. If you want to support my work and what I'm doing here, then please do become a member to my Patreon. You can donate today at patreon.com slash Voices of VR. Thanks for listening.
