I had the chance to catch up with Nicole Relyea, the community manager & VR evangelist at Jaunt VR at the Silicon Valley Virtual Reality Conference. Jaunt has produced a number of 360-degree video experiences that they were showing on the Google Cardboard, Gear VR and Oculus Rift at SVVRCon, and Nicole talks about some of those experiences and which ones have been the most popular.
One interesting thing that Nicole mentioned is that the release of Google Cardboard actually changed their product timeline planning. They were not expecting a consumer release of VR until the end of 2015 or early 2016, but the proliferation of Google Cardboard as a minimum viable VR headset meant that there is actually a demand for video content that’s available to be served now.
Nicole also talks about the opening of Jaunt Studios and what that will mean for learning more about the language of VR and how to best tell stories within it. She also talks about some of the lessons they’ve learned about the language of VR storytelling, and some of the open problems and challenges that 360-degree video producers will be facing.
One big announcement that Janut had after SVVRCon and after this interview was their Neo digital lightfield camera. This is a radical technological leap from their previous approach of stitching together video spheres from an array of cameras. There isn’t a commercial digital lightfield camera designed specifically for VR that’s on the market yet, and Jaunt VR hopes that Neo will be the first one.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:11.974] Nicole Relyea: My name's Nicole Relier, and I'm the community manager for Jaunt VR, which means that it's my job to get out and interact with the VR community, as well as managing a lot of our social media and interacting with people online.
[00:00:24.357] Kent Bye: Great. And so tell me a bit about what's happening with Jaunt. It's been a while since I've checked in with what's happening with both the technology and some of the content and what's happening with you guys.
[00:00:33.389] Nicole Relyea: Sure. So we've released about 11 pieces of content now. They're available on the Google Play Store for Android. They're also available on our website. If you go to our content page, you can download it for Rift. And then we do have one app out now in the Apple Store. So we have Paul McCartney available. That just happened a couple weeks ago. Our biggest news right now is that we've announced that we're going to be launching Jaunt Studios in Los Angeles. The concept there being that we really want to have greater interaction with the creative community and give them the opportunity to access our pipeline and our tools. So that will be our big thing for the summer is getting that up and running.
[00:01:08.518] Kent Bye: Nice. And so what is the intent there is to help cultivate existing directors or to have a lab to experiment as to what the language of 360 degree video is or what sort of the mandate for this new Jaunt Studios?
[00:01:23.046] Nicole Relyea: The idea behind Jaunt Studios is basically sort of be what our CTO keeps calling a genius bar, quote-unquote, for VR experiences. The idea being that whether it's a creative or a brand or an independent filmmaker, people can come to us with their ideas and we can work together in developing it. I'm sure over the course of the summer we'll get more outline of what exactly that business model is, who exactly we're going to be working for, but we're just really excited about having the opportunity to make a lot of really compelling, high-quality VR experiences.
[00:01:53.174] Kent Bye: And so yeah, in terms of what's compelling and also comfortable, what have you found has been sort of like the most compelling and comfortable 360 degree video that you've seen so far?
[00:02:02.854] Nicole Relyea: You know, it definitely varies by personal taste, but we get the strongest reactions to Black Mass, which is our horror short that we did with New Deal Studios. And I think it's because that one puts you in a first-person role, where the characters are sort of interacting with you in the film. That's the one that we find has the greatest amount of immersion. We've done some experiment with moving the camera. Of course, there's always the motion sickness concern. But in the piece that we did with the North Face, we experimented with putting the camera on a drone. And what we found is using a horizon lock to stabilize the horizon really made the motion more palatable for most people. And that one's pretty cool. You get to feel like a bird flying over a canyon watching people skydive and things like that. So it's really neat.
[00:02:44.316] Kent Bye: Yeah, I personally get pretty susceptible to any sort of motion, so unless it's totally, like, stabilized or consistent velocity with no acceleration, then otherwise, for me, it starts to make me feel a little queasy. But it begs the question of, like, how to compose between scenes and cutting, fading to black. And if you see that there's some best practices that are emerging, or if it's still kind of like the Wild West in terms of how to really construct a video that has more than just one take,
[00:03:12.892] Nicole Relyea: So there's still a lot of learning happening there, for sure. We're constantly conducting experiments. There's tons of things that we filmed that didn't work out for one reason or another. We got it back and went, oh, well, we learned something from that experience. And I do think that we will be putting together some best practices and knowledge bases and things like that in the coming months, because we would really like to see some of those standards get set and be able to help everybody else accelerate the speed at which we're learning from everyone's mistakes. I think there's some basic ones like the Oculus of fading to black instead of fading to white. We had one video that did that and went, oh, that's why you don't do that because it blinds people. It's interesting because we've experimented with cuts and there are certain ways they work and certain ways they don't. In our Paul McCartney video, we've got two cameras and we cut between them. And people really like that there's multiple views, but a lot of people find it very disorienting that, like, a firework goes off and suddenly you're in a different place. So, you know, I personally think that ultimately giving people control to be able to cut between views is good. Like you were saying with the motion, figuring out, like, the horizon lock and, you know, no acceleration and steady speed. We also keep that at, like, fairly short pieces. But I think that there's a lot more to learn, and I think we're going to continue learning and growing and exploring. especially with the storytelling front, and I think one of the most exciting things about VR is it's not just about figuring out how to take traditional cinematic tools, right, and translate them into VR, but we're actually gonna find entirely new ways to tell stories and entirely new methods of cutting and fading and all of those tools that traditional filmmakers use, we're gonna come up with our whole own vocabulary, and that's gonna be really cool and exciting.
[00:04:49.653] Kent Bye: And in terms of the tool set, are you using standard non-linear film editing tools and just sort of having a specific type of format that's available? Or is there a special software that people need in order to kind of do their own custom stitching with the jaunt cameras?
[00:05:05.673] Nicole Relyea: So we have proprietary software to do the stitching and render the video once it comes out of the camera. But then once you get it in that format, you can use traditional editing tools over it. So I know that our production team goes back and forth between proprietary and custom stuff, and then sort of off-the-shelf tools that everyone uses.
[00:05:23.203] Kent Bye: I see. So you would perhaps upload it to the cloud or something, and it would automatically stitch it all together. And then you would have a high-resolution video file that then you would have to then edit with?
[00:05:33.761] Nicole Relyea: Yep, that's the goal. We're trying to build a tool set that can be accessible and useful for everybody. It's something that, you know, you don't have to have our team walk you through to do it. But it's definitely a work in progress.
[00:05:44.392] Kent Bye: And in terms of audio, I know that a lot of people that are in the 360 degree video space have said that audio can be just as important as the visuals. So to create that sense of immersion to have 360 degree audio, perhaps binaural, and to be able to have it head tracked. And so what type of microphone solution would be potentially built into Jaunt?
[00:06:05.323] Nicole Relyea: So we actually use a third-party mic, and I forget what the brand is, but it's a tetrahedral mic, so it picks up audio in all directions. And we're very much of the train of thought that the audio experience is probably half of the immersive experience. We think it's really important. That's why we've partnered with Dolby Atmos. So we've got object-based sound. A lot of our pieces have been remixed now in Atmos, and it's excellent. It's really cool. And it also gives you more of the ability to use audio cues to guide the viewer where to look. Because it's very accurate. And it gives you a very clear sense of it. So yeah, we're totally on board with the audio stuff. And I think that that's really important. But we're not designing our own mic, because there's no need to. There's great tools out there.
[00:06:45.508] Kent Bye: And Atmos, is that a tool from Dolby? Or what do you do in that tool? And what do you put in, and what do you get out?
[00:06:51.019] Nicole Relyea: It's a good question for the audio engineers. I just know that it's object-based sound, so the idea is that, you know, if you hear a phone ringing in your left ear and then you go look at it, now you're going to see a phone and it's going to be in front of you, and the sound shifts that way. One of the funniest things that I see, actually, is that people are really used to movies where you hear a sound behind you, but in a traditional movie theater, there's nothing to look at behind you. So sometimes now I'll see people where, like, you'll hear an audio cue and people don't turn to look because they're still used to the more traditional cinematic experience. So, and you know, I think this goes back to some of your earlier questions, too. I think a lot of the learning is going to be in the VR world, but some of it's also around the viewers. And as people have become more accustomed to these scenes and, like, recognizing that, oh, when I hear a sound, I should turn and look now, even though I wouldn't in a traditional movie. So I think there's going to be a lot of interesting things that happen where there's going to be this nice interplay between our audio and our skill sets evolving, and then the viewers really learning how to watch VR and how to really enjoy those experiences.
[00:07:50.310] Kent Bye: Yeah, and I've done my own sort of experiment with, like, multi-threaded narrative, and I've shown it to people, and, you know, I've got, like, three different rooms, and they can choose between them at any given point, but yet, what most people do is they choose a room and they don't switch. There's something that, you're right, that we have to kind of be trained to be like, oh, this is, I'm not really engaged with this, I'm gonna switch to something else, because there's never really been that ability to kind of make a choice and use your agency within a narrative. But that sort of speaks to, like, both the training of, like, yeah, eventually we'll have to just, like, get used to these, like, new VR experiences of actually how to watch them properly. But also, I'm curious if Jaunt has been experimenting or thinking about if you're just going to be doing straight narratives, or if there's going to be some sort of, like, agency or interactivity within these sort of 360-degree video experiences, or if that's even... possible within the constraints of the output that you're putting in the movie file if that needs to be starting to be used like something like Unity in order to bring that interactivity to the experience?
[00:08:49.826] Nicole Relyea: I think initially it's going to be the cinematic experience. You know, and a lot of those are narratives. Some of them are also just like brand experiences or I don't even want to say brand experiences, just, you know, interesting things like being out in nature or seeing something that you couldn't see otherwise. I think interactivity is definitely a possibility down the road. Whether or not it's something we're going to get into is yet to be seen. I also always think about the fact that this is such early days for this technology. You know, I like to say this is the Pong version of this technology. And so there are probably going to be things that evolve in the space that we can't even imagine right now. So, you know, it's hard for me right now to look at the technology we have and say, oh yeah, we'll totally put interactivity in there and you'll be able to move through the space because that would be incredibly difficult with what we have now. That could totally change two years down the road, you know, some other technology could develop, we could develop something to figure that out. You know, I think we would like to get a little bit of, even if it's a minimal amount of positional tracking in there, I think it would be really cool to give people agency and interactivity. It's not something that's on our immediate roadmap, but I would honestly be surprised if it doesn't start happening down the road. But yeah, going back to the idea of training viewers, it is funny. One of our early pieces was like a sci-fi movie. And there's a scene where this woman runs in and she goes, look, behind you, it's Eric. And she's pointing. And if you turn around, you see Eric getting attacked by an alien. So few people turned around. They would sit there and stare at the girl pointing and wait to see what happens. And it was just so funny to me that even if I could put them through the North Space experience or put them through other experiences, and they're looking all around, the second they got to a movie plot, they started looking straight ahead. And it's going to be an interesting thing to shift and change, but it'll be really cool once people get used to it.
[00:10:30.558] Kent Bye: And so we're here at SVVRCon, and you've just been showing a number of different experiences to people. What type of videos were you showing with people in the Gear VR here at the SVVRCon?
[00:10:42.008] Nicole Relyea: So we had a real variety. We were showing things on the Rift, on the Gear, on mobile phones. We had a little lounge experience. One of the things we're really trying to let people know is that VR doesn't have to be this thing where you have this big expensive buy-in and need all this special equipment. It's something that you can download these apps on your phone and sit on your couch at home and enjoy. As far as the videos that we were showing, we put together a nice little reel where we had the North Face experience that I've mentioned a couple times, which is our first experience where we mounted the camera on a drone. So that was a really cool thing to show and people really liked that one. I think something about the sweeping vistas, the beautiful scenery, and really getting to feel like a bird is really cool. The second piece we were showing was called Earth Encounter, and it's one that we did with the guys from Stress Level Zero, and it was our first live-action CGI mix, which was really great, and it's a fun animation. There's a little alien DJ that comes down to Earth and gets in a fight with the police. And that's how the world ends. And then we had Black Mass, which is actually one of our older experiences, but like I was saying earlier, it continues to get the most visceral reaction, and people really, really love it. I just think it's awesome that we were able to create something a while back, before a lot of these lessons were learned, that continues to be so compelling. And then the final piece we were showing was Big Sean in concert. So we, around South by Southwest, so mid-March, announced a partnership with Revolt TV. So we're going to be doing a series of recordings of Revolt musicians and that kind of thing. And so it's just sort of fun to show that. And I think the concert experience is one of the things that works really well in VR. I think in part because you're used to buying a ticket and having your seat and your single spot from which you look all around, so it translates well. But also just, you know, concerts are exciting, they're fun, they're visual, they're, you know, obviously there's a lot of audio. So, yeah, we had a nice little variety and it was really cool to see everybody's responses and I was like asking people which one was their favorite and it really depends on the viewer.
[00:12:30.799] Kent Bye: What was the black mesh or what happens in that experience again that you said that people really resonate a lot with?
[00:12:36.328] Nicole Relyea: So the Black Mass is a horror short where you are first person. So essentially you've been kidnapped and you're waking up and finding yourself in sort of, it's a little bit of a stereotypical trope where it's, you know, like Satan worshippers and there's blood on the walls and you're in a garage and there's a doll that turns into a little girl that turns back into a doll kind of thing. It's absolutely creepy. I'm not even a horror movie person myself and I watched it once and I was like, that was amazing. I never want to watch it again. But people really like that one. People really get into it.
[00:13:08.438] Kent Bye: And one of the things that I see on the Jaunt Twitter account every now and again is like, here's a Jaunt camera at a big sporting event. Are we going to start to see more like either recorded sports, you know, NBA, boxing, you know, having ringside seats and stuff like that. So is that something that you guys are still pursuing?
[00:13:26.190] Nicole Relyea: It is something we're working on. Of course, the world of professional sports is a little challenging. You know, there's a lot of people that have to give us the okay to distribute things and, you know, to do the recordings in the first place, and then we have to make sure the footage looks good, and then we have to figure out how to distribute it. So, you know, there's a lot of that stuff in the works. I'm not sure which thing is going to come to fruition first. But we definitely think that sports is a major application of this technology. So I don't have anything on the calendar right now of when exactly those things are going to be released. What I can say is that you will continue to see new content coming out, a couple of pieces every few months for the indefinite future. Our goal this year is really to ramp up content production. The biggest thing is that we really recognize that for VR to really, really take off, there needs to be a critical mass of compelling experiences so that people will go out and buy these viewers and have a reason to put it on over and over and over. That's not going to happen with 10 or 12 decent pieces of content. You're going to need hundreds if not thousands of interesting things. for a diverse audience. So that's, I think, where you're seeing us put out a real variety of things, like a nature experience followed by a concert, followed by a horror movie, followed by maybe a sporting event. We're really trying to get that field populated and help this whole economy take off around VR.
[00:14:38.739] Kent Bye: And now that there's been launch windows for some of the major virtual reality HMDs, does that mean that Jaunt is also trying to figure out when your public release of some of your technology is to try to get that before some of those HMDs are released so that people can create content? Or is that something that you can speak about at all in terms of when can we start to see this technology more widely available for content producers?
[00:15:02.935] Nicole Relyea: You know, those things are always a factor. I like to think of the whole VR ecosystem, and you've got the content creators, and you've got the technology people, and you've got the HMDs, and we all definitely have impact on each other, right? And there's definitely an interplay there. When Google Cardboard came out last summer, and we realized that it wasn't just a joke, it was actually something that people could really use, that totally changed our timeline. Because we were sitting there thinking, like, oh, it's going to be a couple years before Riffs are out to the market. So it's going to be a long time before consumers have these. Suddenly it was like, oh, if we put an iPhone app out tomorrow, somebody can go download it on their phone and put it in a cardboard box right now. That definitely changed things. As far as our technology being available, you know, we don't have a dead set timeline right now. I will say that with the launch of Jaunt Studios, which we expect to be, we've announced the launch, we expect it to be up and running later this year. That's definitely going to give people more access. And of course, we're using our prototype cameras right now. We're currently creating content. The timeline for how it gets to the public is a multi-faceted thing. There's a lot of variables there. But we're hoping to start getting those things out to people sooner than later.
[00:16:08.041] Kent Bye: So what do you want to experience in VR then?
[00:16:10.702] Nicole Relyea: Oh, what do I want to experience? I think there's some really interesting opportunity for some really artistic things to happen. So it's really cool that you can go to a concert, experience something you've never experienced. It's really cool that you can go back and revisit something. You know, I was at the Paul McCartney concert and it's neat to be able to remember being there with my friends and yet go view it from a better perspective. That was a real aha moment for me with the technology. What I'm looking forward to is when we get to the point where you can do interesting theatrical things and really start blending some of the live action with some of the CG. whether it's creating fantasy worlds or something like that. I think there's some really fascinating stuff there. I also really like a lot of the therapeutic uses that people are starting to get into and can't wait for VR to start really having an impact for social good on the world. Seeing things like speech therapists using this to help people practice speaking in front of audiences or there's a lot of guided meditation things out there that I think could get really interesting. So, you know, I think as far as the thing I'm most excited about, I think it's going to be creating some of those like fantastical worlds or hyper real worlds. I think that's really cool. I also really love the social VR stuff that's coming up and I would really like to see some of like that blend with the video thing. You know, imagine if you could feel like you're at the Paul McCartney concert but turn and have an avatar of a friend that you can interact with. I think that's going to be a really exciting thing once it all comes together too.
[00:17:35.900] Kent Bye: Awesome. And finally, what do you see as the ultimate potential of virtual reality and what it might be able to enable?
[00:17:42.845] Nicole Relyea: You know, I was speaking a minute ago about the social good impact, and I think that entertainment is fun and exciting, and it's kind of the low-hanging fruit of the field, but I really do think that there's a huge opportunity here for people to understand other cultures better. You know, we've seen research about, like Jeremy Bailenson's project around environmentalism, and seeing people that went through a VR experience of cutting down a tree, it actually concretely impacted their behavior and so I think there's a huge potential there for it. But, you know, the real answer to that question is I don't think we can even imagine it yet. It's kind of like, you know, that old joke when you say, if you asked people in the horse and buggy days what would help them, they'd say a faster horse, they wouldn't say a car. I don't think we know what the car is yet for virtual reality. I think there's things that we haven't even conceived of. It'll be really interesting to see what comes up and how it continues to evolve.
[00:18:36.663] Kent Bye: Awesome. Well, thank you so much.
[00:18:38.064] Nicole Relyea: Cool.
[00:18:38.364] Kent Bye: Thank you. And thank you for listening. If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash Voices of VR.
