Jens Christensen is the CEO and co-founder of Jaunt VR, which is a VR startup that has raised over $35 million to develop the toolchain for live-action, 360-degree spherical, 3D video. Jaunt’s technology combines “computational photography, statistical algorithms, massively parallel processing, cutting-edge hardware and virtual reality.
Jens talks about Jaunt VR’s mission to capture the world around you, and to be able to put that within a VR experience. It requires capturing a ton of video data, and they’ve developed a parallel processing data farm process to be able to stitch together a fully spherical video. They’re also developing a method for capturing ambisonic 3D audio.
The Jaunt VR Twitter account has been posting various photos of covering a wide range of different sporting events, and Jens says that Jaunt is very interested in eventually providing a live broadcasts of immersive content. Their primary focus in the beginning are sports events, music events, as well as experiments with narrative content ranging from a horror story to a story set in World War II. They’ve haven’t released any VR content to the public yet, but are looking forward to releasing some experiences here soon.
Jens said that they’re still working on there is no full positional tracking yet and at the moment it’s just rotationally tracked, but it’s something they’re still looking into. Storytelling within VR is going to require the developing of a new language and syntax, and that in the beginning they prefer to keep the camera steady to minimize motion sickness.
Jaunt VR is also more interested in leveraging the existing video post-production toolchain for editing, compositing and color correction. It sounds like they’re likely working on some plug-ins or video players to be able to directly integrate and preview VR content while you’re editing it, but Jens didn’t provide any specifics and said that it’s still an open problem. Finally, he sees that VR is going to change a lot of industries ranging from news, music and travel and he’s looking forward to helping play a small part with the work that Jaunt VR is doing.
Theme music: “Fatality” by Tigoolio
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:11.977] Jens Christensen: I'm Jens Christensen. I'm a co-founder and CEO of Jaunt. And we're basically working on cinematic VR, as opposed to gaming VR. And the whole idea is to capture the world around you in immersive 360, 3D, both video and audio. And then to basically deliver that to people in a VR headset.
[00:00:33.351] Kent Bye: I see. And so is this a stereoscopic or is it like spherical video?
[00:00:37.433] Jens Christensen: It's a stereoscopic spherical video. So it's both.
[00:00:41.134] Kent Bye: OK. Great. And so I guess one of the questions that comes up is file sizes and how much data does it take to actually capture and then what can it be compressed down to?
[00:00:51.648] Jens Christensen: So we capture a ton of data. So we have many cameras pointing in all directions and can have just more data than we know what to do with. And what we do is we crunch that in our render farm and basically from that create essentially a sphere for your left eye and a sphere for your right eye. And once that's all processed, the file sizes are actually quite reasonable. We can compress them down into an MP4 file if you want. There's also the audio, which is obviously much smaller in terms of size, but we support multiple channels because we actually deliver 360 audio as well.
[00:01:28.192] Kent Bye: Wow, and so how many channels is it? Like four nodes when you're looking around that you're crossfading in between? Or how do you actually do the omni-directional audio in that way?
[00:01:36.670] Jens Christensen: Right, exactly. They're actually four, basically are using something called ambisonic sound that uses basically four channels, if you will, to record that sound.
[00:01:47.323] Kent Bye: And so maybe talk a bit about some of your early projects and proof of concepts of doing this type of cinematic video projects within virtual reality.
[00:01:56.577] Jens Christensen: Yeah, so we're doing a number of things. We're doing some things that involve music, which are fantastic. We have not actually released content yet, so that's something that's going to be coming soon. So stay tuned for that. But in general, the areas we have been recording in are sports, music. There's also some narrative content that we've been partnering with people on. One of them is a horror story. Another one is a World War II piece called The Mission. We've been working with New Deal Studios on. So really a wide variety of content that we've been working on.
[00:02:29.840] Kent Bye: Yeah, you've been tweeting out through your Twitter channel on Jont VR some of the places that you have cameras set up. And I think it's exciting is that even though an event may already be over, you're able to sort of go there and have a front row seat for some of these places. So maybe you could talk a bit about some of these sports events or other things that you have set up some of your cameras at.
[00:02:52.118] Jens Christensen: Yeah, so it's a wide variety from some racing events to boxing events to basketball events, all kinds of different events that we have shot video at. And again, it's not something that's been released yet, but it's something that we're looking forward to sharing with the public.
[00:03:08.309] Kent Bye: And I guess one of the challenges with spherical video is being able to have positional tracking with the DK2, and so have you been able to get positional tracking working in addition to having a single location of a camera?
[00:03:22.948] Jens Christensen: No, we don't do any positional tracking, just rotational tracking.
[00:03:26.873] Kent Bye: And so is that something that you know with that you just turn off the IR tracker and the dk2 when you're watching a cinematic video then or just something that people can't lean left or right or is that too hard of a problem?
[00:03:39.245] Jens Christensen: Well, it's something that we will be addressing as that's not something that we have addressed yet I see
[00:03:44.620] Kent Bye: So, yeah, I guess how you see this unfolding going forward in terms of, you know, building partnerships and providing the content that's out there that's compelling enough for people to want to get into virtual reality and experience some of these narratives?
[00:03:58.325] Jens Christensen: Yeah, so basically we are an engineering company, so we're really focused on the technology and building out the camera, the software, the players that it takes to actually consume all of this in. And so we're actively looking to partner with the creative community. and are partnering with the creative community who have a vision for, you know, fantastic, immersive VR experiences that we can help enable and make possible through the technology that we're developing. So we really kind of see ourselves as the technology partner to the creative community.
[00:04:28.426] Kent Bye: I see. And is there ever like a live component for what you're doing at JumpVR? Or do you think it's always going to be sort of more of a captured, produced content?
[00:04:36.633] Jens Christensen: Well, for now, it's captured and produced, but I'm very excited about live. We're really looking forward to providing live VR content. I think that adds a whole extra dimension. If you think about things like news and sports, it's fantastic to be able to do live.
[00:04:50.686] Kent Bye: And in terms of editing, is this something that you're also needing to create customized software for people to be able to actually edit things? Or is there no edits going from one scene to the next? Is that too jarring of a thing?
[00:05:02.579] Jens Christensen: No, we actually work with existing tools. It's very important for creatives to be able to use the tools they're used to. So we're working very hard to make sure that our videos can integrate with the tools that they're used to using for editing, for compositing, color correction, et cetera.
[00:05:18.524] Kent Bye: I see. And so is that there's just a component of virtual reality where if you're in Avid or Premiere, you can just hop into seeing it in VR then? Or how do you actually see what you're doing when you're editing?
[00:05:30.437] Jens Christensen: Well, I mean, so this is a work in progress, so the answer is yes. Sometimes you see it in VR, sometimes you don't. There are different ways of doing it. And it's early days for it, but for the approach we're taking is rather than building up a whole set of special tools, rather we want to integrate with the existing tools.
[00:05:49.005] Kent Bye: Do you see that there's a new form of storytelling that's emerging from this in terms of creating spaces and environments and taking a completely different approach to traditional film approaches?
[00:06:00.626] Jens Christensen: Yes, absolutely. I mean, it's a very different thing to shoot in VR. You know, the sets have to be fully 360. You don't, for example, have a cameraman anymore standing by the camera because he would be blocking the shot. The lighting have to be natural lights or at least well hidden. So there are all sorts of considerations. And then from a narrative point of view, because you can be looking anywhere, you have to often give cues, audio cues, visual cues, etc. to your audience. So it's a whole different medium as far as we're concerned. And so it's going to require new thinking and I think it's very exciting for directors and filmmakers and general just producers to decide on what type of content makes sense and you know it's going to be most exciting to the consumer.
[00:06:45.672] Kent Bye: And can you look straight down and not see a pole or is there going to be something holding up the camera? Are you able to do some magic to have this sort of camera magically floating in space?
[00:06:55.025] Jens Christensen: Well, I mean, movies are magic, so yes, you can definitely do things to do that, yes.
[00:07:00.032] Kent Bye: What about motion and moving the camera around? Is this something that you found that just nauseates, you know, gives some simulator sickness, or is this something where it's a format where you really just need to set the camera there and have the action sort of flow around you?
[00:07:13.118] Jens Christensen: Yeah, so we prefer having a steady camera. We think that really reduces the chances of anyone feeling nauseated or sick in any way. And moving the camera can be done. It's always good to have a frame of reference when you're moving the camera. So we find that can be successful as well. But obviously, you know, comfort is very important.
[00:07:30.619] Kent Bye: A frame of reference like a cockpit or something that's sort of like in the frame that people can just sort of focus on? Is that what you mean?
[00:07:36.881] Jens Christensen: Right, exactly. So that you're not just a disembodied head floating around.
[00:07:41.503] Kent Bye: Great. And finally, what do you see as the ultimate potential for virtual reality and its future?
[00:07:47.293] Jens Christensen: Well, it really has amazing potential. I mean, as I said, we see it as a new medium, and we think it can really change all kinds of industries, from news, where you get a much more objective viewpoint, to music, where you can feel like not just being in the audience, you can actually feel like you're up on stage with your favorite artist, to verticals like travel, you know, visit destinations you want to go to ahead of time. So it really just has a myriad of potential.
[00:08:13.819] Kent Bye: And is there anything else that's left unsaid that you'd like to say?
[00:08:17.488] Jens Christensen: I think it's really an exciting field to be in. It's the very beginning, and I feel privileged to play a small part in it. Great. Well, thank you so much. OK, great. Thank you.
