Max Geiger works at Wemo Lab, which is a content studio in LA that exploring gaming and simulation in VR, but also exploring panoramic VR capture and the software to make that happen. Wemo Lab is located in LA, and they have a number of award-winning special effects artists on staff for creating who are helping create various VR experiences.
They’re focused on bringing emotional investment into VR, and Max talks about the spectrum of cinematic VR storytelling ranging from computer-generated to captured material, as well as differing levels of interactivity within each of those. He says that we’re still inventing the language of VR, and that the most surprising applications and interactions for VR haven’t been discovered yet.
Max could neither confirm nor deny that they were collaborating with any specific directors, but being so near to Hollywood it would not be surprising if they were getting interest from the film industry. He also talked about how close-up magic, immersive theater experiences and haunted houses have lessons to teach VR in terms of how to direct and misdirect attention.
Finally, he talks about he doesn’t like to do too much speculation about VR either in the short or long-term because of Amara’s Law, which states that “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.” While there’s a lot of VR hype train overestimation in the short-term, we tend to underestimating the long-term impacts because so many of the changes are so unpredictable.
Reddit discussion here.
TOPICS
- 0:00 – Intro. Wemo Lab content studio in LA. Showing an immersive ocean simulator called Blue.
- 0:35 – The Blue was an open platform to contribute fish to various environments.
- 1:48 – Exploring gaming and simulation in VR, but also exploring panoramic VR capture.
- 2:15 – Interested in writing software to make it easier for other people to make captured
- 2:36 – Using off-the-shelf solutions at the moment, and investigating other proprietary solutions as well.
- 2:52 – Looking at 360Heroes rig with Go Pros, Wemo Lab’s Dennis Blakey is a pioneer of stereoscopic video who created a rig with 84 cameras.
- 3:50 – Presence is the real selling point of VR, and so Frontrow VR can help provide that sense of presence. Getting fooled by close-up magic within VR.
- 4:50 – Tradeoff vs recreating it in 3D to be more efficient vs video capture. It’s getting easier to store and manipulate large quantities of data.
- 5:57 – When would it be better to recreate vs. when would you need to create? You know how much a camera will bias things, and an editor can weave a story out of individual moments. Interested to see what Peter Watkins would do with VR, who used documentary format to explore fictional stories. Explores film create a world and expectation and biases the viewer towards certain things. The map of a film is the territory of the subject
- 7:50 – Different interactions within VR and approaches to storytelling. 6-7 different levels of experiences spectrum between completely computer-generated vs. filmed and captured experiences. And adding interactivity to captured experiences. Still inventing the language. The most surprising applications and interactions for VR haven’t been discovered yet.
- 9:00 – Getting interest from Hollywood directors at Wemo Lab? Neither confirm or deny working with any Hollywood directors.
- 9:40 – What is Wemo Lab trying to do in VR? World Emotion is the goal. Bringing emotional investment to VR experiences. Combine emotions with physical interactions in VR.
- 10:30 – Directing attention in VR experiences. Look at first-person games and how they direct attention, but also look at other arts of directing attention like how magicians will direct and misdirect attention. Breaking down the fourth wall in theater has lessons to teach us as well.
- 11:54 – Sleep No More immersive theater experience is a high-brow, but there’s also a haunted house or a dark ride and there’s lessons to be learned from them all.
- 12:38 – 3D audio. Not a lot of great solutions at the moment, but there’s a renaissance in that realm not. Binaural audio is a capture technique, and 3D positional audio is the post-production and mixing process involved.
- 13:12 – Ultimate potential for VR – Tries not to do too much speculation, and refers to Amara’s Law of “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.” Hype train overestimation in the short-term , but underestimating the long-term impacts.
Theme music: “Fatality” by Tigoolio
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:11.915] Max Geiger: My name is Max Geiger. I'm a producer with Wemo Lab, who's a content and software development studio based in Venice Beach, California. So we're just down in Los Angeles. And at SVVR, we are showing The Blue, which is an immersive ocean simulator built on a content creation platform we've previously released and are now refreshing and bringing to the world of virtual reality.
[00:00:34.635] Kent Bye: I see. And so, you know, tell me a bit about, like, what is happening in this capture in order to kind of translate it into virtual reality?
[00:00:41.384] Max Geiger: The Blue is not based on any sort of particular capture technology. It was an open platform for makers all over the world to contribute fish. to various ecological habitats that we had designed. So modeled after real-world habitats like Arctic icebergs, kelp forests like we have off the coast of California here, coral reefs like you would see, you know, the Great Barrier Reef in like the Florida Keys, things like that. And so people all over the world were able to contribute fish, and then our own team of Academy Award winning and Emmy Award winning effects artists would also create and animate fish that were in this environment. And so The Blue originally existed as a platform for sharing, trading, buying, selling these fish. A little bit like Steam's workshop for items in like TF2 now, but back in 2012. So what we're showing today is actually a refresh of that aimed towards VR. So rather than being a more observational experience, it's an exploratory experience where you can now get into the environments, swim around and see things from that up-close VR perspective.
[00:01:48.455] Kent Bye: So what else are you doing in VR then?
[00:01:51.056] Max Geiger: So in VR, we are exploring games and simulation and game-like things, but we're also investigating stereoscopic panoramic video capture. So the ability to look at things that have been filmed in every direction, you know, all around you, up and down, but also see everything that's been filmed in 3D with a sort of true stereoscopic 3D for both eyes.
[00:02:14.062] Kent Bye: I see, and so do you have a specific camera gear or software? What part of the pipeline are you doing in order to do this capture?
[00:02:20.665] Max Geiger: We're interested in writing the software to make it easier for other content creators to do the capture themselves and then be able to very quickly composite, edit, and turn their footage around into something that is not headache-inducing like some of the early capture efforts out there.
[00:02:36.251] Kent Bye: I see, and so are you using off-the-shelf GoPros or what are you using for the actual cameras then?
[00:02:41.231] Max Geiger: Well, we're using off-the-shelf solutions right now, but this is just sort of the first step for us. So we're investigating more custom, more proprietary solutions as a possible future path.
[00:02:52.784] Kent Bye: And so talk a bit about how many cameras there are and how many video streams, and I'm just trying to get a sense of how much data you're dealing with here.
[00:03:01.713] Max Geiger: So, you know, we are looking at rigs like the 360 Heroes rig with, you know, a number of GoPro cameras in it. But, you know, one of our pioneers, one of the people on our team who's heading up this effort, Dennis Blakey, is an Emmy Award winning effects artist for his work on Deep Space Nine in the 90s, has actually been a pioneer in the realm of stereoscopic video and image capture for over 15 years now. And some of the things that he's worked with, he worked with a rig And I don't have the exact timeline on this, but he worked with a rig of standard def cameras that was 84 cameras and shot some stuff in Times Square, I want to say over a decade ago. And that looked pretty good. And that was on standard def. So with modern storage systems and modern high definition cameras, the sky is really the limit.
[00:03:50.402] Kent Bye: So one of the things you have listed on your website is front row VR. And so is part of the idea to be able to set up a fully 360 degree camera rig on the front row of sports? Or what does the front row VR mean?
[00:04:02.923] Max Geiger: Well, you know, people have been talking about presence as the real selling point for VR and the idea of front row VR is that you can set up this VR capture rig in any situation where traditionally you would have to go and visit in person to get that sense of presence. So that could be a sporting event, that could be a musical show, it could be perhaps even a theatrical show or venue, something that I really want to see which I think diehard traditionalists would absolutely crucify me for, would be magic tricks. I think there's a real application for getting fooled by close-up magic through a VR system as opposed to having to visit it in person. And, you know, filmed magic is one of those things that just hasn't been particularly interesting or well done except for a few rare examples here or there.
[00:04:48.926] Kent Bye: I see, interesting. I guess one of the things that I think of when I think of these capture units versus the approach that Nani de la Peña is taking in terms of recreating it in, say, a 3D environment. So there's a trade-off I see between doing something efficiently vector-based in a 3D environment where the file sizes may be smaller, but then when you get into these captured experiences, because I've heard the number of like 1.7 terabytes per second with some of these units, So you're dealing with a lot of data that then has to be compressed, but you're still dealing with huge file sizes. So I'm just wondering how you guys approach that.
[00:05:27.586] Max Geiger: You know, I can't speak too much to that, but I can say it's getting easier and easier all the time. We live in the era of big data, and this is a complete misuse of big data and what it refers to, right? But we are living in a world where it's easier and easier to store and manipulate extremely large quantities of data. The real-time capture of it, yes, is something that has to be addressed. And there are solutions that can be improved upon. And that's something that we're certainly investigating.
[00:05:58.027] Kent Bye: You know, my background's in documentary filmmaking, and so I sort of wonder what situation would be better to recreate it, and what do you think is a situation where you absolutely capture would be much better than trying to recreate it?
[00:06:10.745] Max Geiger: That's a very good question, and if you come from the world of documentary filmmaking, you know how much the camera biases things, right? Like the lens of existing cameras in terms of what they show and what they don't show, and how a skilled editor can pull together a number of different shots in the soundtrack. To really weave a story out of what might have just been, you know, a few captured moments here and there is an incredibly powerful thing. And it's something that we're going to have to reinvent as we learn how to capture everything and can capture everything in an environment surrounding, you know, one of these rigs. I'd be very interested to see what a filmmaker like, say, Peter Watkins would do with one of these rigs where he used the documentary format and documentary style to explore fictional problems based on, you know, real world issues. It's one of the things that challenges, perhaps, what he would call the monoform, if you've ever heard of him or his work and what he's interested in. Expand a little bit. Sure. Sure. So I'm going down the rabbit hole a little bit on, there's this one filmmaker, his name's Peter Watkins, and he used the documentary format to tell fictional stories. And he started, actually, with films like War Game and Culloden. And I believe they were in the 60s and 70s, for the most part. Punishment Park is another famous one that he did. A lot of his work explores how film creates a world and expectations and biases the viewer towards certain things different from what's actually being captured or what the reality is that's being captured and how, I guess, the map of a film is not the territory of its subjects.
[00:07:48.994] Kent Bye: Well, I guess one thing about VR is that Cosmo is talking about, you know, three different approaches to doing, you know, interactions in VR space. You know, everywhere from, like, complete and total on the rails, you can just sit there and watch things go by, to, like, completely open, where you can control and do anything you want, and then sort of somewhere in the middle where there's some mix of the two. And so I'm curious, coming from LA and storytelling, you know, what types of approaches you guys have thought about doing?
[00:08:17.157] Max Geiger: So we have a team member who's actually broken out like six or seven different levels of experiences and sort of just identified them as, I guess if you want to look at the future VR content as a spectrum between computer generated simulation like a game and filmed and captured experience like the panoramic capture rigs that we're seeing now. And then things even beyond that, adding interactivity to those captured experiences. There are a number of possibilities that immediately spring to mind there, but I think we're still inventing the language and the most surprising applications and most surprising interactions for VR haven't been discovered yet. And I think we're pushing on that and other people are pushing on that and it'll be very exciting to see where it heads in the next few years.
[00:09:00.550] Kent Bye: I see. And so are you getting interest from filmmakers? I know Alfonso Cuaron has been reported as to have visited Oculus Rift to kind of start to see some of the interactive VR experiences. But I'm just curious if you've had different directors come to you and express different levels of interest in getting involved in this space.
[00:09:18.432] Max Geiger: That's where I have to drop a very government style, I can neither confirm nor deny the possibility that we may be working with any Hollywood level directors. Yeah, maybe we should move on rather than dwelling on that point too much.
[00:09:32.478] Kent Bye: So the answer sounds like yes.
[00:09:34.799] Max Geiger: I can neither confirm nor deny.
[00:09:38.495] Kent Bye: Okay, well, we'll move on. So, I guess, when I look at your website, like, what do you see is your thing that you're trying to do the most in VR? Like, what's next for Wemo Labs?
[00:09:50.058] Max Geiger: So what WEMO actually stands for is World Emotion. And what our focus is between our team of AAA game veterans and top tier Hollywood talent is bringing emotional investment to VR experiences. I think what VR helps us achieve is a greater sense of presence, but I think we're still going to need to figure out how to get people emotionally invested in these worlds once they're feeling physically present within them. And it's really that emotionality that's going to drive and work In synergy, I can't believe I just used the word synergy, forgive me. In synergy with that physical, visceral sensation of being there.
[00:10:27.704] Kent Bye: I see. And so, in film you have a camera where you're able to direct it and direct attention. In VR, you have your body, you can look anywhere. So, how do you direct attention in a VR experience?
[00:10:40.067] Max Geiger: Well, that's something where we have to take the lessons of directing attention in first-person experiences from games. I think that's the starting point, right? But I think there are other ways to use existing arts, if you will, and other things that we can borrow from them that haven't really been explored yet. I mentioned magic tricks a little bit earlier, right? And so much of the magician's art is direction and misdirection, focusing someone's attention when they can look anywhere at what you want them to look at, right? That's such an important thing. I think another world that is ripe for exploration is theater and the developments in theater towards breaking down the fourth wall and engaging the audience in new ways when, again, they can look pretty much anywhere or perhaps, you know, theatrical experiences that have happened live theater, right, as opposed to, you know, cinema screens. Theatrical experiences or those, like, theater experiences where you visit a house and the play is taking place, you know, throughout all the different rooms of the house and perhaps you can just walk through and see some of the characters and you have a very limited viewpoint at any one particular time, yet it's still a living world that you're inhabiting. So I think how attention is directed in those experiences is something that we can borrow a lot from right now as we're figuring out what works best in VR.
[00:11:54.502] Kent Bye: Yeah, back in October of 2011, I had the opportunity to see Sleep No More, which is the immersive theater experience in New York City, and I think it was probably one of the most engaging and immersive experiences I've ever had. And I'm curious if you've seen Sleep No More or have taken different inspiration from what type of things they're using in that type of immersive theater.
[00:12:15.001] Max Geiger: I mean, I think that's definitely the highbrow version of inspiration, and I think maybe the lowbrow version of that is, you know, like a haunted house, right? Or even just a dark ride. And I think there are lessons to be learned from all over, and it's an exciting time to re-synthesize a new language for directing attention and getting people invested and engaged in VR worlds.
[00:12:37.654] Kent Bye: One of the big other issues is audio. How do you deal with 3D positional audio when you are able to turn your head anywhere?
[00:12:45.018] Max Geiger: There aren't a lot of great solutions for that right now, and it seems like a lot of the development in binaural audio has stalled out in recent years, but it seems like there's a real renaissance as people try to figure that out. I guess binaural audio, just referring specifically to the capture technique, and then 3D positional audio as a playback and mixing technique for incorporating that into experiences is another nut to crack, and I think it's an exciting time to be working on all of that, too.
[00:13:12.055] Kent Bye: So maybe we could wrap up a little bit in terms of what do you see as the potential for what VR could be and where it's going?
[00:13:20.365] Max Geiger: So personally I try not to do too much speculation about where VR is headed either in the near term or the long term. I'm a big believer in Amara's Law, which is named for Roy Amara. And what that law says is that in the short term, we tend to vastly overpredict the impact that a new technology will have, right? People tend to get very, very excited about what's happening, you know, in the next year or two. And sometimes those expectations fall short. We're all familiar with the hype train, right? And things getting overhyped. And I think the law bears out in that regard. But then the second half of it says that in the long term, we vastly underestimate the potential impacts of new technologies, right? I don't think when text messaging was first envisioned, anyone would have imagined that it was going to drive, you know, money exchange and African banking, right? And yet there it is. It's a whole booming new business model in a new way, a new innovation and a new sector that is, you know, driving development in the developing world. And so I think VR in some ways is going to be very similar. Right now, it may seem like just a toy to many people and people are talking about, you know, not just game and entertainment applications, but also industrial, medical, scientific, educational applications. But I think in the long run, we have no idea where this thing is going to go. And the most surprising and exciting things are going to be coming down the pipe in a few years from now. And it'll be great to be a part of that.
[00:14:45.567] Kent Bye: Great. Well, thank you so much.
[00:14:47.028] Max Geiger: Thank you.
