Michael Licht is a professor of level design at USC’s Viterbi School of Engineering and has involved in video games for over 15 years and has an architecture background. He paired up with Nonny de la Pena in creating immersive journalism pieces because he wanted to use his skills beyond just making war simulations like Call of Duty.
He talks about the importance of creating real and believable environments because our psyche will think that it’s fake unless it’s based upon real physics and has sound architecture. He also talks about the importance of being able to freely roam around in an untethered VR experience, and how can create a profound sense of presence
With immersive journalism, there’s not a lot of freedom to deviate from the source material because it starts to become fantasy and not a documentary of actual events. And he talks about the importance of creating virtual human characters through motion capture and facial capture to create an emotional resonance.
Michael is looking forward to continuing to collaborate with Nonny on immersive journalism pieces, but is also interested in creating an untethered VR game prototype to see if it’s something that people would enjoy as digital out-of-home entertainment. He sees that people are willing to take the red pill with virtual reality, and to be taken to a new place and have novel experiences there that will really blow them away. He’s looking forward to seeing the medium evolve and thanks Oculus for creating an open platform where people can experiment and innovate with the VR medium.
TOPICS
- 0:00 – Intro. Game level designer and architect for 15 years. Looking for new application beyond war simulations with VR pipeline, and found Nonny & Immersive Journalism
- 1:00 – Importance of creating environment and spaces. Creating immersion. Real environment that behaves in a real way. Architecture needs to be sound otherwise it’ll feel fake. Knowing how things are built plays a part of our psyche. Base it upon real physics and real life. Second part of immersion is that the space is plausible. It needs to seem realistic. Recreate environments based upon photos
- 3:07 – Incorporating full body tracking. Use their own VR HMD system to create an untethered experience. Using a 20ft x 20ft space to freely move around. Creates profound sense of presence
- 4:30 – Level design of games vs. immersive journalism. Forbidden from deviating from the event that actually happened. Use the audio to match it 1to-1.
- 5:35 – Use of omniscient narration and whether that breaks immersion or adds more context
- 6:31 – Virtual humans, and focusing on motion capture and emotional expression. Play actual audio for the mo-cap actors to match to actual events as much as possible.
- 7:47 – Use of Force VR piece about an immigrant who was beaten to death. Recreated footage from cell camera footage
- 9:30 – Where you’d like to go in the future with VR? Love to see more gaming applications in physical spaces and motion tracking within a large open space.
- 10:30 – Faculty on level design
- 10:50 – VR vs 2D screen. People want to take the red pill and taken to a new place. Facebook acquisition got people’s attention
- 11:30 – Insights from Immersion 2014. Technology is so young, and there’s a lot of experimentation and lots of cool energy and looking forward to seeing what people do with VR.
Theme music: “Fatality” by Tigoolio
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast.
[00:00:12.021] Michael Licht: My name is Michael Licht. I've been a video game, primarily a level designer and environment designer for about 15 years. I worked at LucasArts and Electronic Arts. Before that I was an architect. I actually designed buildings and I spent most my time designing in 3D. So after so many years of working in the video game industry, I've been looking for something different that uses similar technology and I came across Nani and USC. I teach video game design and level design at night at USC and I met Nani there. and Nani does this wonderful immersive journalism where we recreate all these events and it was right up my alley. It was exactly the kind of work that I've done. Well, except instead of shooting people, we are recreating people to draw, you know, for a positive impact rather than, you know, war sims. So it's been an interesting career change for me. Very exciting actually.
[00:01:01.376] Kent Bye: Maybe talk about the importance of knowing how to create environments and spaces from the architectural experiences that you had when it comes to virtual reality.
[00:01:10.144] Michael Licht: It takes a couple things to achieve immersion. There's really two main aspects of creating a sense of presence and immersion. The environment has to look and feel and behave in a real way. So, you know, this was a similar issue I dealt in video games where we would have the same problem. People wouldn't feel an environment was real if the architecture wasn't real. Like if we built a building that had two longest spans with no columns in it, people wouldn't say that. They wouldn't say, oh, this isn't architecturally sound. But they would feel it, like people would walk into a space and it would seem fake. And then you add a couple columns in there to support the roof, and people say, ah, it's such a cool space, it feels so real, right? So knowing how things are built actually plays into our human psyche, because when we walk into a space, it should feel like a real space. So I apply my architectural and construction knowledge in everything that we do. When I was designing Star Wars environments to designing World War II environments, I mean, everything came from photoreal reference. No crazy cantilevers. Everything was based on real physics and real life. Whether it would fall down or not didn't make a difference. It had to look sturdy. So that's where really my architectural experience kind of came into it. The second aspect of getting that sort of plausibility and the immersion is plausible. The space has to look like it could exist. You can have photoreal of all kinds of wacky spaces, but it's kind of hard to get immersed in it if people don't have any way to sort of grab onto it and go, I may understand this space, I've been there before, you know. A space needs to seem, rather than just photorealistic, it needs to seem realistic. Right? So everything we do now in these environments that we recreate to build that sort of sense of immersion is we take as many photographs as we can and recreate those events as photorealistic as we can. I've seen some amazing tech at this conference that seems like it does it even better. I'm looking into it. But right now, it's sort of in a traditional gaming sense. We build the entire environment, use texture maps, shaders, lighting, and light maps, and we make it as real as possible so the space feels very real and plausible.
[00:03:07.090] Kent Bye: I'm curious about, you know, there also seems to be an element with some of these immersive journalism pieces that you're trying to incorporate more aspects of the body and to create a sense of presence and not at this specific context but in other contexts I've seen fully body tracked and so talk about the process that you go through in order to bring the full body into the piece and what type of change that gives to it.
[00:03:29.253] Michael Licht: Well, what's interesting is that we don't actually use the Oculus. We have our own system that we put together. It's married with a system called Phase Space, which is an LED marker tracking system, similar to motion captures that you might see in motion capture studios. In our studio, we have 18 different cameras that track the player. The user has LEDs on his head, and right now we're just tracking that, but we can track arm movement, leg movement, and everything. Most of our demos right now don't even have full body. They're just about translation and space. And no one seems to even understand that or feel that right now. All people know is that for the first time they put on the VR headgear and they can take a step. I haven't seen that anywhere else yet, right? Where they're actually walking around a good 20 foot by 20 foot space and squatting and tilting their head, walking around and examining things with their hands free. And so it's that sense of a body and presence becomes extremely profound when you can actually walk even though you have no body in the actual space. It's quite impressive to see.
[00:04:29.456] Kent Bye: And so from your background in game level design, either you're having events triggered by some action or it's completely automated on rails and there's no decisions and the viewer is just able to look around the environment but not a lot of interactivity. Talk about that process of designing that.
[00:04:47.045] Michael Licht: Well, I mean, level design is, even when I was doing World War II recreations, it was all about brainstorming what we wanted to do, how we wanted it to happen, enemies pop up here, machine guns go there, player decides to plant a charge, and if it blows up this way, it goes that way. In journalism, that doesn't happen, right? We are trying to as closely recreate. Matter of fact, we're forbidden to really deviate from the actual events. Otherwise, it becomes less of a documentary, more about fantasy. So, as far as, you know, the scripting side of level design and the user experience stuff, that doesn't happen in journalism. We want this to be, this is the event that happened. Matter of fact, to keep us as close to the real event as possible, we use the exact audio from the event. So you can't deviate from that. So when the person's yelling or talking, we have to make that person yell and talk in the VR. Otherwise, it's out of sync and it seems totally wrong.
[00:05:35.047] Kent Bye: I see. And one of the interesting decisions that I saw in the piece was that you had like a voiceover narration rather than the voice coming in from someone within the scene. And I'm curious about how do you make that decision as to break that fourth wall and kind of have this omniscient voice come in versus having that information come in from the experience?
[00:05:54.841] Michael Licht: Right. I think that's maybe probably more a question for Nani. I think when that happens to me, I don't feel that fourth wall break. I understand that it's a documentary, right? And there's some kind of... I guess I've seen enough documentaries in my time that when that happens, it starts feeding me context. And because the audio's positioned right on the player's head, it's not local anywhere else in the world, it just kind of feels like I'm thinking, right? At least that's the way it comes across for me. And it doesn't pull me out at all. Matter of fact, I find it kind of... interesting and compelling that that data is now being streamed right into my head, you know, and it makes, it gives me more to work with as far as understanding the environment, which is cool too.
[00:06:31.291] Kent Bye: I see. And there seems to also be a very strong component in using characters and avatars within your pieces. And so talk about the importance of using realistic virtual humans when you're trying to tell these journalistic stories.
[00:06:45.036] Michael Licht: Yeah, about 75% of what we do is all about characters in motion capture and facial capture and expression. And I mean, this is something that Nani and our other producer, Vangelis, are really good at. I have spreadsheets and spreadsheets of the different events that we're doing and every single animation that we see is documented and we take it to a studio and we mo-cap it and we play the actual audio from the event. Like we had this Border Patrol beating and we turned on some speakers and played it real loud and had four mo-cap actors acting out the scene along with the audio and with Nani yelling in the background, hit him harder, you know, and just like to really to match it as closely to the actual events as we can, you know, and it's without the character action and without that empathy that you might feel for a character on a screen, it's just a pretty environment, right? And documentaries, at the end of the day, are about people, right? Well, I guess it could be about anything, but ours are about people. These are news events. It's very important that people have to be incredible.
[00:07:48.558] Kent Bye: And so yeah, tell me a bit more about this Border Patrol piece and what happens and what the message is.
[00:07:55.360] Michael Licht: So this Border Patrol piece is about basically this undocumented U.S. laborer who has been living in the U.S. for 15 years. He's got a family of five. And in 2010, during the downturn, he was basically caught stealing food. He was buying food for his wife for Mother's Day, I think it was, and he was caught and deported immediately once they found he was undocumented. He tried to sneak back over the border on the Tijuana U.S. border and was caught by a U.S. Border Patrol agent, and when he was brought in, they roughed him up pretty good. and fractured his ankle. He went and complained to the superior of the officer who fractured his ankle. He went and complained to that officer's superior and instead of doing something about it, the superior told the officer what happened and the officer and about 12 other agents took him out back and drove him to another location and beat him and tased him for 30 minutes until he died. Choked him and eventually he went brain dead and died in the hospital a few days later. There was very little on the news about it. None of the video footage that people had taken, because the Border Patrol had actually gone around. Once they realized what they had done, they'd gone around and confiscated it. Everyone who was standing around confiscated their cameras. I don't know how they managed to do that, but they did. And two people managed to walk out with their videos. We took those two witnesses' videos and recreated the scene twice from both of their point of views. So you get to watch this guy taking video footage of a recreation of the video footage that he took.
[00:09:17.996] Kent Bye: Wow, and whose story was this and how did you get involved in this specific story?
[00:09:21.860] Michael Licht: Well, again, Nani is a journalist and she looks for events to write about. Anastasia Rojas is the name of the guy who this happened to.
[00:09:30.560] Kent Bye: Your involvement with this immersive journalism, where do you hope to see this goes in the future with what else you want to do with this medium? Right.
[00:09:38.002] Michael Licht: I come from a hardcore gaming background and I love environments and I love, you know, this kind of technology. So this has been a lot of fun for me. I know Nani is a passionate journalist and I look forward to working on more pieces with her as, you know, for, you know, awareness and, you know, for whatever causes she wants. Personally, I would love to see more gaming application to this technology and I have ideas about a lot of really fun stuff that we can do with this. Because like I said before, we don't use Oculus or any technology that forces you to basically stay in one place or hold a controller. Our stuff is based on motion tracking, which means we can have a warehouse and put the player in there and run around and give them virtual swords and all kinds of great games can happen. So I'm really looking forward to putting together some demos for that and throwing them up in a warehouse somewhere, bring some people in and have a night out of it. And say, hey, would you pay money for this? And if we get a good response, I want to make a company around that.
[00:10:28.475] Kent Bye: And are you a part of the USC's Institute of Creative Technologies then?
[00:10:32.821] Michael Licht: I'm not part of ICT, I'm actually a faculty, level design faculty at the Viterbi School of Engineering at USC. And then we're building our own headgear on our own now and we're working outside of USC at this point.
[00:10:45.556] Kent Bye: I see. And when it comes to level design for virtual reality, how is virtual reality different than being viewed in a 2D screen, I guess, versus a fully immersive virtual reality experience?
[00:10:56.039] Michael Licht: Right. Well, I mean, that's the big cool hook with the VR headset. I mean, everyone's always wanted to take, it was the red pill, right? The blue pill or the red pill? I forget which one.
[00:11:03.902] Kent Bye: Red Pill was the actual see what the truth is, and the Blue Pill was to not do that.
[00:11:07.744] Michael Licht: Everybody wants to take the Red Pill, right? They all want to throw on the headgear and be there finally. We've been getting promises technology for a long time, and it's never come through. And Palmer made a great headset, and then all of a sudden Facebook throws a couple billion at them, and then the whole world says, whoa, virtual reality's back. It's going to be a whole new medium. It's a great new way to connect and all that. It was cool, but this is going to blow people out of the water when we start making games for this.
[00:11:34.377] Kent Bye: And finally, what are some of the insights that you're taking away from this gathering of Immersion 2014 so far?
[00:11:40.438] Michael Licht: Well, what's interesting about this is it's very, I don't know if grassroots is the word, but it's just because the technology is so young right now, and thank you for Oculus for throwing it out there with dev kits and letting people just go at it. I've seen some very creative work done, you know, people are just trying all kinds of stuff. And that's great, you know, it's kind of like, you know, when mobile gaming started kicking in and you've seen all kinds of wacky games coming out and, you know, Angry Birds and is people are experimenting and that's the kind of cool energy that I like to see with the new platform. So it's been great to see what people are doing and I look forward to seeing people discover and experiment with new stuff.
[00:12:16.797] Kent Bye: Great, well thanks so much.
[00:12:17.778] Michael Licht: Appreciate it.
