#92: Justin Moravetz on developing Proton Pulse and Vanguard V

Justin Moravetz of Zero Transform is a one of the most passionate and dedicated indie VR game developer that I’ve met. He’s been thinking about Virtual Reality since the 90s, and he talks about the evolution of developing both Proton Pulse and Vanguard V. Some of the other topics discussed were:

  • Differences between designing for the Sony Morpheus, Gear VR & Oculus Rift.
  • User experience and input control innovations with tapping
  • Lessons learned from his Kickstarter campaigns
  • The challenges of making it as an indie VR developer
  • The iterative process of game design and tracking progress with video logs
  • Using Gear VR to spread his VR development work beyond the VR community
  • The implications of being able to manifest your imagination to scale
  • His journey into VR and game development
  • Importance of teaching through interactions in VR

Since the recording of this interview, Justin has announced that he’s helping out on a VR Rock Opera called [NUREN].

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Rough Transcript

[00:00:05.452] Kent Bye: The Voices of VR Podcast.

[00:00:11.957] Justin Moravetz: I'm Justin Morvitz. I'm the lead developer and founder of Zero Transform, and I've made Proton Pulse and Vanguard V. Proton Pulse is a love letter to the early 1990s, the day of 16-bit Amiga computers and such, where the birth of VR really struck me. And Vanguard V is my latest title with a third-first-person rail experience that I'm hoping to get out there and develop.

[00:00:34.916] Kent Bye: I see, and so ProtonPulse was kick-started, it sort of came up and then went away, and then you just had this sort of Phoenix Rising coming back, and then also on Gear VR. So maybe you could talk about the evolution of ProtonPulse, of the beginning into where it's now.

[00:00:50.604] Justin Moravetz: ProtonPulse originally was a tech demo that, when I first got the first development kit from Oculus, I had ProtonPulse running on an old Sony 3D Viewer HMZ-T1 with TrackIR. And the hardware wasn't there, so I got this Oculus, and within half an hour I was up and running. The game was... So I'm like, great, I have to share this. I put it on the development forums, and it kind of took off. And the reaction was so great that, okay, now I have to finish this thing. There's no way that I could just hold it back. I put up the Kickstarter, 12 hours I was fully funded. All I asked for was what I needed to get the game done, which at the time was just a Unity license and such. And there was an issue with where I was employed and the conflict of interest and stuff like that. And well, without going into too much detail, I ended up having to take down, after it had already been funded, I had to refund everybody. Now, the interesting thing about refunding is Kickstarter and Amazon still keep their bits. So I had to pay out of pocket to make sure everybody got their money back. and I was not allowed to talk about the details, so it was really hard on my part to get that done. That being said, I didn't give up. I kept working on it and working on it behind the scenes, getting some more music, cinematic events, boss fights, all this stuff, really pushing it as far as I could. At some point in time, early 2014, I finished the title, and at that time I was kind of balancing should I go independent? Could I pull that off? I started working on Vanguard V at the time and there was this pivotal point at GDC where some of the musicians I worked at were performing live and they just surprised me and played Proton music and nobody else there got it but they were all digging it and I'm like yeah yeah I need to do this and so I went independent and now I'm licensed with Sony I have a Morpheus I'm working on the Gear VR and Oculus and all that stuff And so ProtonPulse was kind of the learning curve I needed to get VR going, and I couldn't be more thankful for the adventure it's put me on.

[00:02:41.378] Kent Bye: And so talk a bit about the gameplay of ProtonPulse and the differences of playing it on, say, Gear VR, Sony Morpheus, and Oculus. You know, you're working on all three of these different platforms, and what do you see as the differences between them?

[00:02:55.800] Justin Moravetz: lighting, but that's it. No, it's brilliant. Proton Pulse is so scalable that what you see on Gear VR is what you get on Oculus, is what you get on Morpheus. Now, Morpheus isn't out yet. That gives me time to take the level of visuals up a notch and do a few other polishing things, but the core essential gameplay is exactly the same being a 3D first-person breakout experience. So, I was really amazed how well the Gear VR could handle gameplay, head tracking, all of this stuff. As long as you know what's optimized and where for mobile hardware, you're good to go.

[00:03:28.349] Kent Bye: So what kind of changes did you have to do in order to build something for Gear VR? Since you had already kind of developed it for Oculus, what were some of the changes you had to make in order to make it work on Gear VR?

[00:03:38.159] Justin Moravetz: Well, a lot of the changes were just to make things run faster. So a lot of what you see in Pros on Pulse moves with the beat of the music, it pulses, and how that originally worked is it would listen to the song at certain frequencies and pulse things along. Now that's kind of expensive, it's like having two audio tape decks side by side and one's recording the other, it's not efficient. And so, because I needed as much as I could get out of this little thing, I developed a new system that would take the actual source music file and parse the XML and spit out events in Unity, so that was great. Also, I had to take out pixel shading so I don't have a point light on the proton ball anymore. I have to work with other tricks. Not that the phone can't handle it. I got to worry about battery life, heat, and all this stuff. So you want to make sure that the end user experience is as pleasant as possible. And you have no idea if they're going to get a phone call, messages, or any of that stuff. So you keep it as smooth as you can and hope for the best.

[00:04:30.185] Kent Bye: I see. And so, in the process of doing Proton Pulse and then moving into Vanguard V, I'm wondering, what were some of the user interactions and insights that you got? Because it seems like you're just sort of using just head tracking and not a lot of other VR input control.

[00:04:46.138] Justin Moravetz: So, when I look at PC gaming, you see keyboard and mouse, you see Xbox 360 controllers, you see all of these various input methods, and there's no single favorite. And when I'm using a controller in VR, there's a disconnect. So, I mean, if I was in a game in VR that puts me in a wheelchair, an electric wheelchair, then yeah, that analog stick makes some sense. But for me, it didn't feel right. On Sony hardware, you have the move input controller. Those work really, really well, but you can't use that for anything on the PC side. So I opted to boil down controls as much as possible with what I know they have, and that's just the head-mounted display. And so input methods like tapping the side to press buttons was a way for me to add functionality that wasn't originally with this system. And anything I could do to just make sure that the experience that one person gets with their hardware is just as good as the next guy.

[00:05:40.207] Kent Bye: And so is this tapping the HMD something that you're able to do across both the Gear VR, Sony Morpheus, and Oculus Rift?

[00:05:48.417] Justin Moravetz: Anything with the gyro, yes. However, with the Gear, it's not necessary because it has a touchpad on the side anyway.

[00:05:53.844] Kent Bye: I see. And so what happens when you touch, like what type of actions happen within VR when you're tapping the side of the HMD?

[00:05:59.755] Justin Moravetz: So when I use my Tapscript, all it's doing is looking for a spike on the x-axis of the gyro. But on the Gear VR, it's just a mouse input, and the button is just the escape key on a keyboard. So it's really easy to map to.

[00:06:11.126] Kent Bye: In terms of gameplay, what are you doing with that?

[00:06:13.768] Justin Moravetz: Right now, it's just for interacting with the menu, and the reason for that is some people play the game smooth, some people play it violently moving around, and you can get false positives. And while things work great, the second it's off, you instantly know that that wasn't right. It's like playing with an Xbox Kinect. It can work great, but the second it doesn't, it's very, very obvious that something went wrong. And I try to avoid that wherever possible.

[00:06:37.191] Kent Bye: Have you been using things like Leap Motion or Kinect to try to do other user interactions in terms of virtual reality?

[00:06:43.834] Justin Moravetz: I use Kinect not for gameplay, but for motion capture. The intro cinematic for Vanguard V was all captured using a Kinect. As far as the Leap Motion, I'm working with those guys and developing some variation gameplay methods on Proton Pulse and Vanguard V, but I can't really go into the details on that yet.

[00:06:59.813] Kent Bye: And so, in the process of doing an ambitious Kickstarter, where you set out to do a goal of $180,000 to pay for a bunch of musicians and developers in your own time, and going through that process and not being able to meet that goal, talk about what you were able to do afterwards and kind of make the most of the network that you were able to build and cultivate over that process.

[00:07:22.499] Justin Moravetz: So I am going venue to venue to try to find whatever options I can to get this game off the ground. So right now, I'm out of my element. I'm not in front of my machine developing. I'm not working with my team members. I'm having to lift and build as much foundation as I can to get to the next step, which is the boss fight, which is level two. And it's going to be a much more stepped iterative process than just we're full production. Everybody do this. Here's me delegating off the assignments and such. So it's slowed down. That price point was called greedy. It was also called ambitious. If you take that and divide it by eight, you'll see that that's me working for free and them getting paid half of what they used to make for a year's worth. And that does not include the hardware, the licenses, or any of that stuff. I mean, if you listen, for example, to the Vanguard V theme with the vocals and stuff, that's a full orchestra. That's somebody singing. That just doesn't happen with a solo indie. I can't sing, for one. You have to have a team. I know what my limits are, and I know how far I can go when I have somebody who's specialized in music, somebody who's specialized in this and that, and as long as you manage it right and work with people who are great, you can get some really, really good stuff, and that's the goal with this, is to make a VR-dedicated studio, not something that's afterthought VR or just a little tech demo. This is the full package game, and I needed, when I came up with that price point, that was the absolute bare minimum to pull it off without even paying myself at this point.

[00:08:47.668] Kent Bye: Yeah, I think that's a good point in terms of like it's a reasonable amount to be able to survive but at this point it's like the support of the VR community is not able to sort of fund these higher amounts and so I guess it's an interesting question of how do you survive and how do you keep going and keep doing it even if there's not the financial support there.

[00:09:10.967] Justin Moravetz: Well, if you find the answer to that, let me know. Because I honestly don't yet. I'm working on it. That's part of the reason why I'm here and everywhere I can be is to get that answer. But I'm looking forward very much to getting back to where I belong in VR and making that as best as I can.

[00:09:27.184] Kent Bye: I see. And one of the things that I've noticed that you're doing is doing these video blogs and sort of developer videos as you're going through the process. Tell me a bit about what that is and what type of things that you're showing people.

[00:09:40.430] Justin Moravetz: So one of the things I don't really understand about the industry is that they will give you a teaser trailer that does not represent the game, and then they will go quiet and silent, and then they give you another blowout. It's like they're a marketing team more than an actual game development for the longest time. And it is my understanding, why not share the experience? Give somebody an understanding what game development is like. Most of the time it's trial and error. Even if you get something you like, it might not be the best approach to that particular problem. And so... It's not that one person has all of the answers right away. It's all pre-figured out. If you find somebody who's going into a game and everything's already figured out, that's not how you make a good game. It's always been iterative. If Sony taught me anything for the nine and a half years I was there is that you have to believe in iteration because if something doesn't work, don't stick to it. Make sure you fix it. Make sure you work on it and get the best product you can. And so in these video logs, I'm trying to show this is how I handle getting this on gear. This is how I handle a side job so I can fund more development. This is what being an indie is. And it's not all glamour. It's not me being at trade shows or anything like that. It's a lot of work. Any interaction I can get from that or any way this could benefit anybody else, I'm all for. I don't like holding back trade secrets or anything like that.

[00:11:00.508] Kent Bye: Well, when you look at the Oculus, Gear VR, and the Sony Morpheus, it looks like that the Gear VR may actually be the one that hits market first. And so, when you look at that as an independent developer, how are you planning to sort of use that event as a potential source of income for yourself?

[00:11:18.605] Justin Moravetz: What I need to use the VR for is to find a way to take what I make and bring it outside of the established VR community. Right now, I can talk to somebody within this VR niche and we can communicate all day long, but everybody's trying to find their place in this emerging ecosystem that's coming. I love making VR more than anything. It's not a job for me. That's why I have a hard time paying myself for it. My wife will argue with that to some degree. But the thing is, with gear, it's more of a introducing the world to who you are. It's a good opportunity for that. And I'm not entirely sure how it'll help me financially. I'm not expecting that. That's not something I think too much about. But anything I can do to bring how I see VR to the masses, I'm all for. And while it's not what I expected to be first, it is certainly a really good, easy, approachable system. And I couldn't be more thankful to be part of it.

[00:12:12.667] Kent Bye: And what is it about virtual reality that you love so much?

[00:12:16.868] Justin Moravetz: Take an object in a video game, any object, let's just say a vase with a flower in it. So you have that on a screen and that's good art direction, there's some good lighting. When you have it in VR, the depth, the interaction, it's there, it's physical. Now the first thought is you can recreate a world, you can recreate something that exists. But beyond that, you can recreate anything and make it physical, make it real. We can do sci-fi all day long in video games. It's been going on forever and in movies, but there was nothing physical about it. It was something you were watching. You were watching a story, you were watching a character interact, but it wasn't you. But when you're there and your size, your scale is relevant to what you see, I could make a working desk where I can do work and look out and see the rings of Saturn. with whatever I want. And see, even that's physical. My mind hasn't even reached the point where I can invent things that don't exist yet. And there's always that blend of marrying what's real and what's not to create new things. But I'm really excited to see us kind of pull away from reality and take virtual reality to places that were never imagined. And that's what excites me is how physical it is and its possibilities. How did you first get into virtual reality then? Early 1990s, I saw an article that said virtual reality is coming. I waited. Nope, not for the Genesis, not the Atari Jaguar. It just didn't happen. I had so many head-mounted displays. I think my first real one was the VFX one, and it required a certain video card type and all this stuff. It wasn't the greatest experience however my room was messy and when I mean messy it was like knee high and I was stressed out I couldn't breathe there was some claustrophobia I went in there and there were no texture maps however it was clean and I could breathe again and that always lived with me and so I kept getting hardware after hardware and there was something about being able to move and see and be part of this world. While it wasn't there, I was filling in the blanks, but nobody I knew bought it. So, they were all skeptics when I first got the first Oculus dev kit. They put it on and it was complete 180. I'm like, look, this is what I've been waiting for. I've been trying to hack it together since. Welcome to VR for real.

[00:14:27.379] Kent Bye: Wow. Wow. So, it's been quite a long journey for you, then it sounds like. When did you decide to make it part of your career to be involved in virtual reality?

[00:14:36.403] Justin Moravetz: When I first had VR, I didn't know how to do game development yet, but it was a passion of mine. And then I started working at a game studio, and I was there for nine and a half years, and then I really kind of learned how the process worked. And at that time, I had a Sony HMZT-1 3D viewer. It had 45 degrees of viewing angle, and its latency was about 100 milliseconds. But then I put TrackIR on top of it, and Suddenly I was able to interact with things a little more, getting closer, developing, hacking things together to get what I wanted. That's when I actually started developing for VR for the first time. Engines that are approachable, say like Unity or Unreal, really makes it so anybody from hobbyist level to full-on game developer can really create their own experiences and learn from this process. And so it was somewhere around 2012, 2011 that I actually went from VR consumer, and I wasn't really big at the time, to VR developer.

[00:15:32.196] Kent Bye: And because you've created these VR experiences and been involved with it for so long, what do you think are some key components from a design perspective when you're creating a virtual reality experience that are essential for making it a pleasurable experience?

[00:15:46.464] Justin Moravetz: Teach through interaction. So in Vanguard V, when I ask the player to calibrate the VR, I also teach them how to shoot the bugs at the exact same time. I also teach them how to tap the side to interact with things. And when I'm in the first cinematic, let the player control the camera. You never take control of it. And so, at one point in time, I fade everything out to black, I move the camera, and then I fade back in. So the camera's in a different position, but it doesn't make you sick. And when you fade back in, you'll notice the character you see is slightly too low, like it's off-center. This also tells the player you can look elsewhere than just straight forward. Don't just sit there, you'll have to look around. And this becomes really important as soon as the cinematic is over because then I have the player transition from too low to back to center again. And so it's guiding the eye, guiding the player using sound, using visual cues, do not move the camera, let them handle all of that. These are all really, really important steps in VR.

[00:16:41.772] Kent Bye: What do you see as the ultimate potential for virtual reality?

[00:16:46.177] Justin Moravetz: That's what I'm here to find out, and I'm glad to be part of that, those first steps. The initial thought is, yeah, it can do games, it can do interaction, we can have VR social experiences and such, but it can go on to therapy, it can go on to shopping, you can actually see what the scale of a product is without having a banana in the picture, you know? Imagine if you're in New York and you want to buy a house or rent an apartment in LA. That's an expensive trip. Why not 3D scan the thing and really get the scale, really get an idea, walk around the neighborhood. That's going to happen at some point. The technology is getting closer and closer, faster and faster, where anybody could take any existing space and bring it into VR. On a retail perspective, that's going to be huge.

[00:17:30.540] Kent Bye: Great. And finally, is there anything else that's left unsaid that you'd like to say?

[00:17:33.943] Justin Moravetz: I am really impressed how people making this hardware have made things approachable. So Leap Motion, you can just drag and drop, you're done in Unity. Oculus, their stuff is there. Anybody can make stuff. You just put in the camera in your scene, now you're in VR. Sony is doing the same thing and it just snaps on your head and you can focus the lens It's getting easier and easier and more accessible for anybody. It's back in the day It was really really you had to set this up set that up and there's still some of that going on But it's getting more and more streamlined. This is gonna be for everybody and I can't wait great. Well, thank you Yeah, no, it's been a pleasure

More from this show