Pete Moss is Unity’s “VR Dude” in that he’s been a long-time believer and advocate in the cross section between VR and gaming and how the Unity game engine could play a significant part. He talks to me about his dual role of content creator and dogfooding the Unity engine as well as evangelizing VR to different communities and collecting experience prototypes from different game jams and developers from around the world. Pete’s passion for the potential of virtual reality is clear, and he’s the most excited to see how artists use VR to express themselves given that VR provides completely new pathways into the brain.
LISTEN TO THE VOICES OF VR PODCAST
As part of Pete’s job, he has been going around to different conferences around the world to keep tabs on where the industry is going. He’s starting to talk at more non-gaming gatherings such as different events in Hollywood as the film industry is starting to look into how to best use this new immersive medium for storytelling and entertainment. But he’s also been available at different game jams as a Unity expert as well as witness to the imagination and creativity for how different developers have created
One of the examples that he provided was being at the Boston game jam with an early prototype of Valve’s HMD and hearing Ben Throop talk about his desire to create a soccer game where you head a soccer ball into the goal. Ben went on to finish that prototype, and eventually have Headmaster be selected by Sony to be a release title for the PlayStation VR, which is having their PlayStation Experience this coming weekend.
Here’s a video of Pete helping to launch soccer balls to Valve’s Chet Faliszek and Aaron Leiby with the first working prototype for Headmaster
I really see Unity as the “lingua franca” of these immersive technologies. It’s pretty clear that the majority of VR experiences have been created within the Unity game engine, and Unity will also be a key to bringing immersive experiences to AR systems like Microsoft’s Hololens, Meta, and Magic Leap.
The PR firm representing Unity followed up with me after this interview, and passed along some of these other statistics in terms of how widely Unity is being used.
- Unity accounts for 95% of all virtual reality content developed for platforms such as the Samsung Gear VR, Oculus Rift, HTC Vive, and Sony PlayStation VR
- On the Oculus Gear VR Store, Unity accounts for 88% of the games
- 174,000 games (not just VR) were made with Unity last quarter
- 5 million people in the world are currently using and creating on Unity
- 465 engineers are working on making a better 3D engine every day
Unity’s biggest competitor in the VR space has been Epic’s Unreal Engine, and I wouldn’t be surprised to see Crytek’s CryENGINE start to move towards more of an open and free model based upon what was hinted to me at the Virtual Reality Intelligence gathering. But whether or not these numbers end up being 95% or 85% of the market share, it’s clear that Unity is a significant player when it comes to providing the content creation tools for VR game and experience developers to manifest their creative visions.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:12.017] Pete Moss: My name is Pete Moss and I work for Unity Technologies. I am the VR dude. It's a title I made up, but I was talking about VR back before talking about VR was cool. So I've kind of got my fingers on a lot of pie at Unity about how this tech can be used for our engine and how our engine can be used for this tech and how it can all work together. In a way, I'm an evangelist. I'm a dev relations kind of guy. I am a game developer. I've got to get back to work here in a few hours and actually build content. So, yeah, I'm just trying to figure out how all of us can work together and make this VR thing happen.
[00:00:45.571] Kent Bye: And so, how is it that VR came into Unity and Oculus, and where were you at in that process and conversation?
[00:00:51.932] Pete Moss: I came into Unity about three years ago, right about the time Oculus was starting to hit the scene. We already had some devices, but they weren't very well supported back then. The background that I have in some of the military simulation work and in DSP and talking to systems and all of that all kind of came to the foreground, also computer vision, obviously. And I saw the potential. I knew that we weren't even close to being there yet, but I could look ahead. I knew that this tech was going to take off. It was tied to touchscreen technology that was already going to lead us to better and better tech. And I knew, I knew it, that we had to be there at Unity. Unity is really cool because the same people working in VR, the DIY type crowd, are also A lot of the same people work in VR, and they're working in indie studios, and they're working in things like this. So a lot of our core crowd, our core customer base, are the same people at shows like this. They're the same people working in this tech. And it was very speculative at that time. But a few of us true believers, as you might say, we could see where things were going to go. We knew that in a few years, we were going to have conferences devoted just to this thing. It took a few years. We're still going. Hey, we're getting pretty close to something.
[00:02:00.542] Kent Bye: And so what's your role at Unity now when it comes to VR?
[00:02:04.923] Pete Moss: So I'm a lead software engineer for the Creative Content Studio, which is basically a demo team three blocks from here, here in Bellevue. So I make content, but I also work closely with marketing, I work closely with the evangelism team, I kind of stake my claim, as it were, on the VR space. So a lot of people in the company defer to me on When they need to know something, they come to me. Not that I know everything. I actually don't know it all. But the cool thing is, especially with the travel that I've been able to do and all the people I've been able to meet in this field, if I don't know it, I probably know someone who does know it, or I can help you find that info. So to me, what I see, it's cool. It's all based on the tech. It's all based on just building cool stuff. But to me, it's starting to look more like a constellation of people that I know. And I get to connect the dots occasionally. I get to see a person over here working on this thing and a person over here working on this other thing. And they don't know each other, but they need to. So a lot of what I do is that. I don't know how you, what do you call that? Like what's the job description for that? So I came up with a VR dude. Someone called me that once in public and I was like, man, that's good. I'm going to use that. But the reality is I'm a software engineer. I work every day in the engine. I eat the dog food at Unity. I get to use it in ways that the core developers never dreamed, and then I get to write them long emails about how they built it wrong. Okay, maybe they're not that mean, but at the same time, I'm really just trying to push this tech and make everything better. I'm a Unity fanboy, I still am. I made my living for years on it, so I want to see it improve, and I want to see the whole scene improve.
[00:03:36.770] Kent Bye: And it sounds like you're here at CVR talking, and it sounds like you're traveling around a lot, talking to different groups, evangelizing, spreading the word. What is it that you find that you're most often trying to communicate to these different groups?
[00:03:49.544] Pete Moss: I guess I'm, I mean, we're called Unity, but I'm trying to preach unity, not the tech, not the company, but this idea that we need to work together as an industry to achieve these goals. So anything that's dividing us as a group, and this is a very large group of people, all with different goals and different ideas, but there's a lot of knowledge there. And then one of the things I really like about VR and where things are now is while there's a lot of competition, there's still a lot of information sharing. things aren't proprietary in quite the same way as they are in another industry. So people feel free to talk and share details and show off their stuff. Everyone's excited about it. And that's, I think, what's really cool is there's this excitement. And I get excited. I get really animated when I get on stage and I'm talking about it. I talk a million miles an hour and I lose my voice like I did the other day because I talk so much. But for me, it's this long conversation that we're all having as an industry. And we're all talking and we're all sharing and we all got a piece. And as I learn more, I try to spread that into the community more. So that, again, we can all grow this. No one can do it alone. If we didn't have the cooperation we have in this industry, we wouldn't be here today. Because one company would make their thing and say, this is it, this is the be all end all. But it's not. No one has the solution here. Yeah, there's big players, but those guys don't have all the answers. And what's really cool is I get to go a lot of game jams and do things like that. It's not just talks on stage. It's not just talks with high-level people behind closed doors. It's also I go to game jams and I get to see this stuff getting built firsthand. I go because I'm a Unity expert. They want someone on hand who can just answer questions really quick and give some feedback and ideas. And as I said, I'm a dog fooder. I live in Unity. And so I've seen and done a lot more than most people have in the engine. And I can use that information and I can tell people how to do their thing better. But the cool thing is I get something out of it too. I get feedback. I get ideas. I come back from every game jam with like 20 gigs on a thumb drive of stuff people have tried. Not all of it's good. Most of it's not good. But everything has some little kernel, some little thing that's worth looking at. So I come back to my team, a bunch of artists and engineers, mostly artists, and I run them through all the stuff I saw. Here's how this one couple of people spent their weekend. They built a flying airship, for instance, with this crazy UI where you're pulling on cables and moving stuff around and moving giant sliders like steampunk airship style. That's, you know, just one example. I never would have built that myself. I never would have thought about that. But I got to sit with the team who tried it and they had a lot of fun and I had a lot of fun. And now I have a working example of a flying airship in VR. You may never come to anything, but a lot of times this information kind of trickles down in your brain and in the conversations you have with people just like this one. And they start to think about it, and they start to get ideas. And I guess the other aspect of that is some of the titles that are going to ship for this kind of 1.0 of VR, I was there with the first day when they started. I got an idea to make a soccer game where you just use your head to knock a soccer ball into a net. That's about to ship on the PlayStation VR. They're calling it a Headmaster now. I was there on the first day in Boston when they built it. Actually, when I say they, I mean one guy. One guy, Ben Throop, made this thing. And he's going to be one of the shipping titles on this new generation of tech. And man, that's exciting. You know, you kind of feel like you're in at the ground floor in a way. I mean, I didn't do it. I didn't build it. I answered some questions along the way, but I got to see and try something that I never, ever in my life got to see and try. And I wouldn't have been able to do it if I didn't go there. And the cool thing is here in a few months, everyone's going to get to do that thing. And that's just one example. I have, man, a list of examples of this.
[00:07:23.462] Kent Bye: What have been some of the more compelling mechanics that you've seen within VR since you've seen so much?
[00:07:29.344] Pete Moss: What I guess I'm excited about, especially as the development happens in the hardware, is what do you do with your hands? How do you use your hands? What are controller options available? So there's some things like the Valve and HTC Vive. has hand controllers. Okay, it's just some triggers and touchpad, things like that, and they're positionally tracked, but man, that gives you a lot. So much. Oculus has the touch controllers. Different take on it, but again, you can do a lot with it. It's not exactly using your hands, but I think the cool thing is how people are taking this tech and coming up with extensions, not always extensions that you might imagine. And where is that going to go? I don't know. But there's a lot you can do. And how do you extend the performance or the experience? How do you stay in VR longer? How do you not make people sick? I think all of these things are really at the forefront right now, but especially for input. Input's a hard, hard problem. It's easy to think as humans, because we think with our hands. We use tools. We're a tool using apes. But man, we're having a hard time figuring out the right tools for this. We're not going to know for years. I can imagine a future where it's all standardized, and maybe it is tracking directly your hands. What do you do when you need haptic feedback? I don't have an answer there. No one does right now. Maybe someone does, but they're not talking about it. So for now we're tracking controllers in space. They're basically magic wands. They can look like anything. And it's kind of cool to see what people are making them look like and how they're using them and how such a clunky piece of tech, because it's pretty clunky, let's be honest. It's not about fine control. It's effectively a club that you're holding in your hand. But you can still do some fine control with it in limited scenarios. So the more we gather these limited scenarios, the more we see relations between them, the more we see where the gaps are, and then the more ideas come from that on how we can fill the gaps and kind of generalize the experience so that all the tech can do similar things. It should be about, you know, what you're more comfortable with and what your budget is about. We're so far from that right now, but I could see that in a few years, easily, where the feature set is fairly consistent, but at the same time, there's kind of a strata and devices and hardware and cost and things like that.
[00:09:38.082] Kent Bye: You had mentioned that you've been going to different conferences and gatherings that are going beyond gaming. So maybe you could give us a little bit of a flavor as to where you see VR kind of starting to enter into the different groups that go beyond just gaming.
[00:09:52.350] Pete Moss: Yeah, I think that's actually a really good question. Unity has a gaming background. I have a gaming background. It's easy to see why this is so connected to gaming. I mean, we had this dream back in the 90s that VR and gaming were inextricably linked, but that's not always the case. So one of the areas that I'm seeing some uptake, not as much as I would have hoped, but I think we're still getting there, is architecture, is building, is construction, things like that. Imagine the ability to visualize your house and make a change and have a custom house. not drawn up weeks on a computer or drafting board or things like that, but literally constructed right in front of you. And you can walk through it and see it today. Change all the paint color, do all that sort of thing. We can do that. We can do that right now. That's, I mean, that's already there. Not a whole lot of people are doing it yet, but I think that that's a really cool area where we can actually use this virtual environment to build out our actual real environment to places where we live and work every day. The movie industry, I just spent a couple of weeks down in Hollywood talking at a couple of different events for how Hollywood is approaching it. Their needs are different. Their needs are how do we capture live humans and what do we do when the idea of a frustum is no longer a real thing. The frame is huge in movies. What do you do when you don't have that? A user can go and look anywhere. How do you keep them into the story? What can you do more for stories that you can't do right now? Because we've got single points of view that maybe jump around, but at any given time, you only can look at one thing and whatever they tell you to look at. But in VR, AR, that's no longer the case. But what excites me the most, and I think the book Ready Player One gets into this in a huge way, is education. This is what I want to see more than anything, and I haven't seen enough here yet. The problem is, education never has money. It's terrible. It's horrendously terrible that we have built a society in such a way that educators are begging for money because we're all getting something out of it. The next generation, the real innovators in VR are in school right now. They're in elementary school. My kid is seven years old. I'm more interested in what he's going to think about what to do with this than what I can imagine because most of my imagination is still limited by when I was growing up and the things that I saw. So education, I think, is going to be important, not just for building cooler experiences, but also for getting better education. Imagine a rural community in Illinois or Africa or some other place where access to schools is difficult. It's a challenge. It's far away. Maybe once you get there, the teachers aren't that great. In VR, you have the potential to have the best teachers on the planet. And they may not be limited always to 30 students in a classroom, or 20 is always better. Maybe you can go to a lecture with a famous physicist, and you can be there and hear his voice and see him directly, but you're still living at home on a farm or something like that. So access to information, access to education, I think, is the thing that excites me the most, and I've also seen the least of. I want to see more there. I don't know how to make that happen. I'm working towards it. I think a lot of us are working towards it, but that's what excites me the most, I think, right now.
[00:12:45.680] Kent Bye: And I really see Unity as kind of like the lingua franca of the immersive technologies that are coming up from being able to build it once in Unity and be able to output it to all of the augmented and virtual reality systems. And so do you have a sense of all the different platforms that are going to be supported and, you know, kind of the market share for how much of are using Unity versus other options?
[00:13:08.337] Pete Moss: That's actually, I think I can answer some of that. So we have an estimate that we own I mean, as in our engine is used in at least 90% of the VR market right now. Possibly as much as 95%, but I think 90% is a nice conservative estimate. It's also still a huge number. We're used everywhere. And again, it comes out of this idea that people are already using us and the same people working in VR were the same people already using us. So in a way we lucked out. We wound up in this situation. I saw it coming in a way, and I've been trying to encourage people inside Unity, and several of us have actually. You mentioned Timoney West that you talked to. She's doing the same kind of thing. The cool thing is the organization is listening now. Money's flowing, VC money's there, so it makes it easier for them to listen. There's a demonstrated business case. The growth and all of this that Unity is a part of, I think it's critical. Where is it going to go? I don't know, but I know that we're going to do it in typical Unity fashion. We're going to make it as easy as possible for our customers. We're going to make it a single checkbox that will target any platform. We're not there yet, but we're working in that direction. We've already made several announcements with business partnerships and things like that to help make it easier for end users and end customers. But at the same time, we've got other announcements that we haven't announced yet. There's more coming. We're rolling it out all the time. There's always improvements to be made. And I think what you're going to see, like I said, typical Unity fashion, it's just going to work, it's going to be easy, and it's going to support whatever. We're not there yet. We're not going to be there next year. But we're going to be further along. All the equipment isn't out yet. Not even all of it's been dreamt of and built, even a prototype yet. So we're supporting what we can for now, and we're adding to it as we go. But who knows? Maybe Oculus is kind of, the fanboys love Oculus right now. They think they're going to be the winner. Maybe they will. They're going to be a winner. But maybe the real winner hasn't even built a prototype yet. We don't know. We don't know where this stuff's going to go. It's going to evolve, and it's going to change rapidly, at least for several years. I know that much. So we need to be on top of all of it. That's why I like to go to shows like this. I want to see what everyone's doing. I don't care what you're into, because I want to see where the industry as a whole is going.
[00:15:07.916] Kent Bye: What do you see as some of the biggest features upcoming for virtual reality when it comes to Unity that are yet to launch yet?
[00:15:15.080] Pete Moss: Well, we need to improve our input system. I mean, I'd like us just to delete our current input system and just start from scratch in a way, especially with thinking about how do controllers work, across all systems, like what does it mean when you have a track controller or things like that. So I think that that's one of the biggest areas. We're getting better at rendering, we still got a long ways to go. Our position in the industry is actually quite good because we work closely with the people, the movers and shakers and graphics. In a way we are one of the movers and shakers in graphics. So we get to help guide the industry a little bit, but we also get a lot of feedback from the industry so that we can work a lot more closely together. Clearly VR is based on video, but it's also based on audio, so we're trying to help grow that area as well. It's probably the three pillars right now that are most important, video, audio, and input.
[00:16:03.702] Kent Bye: You had mentioned at one of the previous hackathons here in Seattle that you're working on ambisonics support within Unity. Maybe talk a bit about what you see, the importance of that.
[00:16:12.169] Pete Moss: Well, I think where we are right now is we have a really great audio plug-in format. So we can make it very easy for any type of spatialized audio system to work within Unity. You just tell it which plug-in you're using, if it meets a spec. we just send it data buffers directly and it outputs data buffers. What is it happening behind the scenes? I don't know. So a lot of stuff is using ambisonics behind the scenes. There's HRTF support built-in in Unity. We have a kind of a simple version, but there's much more expansive versions. Not expensive necessarily, but expansive. They can do more. We're trying to make it easy for those guys to work. So Unity has always been about providing the tools for people to make things better, not necessarily just the tools you must use. Most of Unity could be considered, here's a working example of things you could do. But we also try to make it easy for you to do something better if you have a better idea. There's some really good audio plugins out there right now. There's one called Real Space 3D that I'm using for a project that's by a company called Visisonics. It's amazing, it's the best one I've heard yet, but it may not last that way forever because I've been using an Oculus plug-in recently and they're using Ambisonics under the hood. They're still working on it, they're still building it out, it's quite good, but it's not good enough for my particular need right now, but that could be very different next year. So trying to make it easy for those guys to build something is a big part of it, but also make it easy for our customers to have options and things that they can do. And we understand that we don't always make the best application of our own tech. So we want to make it easy for other people to make the best application of our tech. There's actually a whole ecosystem there of people that make money on selling things on the asset store. Some people make very good living selling things on the asset store. There's a handful of people I can think of right now that are making a better living than me. and they're using the Asset Store, and they're filling some of these gaps that we've specifically left there. We did it on purpose, because the community is the most important part of Unity, and we want to make sure that they can be successful, not just if they're using our engine, but how can they make our engine better? We don't know it all. We know that. So we want feedback. We want to help. And sometimes we buy one of them up, and sometimes we provide a similar piece of tech to something someone else has come up with, because we see that it has superior value, and we want to make that easier for everyone to use. That's not going to change. It's not going to change for a long time.
[00:18:24.642] Kent Bye: Great. And finally, what do you see as the ultimate potential of virtual reality and what it might be able to enable?
[00:18:30.585] Pete Moss: Man, that's a hard question because I don't think I can imagine what the ultimate reality is for where this VR, AR stuff is going to go. Our ultimate potential. I'm excited by this idea that we have a new piece of tech. I'm an artist. I'm a trained artist and composer. And you might think I'm all into computers, and I am. That's not my trained background. I'm actually a classically trained composer, weird enough. But what I know is that as new tech develops, it gives us new pathways into the brain. Perspective painting back in the 1600s is one of these. They applied math. They took math. They didn't know all the math, but they were working it out. And the same people that were physicists and inventors of the time were also painters and artists. There was not this division, this artificial division that modern society has placed on it. I'm an artist. I'm also a tech guy. I go to events like this and talk a lot. I do dev relations and that sort of thing. But what I do know from all of that is that there's pathways into the brain. It's all it is, if you think about it at the low level. VR is a new pathway into the brain, and it's going to allow experiences, many of which we haven't even begun to dream of yet. I have some ideas on where things might go, but I'm probably wrong about all of them. We're all probably wrong about all of it, but it's okay. We don't have to know it all right now. We do know that this is a new pathway. It's a new way to experience. All experiences that humans have are mediated by our perceptions. And when you start playing with the perceptions and have new perceptions, you have new ways of coming up with stuff. I'm interested in what the gamers are doing and what Hollywood is doing and all that, but what I'm really interested in is what are artists going to be doing with this? Because artists, like pure artists, are the best placed people to make the real future, the new perspectives on existence and reality. possible, and this is a great tool to help with that. That's what excites me the most. I almost kind of want to be frozen in a tube for 50 or 100 years just so I can pop out and see where things have gone. I'm not going to make it that far. None of us are. I'd like to. I'm interested to see where this goes. You read about it in books. You read about it in things like Ready Player One, which I mentioned. Or, you know, Snow Crash is the classic example. You know, how does this stuff integrate? What does it change about existence and daily life for people? And it's helpful. It gives you an idea of where things are going. But the reality is none of us know. None of us know. Man, it'd be great to come back in 100 years, 200 years, 1,000 years. Maybe this is gone. Maybe it's all a blip. I don't think it's going to be. Maybe it will. Who knows? None of us can say. But there is a zeitgeist with this. There's a lot of devotion. There's a lot of attention. And I think when you get this level of participation by all these disparate elements in the world and start to work towards a goal, it's viable. It becomes something. Yeah, we've got to commercialize. I've got to make money. Everyone's got to live. And they've got to eat. That stuff doesn't excite me as much as the artistic potentials of where this stuff could go. How could this change the nature of my existence? I don't know. But I'm excited to find out.
[00:21:21.565] Kent Bye: Awesome. Well, thank you so much.
[00:21:22.993] Pete Moss: Yeah, thanks for having me.
