Pete Moss is known as the “VR Dude” within Unity, and has long seen the potential of virtual reality. He’s really been on the frontlines of this virtual reality revolution over the past 3-4 years creating a bridge between the hardware manufacturers and content creators. I had a chance to sit down with Pete at the Intel Buzz Workshop a few weeks back to let him reflect on the consumer launch of virtual reality, and some of the emerging trends that he sees. VR locomotion is still a big open problem that he’s actively researching, and he encourages people to reach to him if you have innovative ideas. He also shares his perspective of the future of audio in VR, and the vital role that VR artists will play in expanding our minds.
LISTEN TO THE VOICES OF VR PODCAST
Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to The Voices of VR Podcast. Today is episode number 400 of The Voices of VR Podcast, and I wanted to feature somebody who has been a little bit on the front lines of this VR revolution over the last number of years, and that's Pete Moss from Unity. He's on the creative content studio for Unity Technologies, and so He's constantly making new demos to show to different businesses to show the potential of Unity as a technology, but also what's happening with virtual reality. And so he's been a longtime evangelist of VR and gets to see a lot of the latest technologies. And so we'll take a step back and kind of reflect about the consumer launch of VR that happened in 2016. We'll talk about audio, so continuing on with this little mini-series about audio and what the future of audio is in Unity. dive into Pete's more artistic side and his background and fine arts and what kind of insights and reflections that artists could provide onto this new VR medium. So that's what we'll be covering on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. Today's episode is brought to you by The Virtual Reality Company. VRC is at the intersection of technology and entertainment, creating interactive storytelling experiences. The thing that's unique about VRC is that they have strategic partnerships with companies like Dbox, which is a haptic chair that takes immersion and presence to the next level. So they're making these digital out of home experiences for movies, studios, and original content. For more information, check out thevrcompany.com. So this interview with Pete happened at the Intel Buzz workshop that happened in Seattle on June 22, 2016. So with that. Let's go ahead and dive right in.
[00:02:01.204] Pete Moss: My name is Pete Moss. I work at Unity Technologies in our Seattle office, which is actually in Bellevue. And I work on demo content creation. So I'm part of a team called the Creative Content Studio. And we basically build the coolest stuff we can and try to break the engine as much as we can too in the process so we can find problems before they hit the street. But I get to work in a lot of the VR content space as well. Given the background that I come from, where I've got a lot of the pieces under my belt, and also the location in the Seattle area, there's just so much going on in the tech. So it's partly I'm just lucky to be in the right space at the right time, so that I have the access to really keep my finger on the pulse of what's going on in the VR space right now.
[00:02:46.186] Kent Bye: Yeah, I think the last time that we talked was back in November at the Immerse in Seattle, and since that time we've had the consumer launch of both the Oculus Rift and HTC Vive and Samsung Gear VR even, and so we have consumer VR now. So from your vantage point, what are some of the big things that you're noticing and kind of paying attention to right now?
[00:03:07.266] Pete Moss: Well, yeah, we have consumer VR. I mean, that alone is a pretty major point. It's easy as a dev or kind of someone who's been involved behind the scenes for a long time to get jaded about it, you know, like, well, OK, it's finally at the door, but that stuff is already a year old to me, you know, that sort of thing. But the beauty is, this is really, it's a historic moment. I mean, it's, I have friends, and this is the part that I think really touched me about it, is I have friends who aren't devs in the same way that I am, and they're not deeply involved in the space. They may make games, they may work around computers or something, but They're not necessarily directly connected to VR, but they bought an Oculus Rift, or they bought an HTC Vive, and they've installed it. And what's really cool is I get to relive the experience and the excitement through their eyes, because they'll write to me like, hey Pete, have you tried out this game? Oh my god, it's so cool, and you can do this and this and this, and I'm like, yeah man, in fact... I know those guys. You ought to see what's coming next. I mean, it's going to blow your mind. But at the same time, just to see that excitement and to see how people are living it allows me to relive it all over again. It takes me back to the first time I got to go to the valve room or the first time I got to try an Oculus Rift or the first time I got to try some fancy like haptic controller or something like that. You could always see the vision back then of where it might go. But what's really amazing is We're there, we're there, and other people are picking it up. People that aren't in this inner circle, they may be close to it. It's not like the average Joe is picking it up just yet, but we're getting there. And I think that that's one of the most exciting parts in the last year. A lot has changed, but at the same time, there's a lot of the same focus. Making the situations better, building out companies so that you can build really cool teams, getting some of the ideas mature so that you can start to get them out into the market and see how people react to them. All those are really important and are going to be ongoing. But again, this is a magical time right now. There's never going to be another 2016. There's never going to be a first year of real VR for the masses. This is it. This is a pretty special time.
[00:05:05.742] Kent Bye: Yeah, I think the really exciting thing to me is that there's all these new genres of games and experiences that are going to be coming out with VR. And so, from your perspective, what have been some of the most striking or surprising experiences that you've had?
[00:05:18.332] Pete Moss: I'm looking forward to good cinematic experiences. I think there's a lot of growth to do there. I've been to Hollywood a few times now. I've talked to a bunch of groups down there. It's interesting to see what their focus is, but they haven't explored enough, I think, just as a group. I don't want to lowest common denominator it, but I think overall Hollywood has a lot of growth and it's still really attached to this idea of 360 video. I want to see them move beyond that. But from the gaming side, again, it goes back to I can actually buy games for hardware that I own at home, which is pretty amazing. And one of my, I guess, my most exciting games that I've played not enough of this year so far is Hover Junkers. I was really excited about that one because it had the potential to solve the FPS problem. I think a lot of VR developers early on, especially early on, like four or five years ago now, looked at it as, well, we could just do a first-person shooter game, we could put a camera in the head of the main character, you can run around at 90 miles an hour and you'll be fine. And that's like the number one thing you cannot do, we learned early, early on. What's interesting is Hover Junkers basically turned that on its head. And you can run around at 90 miles an hour and shoot at your friends and have a good time and kind of have that kind of fun, but it's not in the way we expected. They're mediating it. You've got a platform that you're flying around on. The way the controls work for it are really, really smooth. And I think that that's a testament to that studio and the work that they've done. But it is interesting because it's one of the first experiences that we learned was bad. And we kind of, a lot of people wrote off. I know I did. Like, oh yeah, you're never going to do a running around shooting game. But they actually have done it, and it's quite good. It's fun to play, and it works really, really well.
[00:07:00.681] Kent Bye: Yeah, one of the things that you told me that really stuck with me the last time we talked was that you're really interested in seeing what artists do with the medium. And, you know, one experience in particular that I really found enjoyable is a VR creator named Kibibo with Blarp and Loon. And I think a lot of these more abstract, you know, irrational exuberance is another example that I can think of. that is not necessarily a game per se, but more of an experience, but also with some game-like components. And I'm just curious, from your perspective, what kind of, like, you know, pure artist type of experiences that you've seen that are really striking to you so far?
[00:07:35.100] Pete Moss: You know, I definitely want to see more in that space. I am an artist. My background is in studio art and interactive tech-based art, sound art, and a bunch of related things. What I think is fascinating is that, yeah, there's, like, Loon just came out this week, I think, in Steam, so everyone can get it now. It's only a couple bucks, you know, it's super cheap. Like everyone has to go by this and try it out. It's funny because we're here in Seattle right now, we're in the downtown Pioneer Square area and there's a lot of art studios around here, a lot of galleries. I've exhibited at galleries just around the corner from here. So I've seen artists go the gallery approach where they're installing hardware, it's basically a white room you walk into, there's a black headset on a table. You put it on and you're somewhere else. And I think that's really cool. I definitely want to see a bigger push in that direction of this kind of publicly accessible VR type experience. But I'm even more excited by the idea, like the Kibibo thing, where you can release on Steam or some other platform, you can release your work. You don't have to control it in a gallery setting. You don't have to control it like it's only available during this time. You know, there's a Bjork example that's out right now. You have to go to one specific museum and during specific hours to try the thing. I want to do this. I want to do it. But I, you know, it's involved. I haven't done it yet. I'd much rather be able to go home, like I've dedicated a room. I'm one of those crazy ones who's like, yeah, I got a room. I'm ready to go. I'd have room scale VR going at home. I want to experience those artistic endeavors in a place like that. I want to be able to bring art into places that it hasn't been. People are used to going to galleries to see things. But I'm much more excited about the potential in the future to directly connect as an artist to people and places that you can't possibly access, and to allow access into your world for people who are very remote from it. There's this, I guess, kind of a negative connotation about art, especially from non-artists or from people that aren't involved in art circles, that it's elitist, that it's kind of special. I've got some friends that have repeated the refrain, oh, I don't get art. It's not a thing that I do. I think that's kind of BS because art is not special, it's not alien, it's not inaccessible. I mean, yeah, clearly there are examples where that's true, but that's what we call bad art. Art in general, like the big umbrella term, is a human function, it is a human activity. And getting it into your home and getting it into, you know, as AR starts to come down, you know, being able to see art in augmented space, I think, is another real potential avenue that I don't think anyone's explored yet. I'm not aware of anyone exploring it yet, but I hope that people are. Because that's the frontier we need to cross, that's the place we need to go. And it'll start to bring multiple worlds together. And maybe, hopefully, and this is one of my dreams of public art in general, is start to get through to the folks who don't get art, who claim to not understand it. Because I think a lot of it can be understandable, but you have to have a frame of reference and a way of approaching it. And I think that VR and AR is a great frame of reference for that. It's a great way to get your foot in the door in a way that you never would have been able to before.
[00:10:41.790] Kent Bye: Yeah, you mentioned something on the panel about just coming back from E3 and kind of seeing some of these big AAA games kind of like as an afterthought, adding in virtual reality as a straight port from a 2D game. And so maybe you could kind of elaborate on that point there.
[00:10:56.933] Pete Moss: I think the point I was trying to make, and this theme has been going around a little bit, but I think it's a really valuable thing to look at. So the point is Sony put out some demos based on some of their large franchises. So there was a Resident Evil one, there was... Final Fantasy. Yeah, Final Fantasy was another. Now, these are pared-down experiences. They're not necessarily the full game. But then again, Bethesda's saying Fallout 4 is going to be VR-ready this year or later this year. So, hey, that's great. I love Fallout 4. I want to try it. You know, I'm all over trying it. But the fear is that there's a little too much siloing going on in organizations like that and possibly some arrogance in that they have this experience, they're well-known, they're AAA developers. They know how to make their game, but the problem is VR is a new medium and it's going to present new challenges. When you're so siloed off, you may not know that you can't just drop a camera on the first person player's head and it's all good. You have to discover that. What I think is really interesting about that is It's not about Sony making a blunder. That's fine. Make mistakes. Everyone should be out there making mistakes. You shouldn't be afraid of it. So kudos to Sony for showing that and taking a few punches on the chin from people complaining about it. You know, we are a vocal group in VR, and if you're not doing it right, you're going to hear about it pretty quick. But what I think is really interesting is the hope that it can give to a lot of indies and a lot of folks that maybe psychologically don't feel like they're capable of competing at the same level as like a Sony game studio. But nothing could be further from the truth. Because of the connections, because of the community, because of all the activity and excitement, indies are actually doing really, really well right now. They're an added advantage. not a disadvantage when it comes to making VR content, because they've gone through the experiments, they've gone through all of the work, and they've tried things and they've failed already. So, you know, you get those failures out of the way, you kind of move along, and then you can go to create some true success. So I look forward to something great from Sony, I look forward to something great from Bethesda, but it's kind of clear that they still got a little learning and growing to do. And part of that is the problem of just trying to port an existing title to a new medium, a new platform. It sometimes works, but more often than not does not work, especially without major rework. So kudos to them for trying, but I think a lot of us are rooting for them so that they can really succeed in that area. I mean, if they succeed, we all succeed. But again, we don't need them to succeed right now because there's already a lot of success in the indie scene and in the kind of young VR scene. And that's really exciting for me.
[00:13:28.806] Kent Bye: Yeah I think the concerning thing to me is just the kind of the public perception of right now there's a disconnect for some people in terms of the understanding the unique affordances of VR and then kind of having this idea of what would be cool in VR. Like people think that it might be great and really cool to go and run around in an open world like Fallout 4 but yet you know some of the unique affordances of VR are kind of pointing to for some people that's going to be a terrible experience in terms of motion sickness and just not comfortable universally across all players. For some people, they may be totally fine. So we're kind of in this situation where it could poison the well in some sense of having people's expectations and then them trying a game that they think is awesome, but yet, you know, it could be something that is designed from the ground up for VR. Something like Werewolves Within, which is Ubisoft's, like, social VR game, which is, you know, it's clearly not a port. It's something that was designed with the social interactions in mind. rather than trying to take these other big games that people have this expectation of, but yet they really don't know because they haven't had a chance to actually try it yet.
[00:14:29.063] Pete Moss: Yeah, I agree with that. I think what's interesting about, say, Bethesda or Sony or a similar large company, you know, Bungie's eventually going to jump in, I'm sure, at some point. There's a lot of IP, there's a whole universe they've already built, sometimes through multiple games. It's fine, but what I think they have to understand is that you can't take the same game mechanic and just slap it onto this new medium and have it work. But what they can do is they have a rich universe they can build on. They have a lot of assets, many of which are VR ready right now. They're already good 3D high quality assets. What you need to do is take the step back and think about what you're really trying to build. And there is a chasm there. There's this idea of, well, here's what I think would be cool. And there's the other side, which is, here's what really works. And they don't always cross over. In fact, there's usually a gulf in between them. What we find, especially with the experiments, is we try the, well, this is the way that we should do it. This is the way that this is going to be a really cool thing. And then you try it, and it's horrible. It makes you sick or worse. You're fine, and you think it's good. because you've got a cast-iron stomach, but then you show it to everyone else and they all get sick trying it. It can be really defeating as a developer to see that experience. But that's okay, because that's not the important side. It's not about what we think will be cool, it's about what we can make that will be cool. And that's the other side. And that's where I think we need to get to, and that's where I want to see more people get to. is get away from what they thought about as cool back in the 90s or the early 2000s when they first got into it, but what is really viable now. I think if we knock out some of these viable things, we're going to have a good time and that'll help lead the direction for where we're going to go. But, you know, it's funny when you speak about it in a large open world. I love open world games, man. I run my own ARK server at home. I play with a bunch of friends, a bunch of Unity peeps. I'm crazy into Fallout. I harp about Elite Dangerous all the time, which is perhaps the largest open world game in existence right now. It's four billion stars or something like that modeled. All of that's really cool, but the more you move around in a large space, this is kind of a problem I'm having right now. In fact, I'm doing a bunch of my own locomotion studies, because you have your room-scale experience, you can move around freely, but you also have Blink. Blink is great, but Blink is also kind of the devil, because what I found is that all the games that rely on Blink I don't care that I have a large room to move around in. I don't do it. I stand in one spot and I blink everywhere I want to go. So, in a way, a Fallout type experience where you're blinking your way across the wasteland instead of just walking across the wasteland. Like, you don't want to walk. You don't want a joystick to move your character. That's nausea city. We know that. I don't know anyone that solved that yet. I don't know if it's solvable. I don't believe it is currently. We'll see. Blink obviously works quite well, but what you find is now I have this body feeling. That's why VR is so great. I can put my body into the experience. But we're in a situation now where I can move over vast distances, but I literally don't feel my body moving. I feel like I'm standing in a room, and there's a disconnect that's forming there. A lot of people have glommed on to this Blink idea for locomotion, but I'm already starting to hate it. I'm already starting to want to move beyond it. It's valuable in some ways, and I've certainly written my own stuff, making lots of use of it. And it's interesting how people are kind of settling down into this very similar interfaces, you know, the arc and everything, so you can see where you're going to land and that sort of thing. But we still got so much to do there. And I want to have this feeling like I'm walking through the world. By the same token, I don't want to walk all the way across the wasteland because it's very tiring. And, you know, video games have an advantage in some way that my character can do physically much greater things than I, as a human, can do physically. So finding the right experience, finding the right way to balance it is a big challenge, and getting the right mix of physicality in there. It's still so early. It's so much easier to identify problems right now than it is solutions. But I think as you work through the problem space more and more and more, the solutions start to present themselves. I'm looking forward to solutions that we haven't even dreamt up yet.
[00:18:27.322] Kent Bye: Yeah, I do find that the Blink does break presence. It's comfortable, but yeah, it does kind of take you out of that and kind of break the embodied presence that you have, as well as the feeling of time passage, because you kind of have this estimation of how long it's going to take to walk from one place to the next. And when you start to teleport around, then it kind of starts to skew a little bit with your own sense of time perception, I think. part of the magic of being able to explore an open world is to be able to have some of those other indicators of time passing, which is your ability to estimate how far it takes to walk through different areas.
[00:18:58.682] Pete Moss: There was actually, I was watching some YouTube vids the other day. I've been watching everyone's locomotion stuff lately. In fact, if you got some that I haven't seen, throw them my way because I'm eating this stuff up because, again, I'm trying to learn as much as I can from what other people have already done. And sometimes they do it, they're like, yeah, this doesn't work so well, but you know, you see a little kernel of something in there and you're like, well, it doesn't work in that case, but if I change it this way, I might get a little more. So for instance, I was watching one the other day, it was actually, it was a locomotion comparison video. It's been going around, so maybe you've seen it. The guy was describing a modified blink. I think the game is called Spellfighter, but I'm really hesitant. I'm probably wrong in that title, so forgive me, listeners, if I'm saying it wrong. But the idea was he had a blink mechanic, but instead of instantly teleporting there, there was an avatar that would walk and human speeds would walk from where you were to that point. And if you interrupted the blink halfway, that avatar, wherever he was at that moment, that's where you teleported to. So you didn't even always complete the blink. But what I like about that is there's this aspect where you start to see that avatar is you. There's another game out in Steam demos right now called Runes, I think. And they do this too, where you drive your avatar around to walk around and then you kind of astrally travel back into your own head sort of idea. There's an interesting body connection like all those games that I played like that I feel like pretty quickly like that's me. I'm over here with my eyes but clearly I'm looking at the back of my own head walking across the floor. So I think it's cool in that regard and has a presence opportunity that isn't really well explored yet but also has this idea that it's bringing real-world time back in like it takes time to cross landscape and one of the dangers is if you're going to do a blink mechanic like going back to the wasteland idea you could blink blink blink blink blink blink and move a thousand kilometers you know in a very short amount of time just by spamming the blink button. It's not a good game mechanic in that regard because now I'm effectively able to evade every danger in the world. which that's not what that game's about. That game's about getting right into the thick of it and trying to use your wits and some skills and maybe slow down time from time to time to get out of these scenarios, not just being able to spam a game mechanic that has been crafted on after the fact like a blink mechanic.
[00:21:08.720] Kent Bye: Yeah, a game like Thunderbird which is trying to really constrain you to a single space and not having any teleportation or blink, I can really tell the difference of going back to that type of mechanic that it really does increase your sense of presence and so that's another example. I just wanted to move on to audio because I know that you've been concerned into ambisonics and you know since last time I talked to you I've had a chance to talk to Dolby about their Atmos system, which is a little bit more of a proprietary system for a certain format that they have. That's usually more aimed towards, say, like a 360 video. I'm not sure if it, you know, something with Unity you'd be able to actually kind of recreate that sound field. But it seems like a way to be able to take live recordings and be able to remix them in certain ways. We have also hardware with AUS-X and being able to have a little bit more higher fidelity, being able to actually measure the HRTFs in your ears and then perhaps give you a little bit more high fidelity level of sound, kind of like the Oculus Rift of headphones. But I'm sure that there's also new things coming in the Unity engine in terms of ambisonics and support moving forward. So it seems like we've got the visuals down with Oculus Rift, but I think that, you know, in my mind, audio is going to be the next realm, and specifically even talking to Ming-Ling, who at University of North Carolina, talking about, like, Moving towards a future. We're actually simulating the sound as well rather than within the recorded sound objects But actually using the physics at 48 kilohertz or whatever rate it's going to be actually simulating the sound live So from your perspective, I'm just curious to hear your kind of state of the audio and where that's headed
[00:22:40.332] Pete Moss: Yeah, I think, so that concept of creating the sound from scratch, they call it physical modeling, and my background is heavy in audio. I worked in audio engines called CSound and SuperCollider, and I've worked with a lot of spatial audio in the past. I mean, a lot of my art history revolves around that. So yeah, I'm clearly really excited. A lot has changed in the last year. I used to do talks and kind of have a slide of, okay, here's what we really need to be focusing on. And one of them has always been audio because people never think about it early enough. Movie makers don't think about it early enough. Game makers don't think about it early enough, but audio sells the space. And when you're talking about an environmental thing like VR, It's not just enough to have this like mental model that I'm in a space by turning my head and looking around, but if you don't hear it, you're not going to get the same sense of space. In fact, you can kind of make it hyper real in a way, make it bigger, make it stronger. than you would, and movies are going to want to do that. Games are going to want to do that too. But yeah, the tech's evolved quite a bit. One of the cool things in Unity right now is we've got a plug-in interface where you can drop in a spatialization plug-in, and it works almost seamlessly with the engine. There's, you know, improvements always to be made, but Oculus has a spatializer plug-in. A lot of folks are using that. I know the guys at Visisonics, and I've been using the RealSpace plug-in myself. For me, it's the tops in sound quality right now. It may not always be the case, but I've really been enjoying working with them, and I've given a lot of feedback from my point of view of this is what I want, but this is also what I think developers are going to want. So how about we help advance your tool, and they've been very, very open in working with me. And we've also been working with the Phonon plugin from It's not Two Big Ears. No, Two Big Ears, that's an interesting one that got brought up. They got bought by Google. It shows me that Google's thinking about some stuff. So I want to see advances like that, and we're starting to see them. It's not to say that just using a 3D plug-in means it's all solved. Honestly, the plug-ins still got some growth to do, all of them. But it's a start. People are thinking about it in a bigger way. And I'm starting to see some really cool examples pop up. But it's one of those things that you don't have to look at an environment to know you're in an environment. You use your ears. And the more people start to realize that and pay attention to that, the better off I think we're all going to be. So, like I said, it's a beginning. I'm actually feeling okay these days. A year ago, I was like, man, no one's thinking about this. They all need to be thinking about it. What is going on? But the reality is, it is happening. And, you know, with Phonon, or with RealSpace, or the Oculus plug-in, or even just rolling your own. I mean, there's a lot of other options that you can do. But I want to see it easier. I want to make this turnkey in Unity. I don't want you to have to think about it too much or go get someone else's stuff. We should have a pretty good, built-in, easy-to-go method from the start. It may not be the best. That's okay. We want to enable people in the asset store and our customers to have greater success than we could give them with the tool. So I like the openness and I want to keep that, but we still got a ways to go for sure. It's a great start. It's a great start.
[00:25:44.832] Kent Bye: In terms of audio hardware, I know that I can think of at least a couple, which is the AUSIC headphones, which were kick-started and will be coming out within the next year or two. We have the sub-pack, which is being able to translate the low frequencies into kind of a visceral haptic feedback in your body. Are there any other hardware audio peripherals that are worth noting?
[00:26:04.270] Pete Moss: I'm looking forward to trying the AUSIC. I haven't done that one myself, and I'm very eager. I do have a couple of subpacks, and I will say, at times, yeah, you feel like you're wearing a backpack, but once it starts rumbling in a way, it melts away really quick. We've been experimenting with it quite a bit for some of the higher-end demos that we do. I'm very excited by that tech, this wearable tech. I mean, and that's the key, is it's not a subwoofer in the room with you that everyone can hear. it's a thing that's vibrating your chest, and it's gonna enable sort of a haptic response that you're not gonna be able to get in any other way, and it's gonna make things more visceral. But no, I think there's a lot more. There's the headphones that I think Samsung's working on that, you know, they shock you in the back of your ear, basically, to do this vestibular response so that they can make you feel like you're turning, and I'm actually really excited by that idea. It sounds weird if you just say it out loud, but at the same time, this idea of, managing your vestibular system so that you have some control and actually reduce nausea. I'm super thrilled by that idea. And of course you would have to do it through headphones because that's where all that stuff's located. It's all in your ears. But by the same token, there's bone conduction techniques that folks are working on. So it's not even really speakers that are driving. You just kind of, maybe it's in the HMD and it just vibrates your head in such a way that you hear the sound in your ears. but there's no actual air vibrating to make the sound. The hardware is very, very early, even earlier than the VR hardware right now that we have for visuals, so I'm excited to see where it goes. It's an area that's close to my heart, so I watch the space a lot, I browse game audio lists, I look at hardware makers and try to keep up with what they're doing. You know, of course, the best thing is just getting as many demos as you can, try it out, see what it works. And, you know, call them and see if you can borrow some hardware for a while. I called the guys at SubPAC because I wanted to try it out with our demo, and they sent me two. I'm at Unity. There's maybe some special access available that way, but there's still also a lot of openness, and, you know, you never get what you don't ask for. So the more you can demo and try to find ways to apply it to what you're building, the better, and the better for us in the industry, because we get more examples and more things people have tried. and hopefully more mistakes people have made that the rest of us don't have to make or make as often.
[00:28:20.811] Kent Bye: So when I think about, from the perspective of a VR designer, there's really kind of like five different approaches that you can take when designing a VR experience. Either it's mobile, where, you know, we have Daydream or Gear VR, sort of in the same class. You have desktop VR with Xbox 360 controller. You have full room scale with 6DOF controllers. You have beyond room scale, which is, you know, something that is going to be an installation or something that may not be in people's homes. And then you have something like augmented reality, which is sort of similar, but kind of an entire new class of its own. So from your perspective, are there other kind of design decisions that are different from a VR gaming perspective?
[00:28:57.980] Pete Moss: I think, yeah, there's a lot of design decisions that you need to pay attention to and a lot of it does indeed depend on the hardware and the exact presentation style. So something that is rotation-only, there's a million things I'm not going to ever be able to do. Also, what's interesting about rotation-only systems like Cardboard or Daydream, whatever you want to call it now, or Samsung Gear VR is, yeah you have rotation and the graphics look pretty good, but you're limited in being able to look around objects, but also there's no motion controllers, you don't have hands, there's a lot of limitations there. What I want to see and what I think people are working towards, I know Carmack is pretty famously working on this right now at Oculus, How can we take the mobile solution and add things like inside-out tracking so that we get the sense of positional in the world? You're not going to do it with boxes with lasers shooting out because how are you going to walk down the street with that? I don't know that you're going to do it the way HoloLens is doing it with a bunch of spatial reconstruction camera systems, but I think that approach isn't bad. I don't think you're going to be doing it with your phone either. I think what you're going to be doing is a headset that maybe is built on phone guts, but has some extra stuff in it, closer to what HoloLens is right now. Yeah, I like my sunglasses. They're narrow and light, and I'm probably going to have to build something really bulky in the near term. But ultimately, it'd be great if this form factor worked. I could see partly or completely, or maybe it's fully opaque. Where's the line between VR and AR? It may become very, very blurred in the future. And especially if you have an inside-out solution where I'm tracking where I'm at, I can look around objects. All of those are going to be really good. So I think down the road, It's all going to be a lot more the same than different, but it is different right now, and it's going to make you build stuff in different ways. And we may get to a point where 360 is getting tapped out. I think his name's Chris Milk was talking recently about this idea that, you know, he's tried a lot, he's done a lot, and it's really cool stuff. But at a certain point you start to run out of ideas on what you can do with a 360 video, right? Like at a certain point you're just ahead on a swivel in a room and the room seems a little larger than it should be, you know, like scales off. So I'm eager to see what Hollywood folks do with that medium. I don't think that that's the medium. I think real true VR with positional tracking the ability to move around objects, the ability to maybe place them physically in the world and lock them to a location so that that game is running at home in your living room always or it's a game that you only play on the bus and it knows when you're on the bus and draws dragons flying along with the bus outside, I don't know, just some crazy idea. It's going to boil down to situations like that, like what do I want to do and how do I want to present it and It's not even, is it full VR, is it full AR? This digitally mediated world idea, I think, is where it's really going to be. That we're remediating all of our sensory inputs through this digital interface. And that includes sound. That includes blending sound with what we hear on the street, with what we're hearing in the world. I think HoloLens is moving in that direction, but we need all the creators to move in that direction. And we need to make the hardware a whole lot better than it is right now. And that's across the board. That's not just AR hardware, which is a little newer on the scene, but VR hardware still has a lot of growth to do. Clearly we want things to be mobile. Removing the cable from the back of your head is great. We got to do it. There's a whole lot of hurdles though. And what's interesting is the position that I have. I get to meet with people kind of behind the scenes and learn more about the hurdles and meeting with some of the Hollywood folks and talking about wireless transport for wireless video streams. Yeah, they don't know. They don't have it worked out yet. It's not like there's some solution waiting in the wings about to come out. But now that we have a mission, those solutions will start to present themselves and be more obvious. So I want to see that. And I think we will see that. But it's probably not going to be Generation 2 or Generation 3. It might be Generation 5 or 8. If you think about it as a year to 18-month generation cycle, We're still in Gen 1. We don't even know how long the cycles are going to be, right? I know folks are already working on Gen 2 stuff, but there's no deadline there. There's no calendar release date created at all. In fact, I have not even seen all that stuff yet. I mean, it's still very closed up. But it won't be that way forever. And we got a trajectory started as an industry. We're not where we're going to be. But all great journeys aren't about the destination. I mean, the best journeys in your life, your entire life, honestly, isn't about the destination. You know where you're going to be when your life's up. You're going to be dead. So it's not about getting there. It's about the journey along the way. And I think that VR, AR, this mixed reality concept, this expanded reality maybe, maybe that's a better term. I just made that up. So I'm going to trademark that. But this concept is the new thing, and how can we apply it, and what are we going to learn along the way? I think it's going to teach us how to build better worlds, but it's also going to help us understand our world. I was 16 when I first traveled outside the United States. I went to Europe as an exchange student. And I got to say, man, the best way you can learn about where you're at is to get outside of it and look at it from a different angle. And I think that technologies like this are going to allow us to do that with our entire human existence. That's what really excites me the most. That's what keeps me waking up and going to work every day. And that's what I dream about still. I've been dreaming about it for years. A lot of us have. We're starting to see some of those dreams come true, but we still got dreams. And what's cool is seeing the little parts of it come true just stimulates you and the dreams grow bigger or they morph and they become something else. And you're going to start having dreams that you didn't even know were possible. And that's going to take us to places that we didn't know were possible. It's an exciting time. Not too often you get to be in on the ground floor at a time like this. Awesome.
[00:34:49.716] Kent Bye: And just the last time I talked to you, you said the ultimate potential of virtual reality was like creating new neural pathways into the brain. And I'm just curious if you have any other thoughts about where this is all going.
[00:34:59.502] Pete Moss: Yeah, as an artist, I always think about that. Art's not an artifact. Art is the experience that you create, and it's the neural signals that you're able to create and modify inside other people's brains. It basically gets other people to come into your brain in some way. So, the ultimate goal of all art is to change your brain chemistry. I mean, in the broadest level, that's what it's all about. In fact, there's many modern artists that have taken that as their mantra. That's just what they're all about. So yeah, for now, we do it through the senses. We do it through vision, and we do it through sound. And touch is starting to become a thing, but it's going to take a while. But I think the fascinating part is that somewhere down the road, this matrix idea of plugging a cord into your brain is going to be what people want to do. I don't think that's what people really want to do, but I think if it could provide a more direct path, we may find a way to do that. But then again, we don't really need that. We've got a really good sensory network. Our imaginations are really rich, and the best experiences that we have in life aren't the ones that are handed to us fully formed, ready to go. They're the ones that excite something that allows our brains to start to run with it and create our own reality around this little nugget, this little kernel of some reality that someone gave us. And art clearly is the way that way. Film is part of that. Games are part of that. You know, I went to art school and I don't know that everyone who I studied with would agree that games are art, but I look at it that way. It gives people experiences that they would not normally have and hopefully causes them to think about things that they wouldn't have thought about otherwise. You can do that through games. You can do that through cinema. You can do that through drama. You can do that through a variety of methods. And I think VR is one of these ways that is going to be really, really popular and really accessible and cheap and easy, you know? I could train for a lot of years, or I could put on my VR headset and kind of do it tonight sort of thing. And I think that's exciting. Again, we have a trajectory. There's a path started. We don't know where it's going to go, but it looks like it's pointed in the right direction for now, and that's pretty good for now.
[00:37:04.461] Kent Bye: Awesome. Well, thanks so much, Pete. Yeah, thanks for having me. So that was Pete Moss. He's a part of the creative content studio at Unity Technologies, and he's had a front row seat of this whole immersive technology revolution over the last number of years. So I have a number of different takeaways from this interview, but before we get to that, let's have a quick word from our sponsor. Today's episode is brought to you by The VR Society, which is a new organization made up of major Hollywood studios. The intention is to do consumer research, content production seminars, as well as give away awards to VR professionals. They're going to be hosting a big conference in the fall in Los Angeles to share ideas, experiences, and challenges with other VR professionals. To get more information, check out thevrsociety.com. So I have a number of different takeaways about this interview. First of all, I kind of have to agree with Pete in that it kind of takes other people experiencing the consumer launch of VR to kind of reflect people who have been immersed in this technology for a long, long time, meaning that there's a number of different moments within the history of VR over the last two or three years that I remember. You know, people first receiving their Oculus Rifts back in the spring of 2013, and then getting to really fully experience the motion track controllers for the first time when the Vive came out. It was launched on GDC of 2015, and then getting to try the Oculus Touch demos for the first time with the Toy Box demo, which was a huge thing that happened at Oculus Connect 2. So there's these different milestones and moments, I think, within the history of VR that are much more memorable than the consumer launch. The consumer launch was kind of like just following through and actually delivering on the hardware. But I think it's important to take a step back and say, yeah, we made it. We have consumer VR out in the world right now. And it kind of takes people who are discovering it for the first time to reflect some of that childlike spirit and wonder and awe of experiencing this vast new world for the first time, for people who have been VR veterans for a while to really fully appreciate this consumer launch. I think, at least, that's my experience. And I think that I kind of heard that same sentiment from Pete. And the other thing that really sticks out in this interview is just that VR locomotion is still quite a problem, an open issue that I think that is probably one of the biggest things. Just today somebody asked me, what do you think the thing is like the most surprising about VR? And I think the thing that's most surprising for people to discover is that the locomotion and moving around in VR is such an intractable problem in a lot of different ways. Something like Hover Junkers is able to start to solve that problem by having a spaceship where you're flying around and you're able to get this sense of mobility and moving around in the space and to still have that first-person shooter kind of feel. And perhaps we'll just start to see more of those cockpit-based approach to be able to address some of the locomotion issues. I'm going to have a deep dive into VR locomotion with Jason Gerald here sometime within the next week or so. really talk about a lot of the research and a lot of different triggers and theories about what is causing motion sickness within VR. There's at least five major theories of why that happens and so we'll be digging into a lot of the high-level explanations of what's actually happening that's causing the simulator sickness. But I just really wanted to feature Pete on this episode 400 just because I think he's somebody who really has an inside scoop as to what's happening within the industry and has seen a lot of things and demoed a lot of things that nobody has even heard about publicly yet. And so he gets the chance to have a lot of early access to technology and start to play around with it. And some of the things that he was talking about in terms of these headphones and stuff, I'm not sure if that's actually been released or talked about yet. I haven't heard about it in the wider VR community, but I just may have missed it. A lot of new hardware technologies with audio and sound. I do think that audio and sound in VR is going to be one of the next frontiers of big issues that people are really going to focus on. I had actually done an interview with Two Big Ears talking about some of their audio specialization mixing platform that they had released. And the other big point that I just wanted to bring up again was going back to the first time that I did an interview with Pete back in November of 2015 at the CVR, which was the Seattle VR Expo. It's now being called the Immerse. But anyway, back in November when I did an interview with Pete, he said, you know, the thing that I'm going to be really interested in is seeing what the artists have to create. I really have to agree with Pete on that. And some of the most interesting experiences and interviews that I've done, I think, have been some people who are just kind of coming from a completely different perspective and able to take me to these other worlds that are just aesthetically beautiful or just are able to really transport me in a new way. A couple examples are Kibibo with Loon, as well as Blarp is a game that I really enjoy within VR. But also the Night Cafe from Macaulay is something that's very transportive into giving you this deep sense of presence. And so I, right now, am actually physically in Austin, Texas. When this airs, I'll be in New York doing a lot of interviews about artificial intelligence. But just wrapping up with this conference that happened with a lot of different illustrators, And I got up and gave a talk about VR to a lot of people whose primary medium are in ink and paper and 2D. And there's a lot of illustrators who are native digital users who have their entire pipeline digital now with different brushes, like Kyle Brush, which gives you this feeling of realistic paint strokes and just textures that are actually kind of difficult sometimes to tell the difference between whether or not it was created digitally or created in analog reality. And so all that just to say is that I'm super excited to see where people who are coming from a more artistic background to world build and create experiences that open up new neural pathways into our brain. So that were all my big takeaways from this interview. I really appreciate hearing from Pete and his perspective. And just know that he's got a little bit of an inside track as to where everything is headed. So with that, I just wanted to thank you for listening. And if you enjoy the podcast, then please do spread the word and tell your friends directly or tell a lot more people indirectly if you want to leave a review on iTunes. That would be awesome. And if you'd like to send a tip and just help support financially what I'm doing here with the Voices of VR podcast, then please do consider becoming a contributor to my Patreon campaign at patreon.com slash Voices of VR.
