Jason Storey (aka apieceoffruit) is a VR developer, Left-Handed VR podcast co-host, and all-around VR enthusiast who has worked on number of different VR experiences. He talks about working on the gaze-control mechanism in the cinematic VR experience of Colosse, which was a Silver experience award winner in the Oculus Mobile Game Jam. He’s also working on Echo Red with Nick Pittom, which is an experience that uses the Oculus Touch and Vive motion controllers in order to make you feel like a super hero. Jason has been able to work on projects ranging the full spectrum from passive cinematic VR experiences to the other end of the interactivity spectrum with full motion controls, and so he’s got a lot of interesting perspectives and ideas about the strengths and weaknesses of VR as a medium for expression.
LISTEN TO THE VOICES OF VR PODCAST
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR podcast.
[00:00:12.058] Jason Storey: I'm Jason Story. Most people know me as a piece of fruit from the Left Handed VR podcast. But today I have my developer hat on. So I'm actually working. I worked on Coloss for those who don't know. I was a programmer on that. And yeah, I'm currently working on a game with Nick Pitom called Echo Red. We're going to release some stuff next month.
[00:00:30.038] Kent Bye: Great. So yeah, talk a bit about what you're looking at in terms of motion controls, in terms of what you've seen and what you want to do with those.
[00:00:37.743] Jason Storey: Well, my one complaint with VR environment at the moment is everybody's doing a very sort of button-level interaction, kind of interactive worlds with a few things. But I want to be like a superhero. So my goal is to make a motion input game which uses a lot more visceral stuff, uses a lot of throwing and velocity-based powers and stuff. Our game, Echo Red, is using the kind of idea that you could use telekinesis, you can reach over, grab something, flies into your hand, you throw it in the air with your own velocity and then light it and fire and stuff. So I'm basically in line right now for the Toybox demo so I can play with the touch controllers and see if they'll suit my game better than the Vive.
[00:01:18.087] Kent Bye: So, yeah, tell me a bit about the first experience that you had with both the Vive and also the Toybox.
[00:01:24.213] Jason Storey: Sure, the Vive, basically I went over to London to do the Vive Jam with John from Crystal Rift and that was really fun so I got to try it out there and that was the first time I tried it. So my very first experience was the blue to the big whale and immediately level of immersion from having room scale is just mind-blowing. But then after that they moved on to some of the more interactive demos, the first one obviously being the Big Bad Robot demo. So the first one was actually the Portal one, and opening drawers and interacting with stuff, and it was really cool having, even more so than just having the controllers that were tracked one-to-one, they actually map the controllers to the real-life models. So you're looking at this controller in your hand, and it's the thing you're holding. You can feel the crevasses and the corners of it, and it's so intense a feeling. So it's very cool. Then I went to another meet-up, which was this little hosted event that Oculus had. It was in the Facebook offices. So they brought a few devs over, no press, just a bit of a chat, showed a few things, and I got to try Toybox there. and probably Toybox is the most instantly engaging one I've played. Immersive's not the word I'd use. It's not that it's not immersive, but that's not what I'd call it. It's engaging to the point where I forgot to critique it. I immediately just started playing, throwing stuff around and... the curvature of the controllers to your hands make it just so naturally just grabbing, dropping, it's, yeah. So, they're both really cool.
[00:02:50.119] Kent Bye: Great. Yeah, so, tell me a bit more about the gameplay ideas of Echo Red, of like, sure you're becoming a superhero, are you doing it with other people, is it a social experience, or what's sort of the goal or intent?
[00:03:02.452] Jason Storey: The word I always use is empowerment. I think VR has a great capacity to give you a sense of control over the world that's inhuman. You can completely flip it and rotate it and change it and destroy it and our goal is to make the world a toy. So everything you can do in it, you can rip the floor up, you can suction tiles off the wall, make a ball and throw it and It's going to be single player, but it's kind of a score based game. So it's going to be waves of enemies coming at you and it's not even about being accurate. We're actually going to do a couple of clever tricks that may or may not work for aim assist. So our game will very slightly skew in your favor. So a lot of the things you do will be a little bit more perfect than they should be because that's not the point. The point is about you just throwing things and hitting stuff and getting targets done and waves of enemies just flying off platforms as you throw more energies and balls and stuff. A prime example of that empowerment feeling is one of my favorite tropes from comics and cartoons in general is when an evil villain runs out of power and he has to go to like the power station to charge up. So the way our game works is if you run out of energy for your attacks there's these two lightning towers and you reach your hands up into the air and you just suction lightning into your hands and that's it, you're recharged. That's the whole point, just, I want to make you feel like a badass and not have to think about accuracy of shooting or accuracy of something, just destroy things and just, yeah.
[00:04:25.321] Kent Bye: And so yeah, maybe talk a bit about the Left Handed podcast and sort of the intent and what you guys sort of see yourself as covering.
[00:04:33.335] Jason Storey: Sure, sure. It was kind of my idea. I'm going to take credit for that. Basically, I was talking to a few guys on Skype and I was complaining about the fact that there's a lot of podcasts, but they're great for information. You know, you need a brain dump of everything that's happening in the community. That's fantastic. There's lots of podcasts for that. But one thing that I couldn't find was just a bunch of idiots having fun, because sometimes you're sitting there on a long trip and you're just going I just I want to be at home chatting with friends about my hobbies and VR is a great hobby so we said no rules we'll just hang out get drunk and talk about stupid stuff so it's not very structured you're not going to get all the info if the info you do get will probably be biased and stupid we'll make fun of each other a lot but it's a lot more as if you're in a room with us just chatting and I used to listen to a couple of podcasts like Podtoys, fantastic for that. It was just a podcast about a bunch of people just messing. And there's so much joy in being involved in private jokes because you're there and you're hearing them and you're hearing everyone just have fun. And yeah, I thought that was missing. So I mentioned it casually and then Matt and Bob just went, we'll do it. And then Literally from that conversation to 10 minutes later, we're like, should we just do one now? Sure. No script, no nothing. We just did a podcast. And then we said, so same time next month? And all of a sudden, it was born. And now we've been doing it for over a year now. So yeah, it's going well.
[00:05:57.324] Kent Bye: And would you guys identify as sort of like gamers who are into VR, and you're kind of like trying stuff out and then talking about that? Or maybe how do you identify as?
[00:06:06.738] Jason Storey: Yeah, that's probably a good... Gamers is probably a good one because we often get sidetracked and talk about GTA or talk about other things, but personally for me, I consider myself a programmer, a coder first and foremost. And it all plays back to that same empowerment thing. I like to build a world. I like to see a world change based on some line of code I wrote. So for me, it's all about... I love the power side of it and kind of how amazingly creative you can be. So Matt, on the other hand, is a musician. So for him, he's really interested in that aspect of things. And Bob does music, too. So we have disparate interests and completely different personalities, which kind of works. We're gamer nerds, you know, the usual cliche.
[00:06:48.445] Kent Bye: Right. Yeah, and it seems like you've also worked on Kolos, which was part of the mobile game jam, and a winner in that as well. So maybe talk about your involvement with that and what you were working on there.
[00:06:58.828] Jason Storey: Yeah, yeah, Colossus was great. It's a prime example of where my abrasive personality actually comes in handy sometimes because I played Nick Pitom's demo, the American Pie game he was making. I chatted up to him on Skype and said, love the game, it was great. but your gaze tracking sucks. I like had to hump the air with my head for about two minutes to get it to connect. And it's just, no, no, this is how you should do good head tracking. And I wrote a few lines of code and said, use this. If you want, I'll do it for you because I want to play your stuff. It's cool, but I want it to work. And he said, sure. And then later on, he's like, Hey, do you want to do more stuff? Want to work on things? I'm like, yeah. Okay. And then when him and his team, were making Colossus. They basically got to the point where they needed a programmer and Nick was like, I know a guy and I was that guy. So I was quite lucky to be sort of dragged into that project and really fun, really fun trying to do narrative storytelling in VR because it's still a little bit of an unknown. We did a lot of gaze tracking stuff and detection of people. It was hard to do initially, trying to get eye tracking and detect whatever the player's doing. We're using a lot of stuff like we can detect when something's in your periphery versus when you're looking at it versus behind you and we're going to open up a lot of doors for how to augment the story based on what the person is doing. One of the big examples is there's a giant footfall in the game, and we made a point of setting that up so the footfall falls where you turn. So you hear a noise behind you, you turn around, and that's the side the foot slams down on, so it's more visceral.
[00:08:29.707] Kent Bye: So yeah. Yeah, and it seems like, you know, you're using the gaze controls to be able to kind of pace, you know, because it is a 360 degree experience, you know, you don't really control where people look, but you can kind of trigger things once you know they've at least looked in certain places, and so if they're looking in the general direction you can at least know to kind of trigger things. So Nick wrote up a number of different post-mortem posts talking a little bit about, you know, some of that stuff. So maybe you could expand on that a little bit in terms of what you were actually doing in order to kind of trigger different events within this cinematic experience.
[00:09:01.172] Jason Storey: Sure. Unfortunately for us, time was a constraint. We didn't get as many things in as we liked. The main part is that what we're doing is we're detecting where you're looking and then comparing the dot product for the angle so we get a really clean number between minus one and one. for your relation to that object. So we can tell how much you're actually looking at it. So that gives us the ability to have things that fade out when they're in your periphery if they're not important, or if something's supposed to be interesting, we can make it glow in kind of just off the edge of your vision. The goal I have for future projects, which hopefully is going to be a lot more visceral, is I want the story to be driven by your actions passively that you're not aware of. So an example I had was imagine being on the front of a boat and there's a giant vista ahead of you. There's a village to your left, a giant castle to your right, and then a murky kind of waterfall in front of you. And you're just sitting on the boat looking around, and the boat's just bobbing along, and you're just paying attention, looking, oh, this town looks interesting, I wonder who lives there, or I'm kind of worried about that waterfall. And then you look up at the castle going, oh, that's really interesting. Who's in the castle? Why is there a castle there? And we've just been paying attention to how long you've looked at each item, and we notice the castle intrigues you the most. So all of a sudden, you notice the boat moors at the castle. which will be a very different experience than somebody else who's way more interested about the big waterfall and they're panicking about it. So you could really drive a narrative based on something the player doesn't even know they're doing. And that really excites me as an idea because you could then have a dialogue afterwards and discuss it and it's completely personal to you.
[00:10:33.997] Kent Bye: Yeah, and it seems like, you know, Colossus is something that is a pretty linear narrative. You know, there's not a lot of branching where, depending on what you're looking at or doing, it's not really impacting the story. It's more of the pacing, and so... But there could be different variations of branching behaviors. It sounds like what you're talking about is, like, potentially a completely different experience based upon what the viewer is actually looking at.
[00:10:56.293] Jason Storey: Yeah, Colossus was an idea, it was a test bed. It was very scripted, it had a strong narrative. At one point it even had voiceovers. So it was very, very particularly alleged to tell a tale. And we wanted to see how people reacted to that. Now, we did learn one lesson, which is People do want some interaction. So the passive stuff was great, but people weren't aware of it, obviously, because that was the whole point. And it kind of made people feel a bit cold to it. Another great example is the Henry demo. If you stare at Henry, he stares back. And if you move your head around, he'll eye track you and kind of look at you a little bit. But they don't tell you that. So when it happens, it's a big, whoa. But it's very much. Connected feeling so we decided we're going to still run with this idea of passive event changes and make big future projects way more Customized, but we're also going to give you powers and give you stuff or give you some level of interaction So that's why echo red our next project is going the complete inverse We're going whole hog on just you interacting with everything and then once we have both extremes completely passive and completely active experience we'll pick the best from both and hopefully our next project will be something that merges them well.
[00:12:07.244] Kent Bye: And so you're also known as a piece of fruit in different online VR social spaces and so talk a bit about the different social VR platforms that you've tried out and sort of your take on that.
[00:12:19.591] Jason Storey: Yeah, that was actually my introduction to the whole VR thing, really. My story for backing Oculus wasn't as glamorous as everyone else's. I just happened to have some income and was bored, never heard of it. Went on to Kickstarter and went, yeah, okay, whatever. I just got a bit of a windfall, so I thought I'd give it a try. And then in the wait for it, I decided to just check out online and see what was going on. Yeah, I stumbled across VRChat back in the super early days where every character's in a T-pose. And we ended up chatting and I met a few people. In fact, I went there in and around my birthday and we ended up singing Happy Birthday to me and one other person. and we officially called it the birthday crash because too many voices at the same time broke it and it exploded and that became a bit of a tradition for a while as a new test every time a new build came out we'd all sing happy birthday to see if we could break the demo and it's great it got me in touch with people and all the people here there's 70 odd people here I've met that I know online I've been talking to for a year and a half and I've never seen them before until today So VRChat brought me there. And then from that, Riffmax was the other one I tried. And Riffmax was great because it was the first time you had hands. You could pick things up and you could slap someone in the back of the head and you could just, you know, wave at people. It's great. And that was really fun because it was a shared experience. as opposed to just a conversation. And then there's also Converge, which is just really cool. I love the art style and the bobbly heads. It's fantastic. And Old Space, of course. And Old Space have apparently just added mobile, as in Gear VR, and Vive support. So that sounds fun. I can't wait to go and play with the Vive on that. I haven't actually been there outside of two beta days they had to try it out, but I'm looking forward to give that more of a run.
[00:13:58.023] Kent Bye: It sounds like being overseas and perhaps somewhat isolated from a lot of VR people in your immediate vicinity, you've found a way to be connected both through Skype to the podcast, but also through these different social VR experiences.
[00:14:09.912] Jason Storey: Oh yeah, it's absolutely necessary. I mean, what really, really struck me, one of the most mind-blowing things I've ever experienced, is I spent a year online talking to people. And from my perspective, it's a room full of ten people, just randomers online having a chat. The scope of VR never hit me because, you know, people would wander in and out, just a standard chat room. But then, I was invited to London to take part in a Vive Jam. and I met a lot of developers. I'm standing there talking to people and these are people who've made games you've played. These are people who've worked on Batman Arkham City and people who've Like the guys who made Gang Beasts, just sitting down having a beer going, wait a minute, this isn't just four or five people in a cupboard. There's a big thing going on here and that sort of kicked my ass to get out and go to meetings. Because the VR space is great when you're not there and it gets you in touch with these people and it makes the meeting of them in real life so much easier. Yet, until we get some really good haptics and some really good body tracking, you can't be in person. So, yeah, coming to a place like this and looking around is just mind-blowing.
[00:15:13.982] Kent Bye: And finally, what do you see as the ultimate potential of virtual reality and what that might be able to enable?
[00:15:20.753] Jason Storey: Ooh, that's a very big and dangerous question. Personally, for me, again, to play on the themes of empowerment, I think we've never had a truly perfect input system in anything. The mouse is really, we're familiar with it. Everybody knows how to use a computer mouse, but at the heart of it, I never liked the idea that you're just moving this thing X and Y to interact with the screen. It always felt so disparate, and it feels like we've gotten used to it as a way to solve a problem, but it's not good. In VR, though, you are in the world and you can do gestures suck we all know most gesture games are just terrible but they don't have to be when you are in a position where you know everything about the person playing so for me the peak of VR is going to be the moment that we have games that just know us. They know what we're saying, they know what we're doing, and you have muscle memory. You're not pressing buttons anymore. You're just reaching for stuff, but you're not picking it up with your hands. You're just gesturing vaguely and it flies into your hand because it knows that's what you're trying to do. Or you're just walking around and you're thinking about something, so you look over at the corner of your eye, and just because you're turning your head at a certain velocity and we know the look direction, it kind of moves over in preparation for you looking at it because it knows that it's the thing you're interested in. So yeah, when we get a world that responds to us dynamically and organically as if it is symbiotic, that to me is going to be fascinating. And imagine how creativity and workflow and art and every single industry can become so much better when tools are an extension of you as opposed to a thing you use to get stuff done, you know? Great. Well, thank you. Thank you.
[00:16:59.752] Kent Bye: And thank you for listening! If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash voicesofvr.
