Owlchemy Labs‘ Job Simulator was recently announced to be one of the bundled games to be included with the HTC Vive, and it’s also going to be a launch title for the Sony PlayStation VR as well as for Oculus Touch. I had a chance to catch up with Alex Schwartz and Devin Reimer at the Unity VR/AR Vision Summit to talk about developing across all of the major VR platforms, the magic of hand presence, and the range of behaviors they’ve seen when given an open world physics sandbox.
LISTEN TO THE VOICES OF VR PODCAST
Alex has observed that people either mimic their natural behaviors from reality or they will do the most extreme things that they’d never do in real life, and that experiences like Job Simulator could be a sort of psychological personality test. Devin also discovered that people have had so much fun in Job Simulator that they experience time dilation to the point of misestimating their how long they were in VR by a factor of 4-5x.
They also talk about developing across each of the VR systems with hand tracked controllers and how they had to calibrate each system so that the virtual hands matched the position of your actual hands. One interesting discovery that they made with hand presence is that they found that it felt better to make your hands disappear when you’re holding an object. They’ve coined this phenomena as “tomato presence,” because hand presence seems to be transferred to whatever object you’re holding at any given time.
They’ve also noticed that there’s a certain performance art vibe that happens when people play Job Simulator within group gatherings, and that they expect that group experiences will inspire people to take suggestions from the audience and to do them in a way that’s as entertaining as possible.
They realize that there’s an inherent marketing problem with Job Simulator, in that people aren’t inherently motivated to simulate ordinarily mundane jobs within VR. It’s a problem that VR faces in general in that you have to try it out to really get it. But they’ve noticed that they have a lot of diehard fans who have been evangelizing and defending Job Simulator on Internet threads saying that you have to really try their experience to really understand why it’s so compelling.
It’s probably best summarized by this Tweet from Devin:
VR has two development camps. 1)Building what people think they want 2)Building what people don't yet know they want. #1 is very dangerous!
— Devin Reimer (@DevinReimer) March 3, 2016
Job Simulator is definitely a VR experience that most people don’t yet know that they want, but it illustrates the magic of hand presence in virtual worlds in a way that is both really surprising and delightful.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:12.034] Devin Reimer: I'm Devin Reimer. I'm CTO of Alchemy Labs.
[00:00:15.294] Alex Schwartz: I'm Alex Schwartz, the CEO and janitor of Alchemy Labs. And we're working on Job Simulator, which will be a launch title on all the VRs. So we'll be launching on HTC Vive and will be a launch title on Oculus Touch and a launch title on PlayStation VR.
[00:00:31.591] Kent Bye: Great. So tell me about what happens in Job Simulator.
[00:00:36.194] Alex Schwartz: So in the future, in 2050, robots have taken all human jobs. And so the humans, so that they don't forget what these ancient professions that are no longer around, what they were like, the Job Simulator was created. So you can go back in time as a historical reference as to what it was like to job. So you can go through a variety of different experiences. And so we've been showing the kitchen experience, which is a gourmet chef. You're basically making recipes and going through that whole side of the world. There's also an office cubicle worker, where it kind of feels office space-y in that sense. And we've been showing that around. And we've also shown the convenience store clerk, where you're basically kind of like Apu in the Quickie Mart making slushies and old hot dogs. So that's kind of the vibe of the various work experiences in Job Simulator.
[00:01:22.382] Kent Bye: And talk to me about some of the reactions that you've been getting so far from your game.
[00:01:27.830] Devin Reimer: The reactions have been fantastic. I mean, better than I could even have hoped. People just love to play. If you give them the opportunity to use their hands and actually interact with the world, people just love that.
[00:01:37.784] Alex Schwartz: Yeah. It's bringing out some kind of inner child in players, but I think it has something to do with manipulating physics items with your hands, and it's a very childlike wonder that comes out in VR, and that's why we think hands are so important to kind of completing that whole loop of presence and interactivity, because, you know, you start off with something where you can look around, a passive experience, and VR just being immersive is great. And then once you have the ability to actually move around positionally, you add a whole other layer of feeling like you're there. but you're still gimped. It's like you've got a straitjacket on if you don't have your hands. And seeing something on a table right over there that looks so juicy, I want to grab it with my hands. It's like once you've had the ability to pick something up and manipulate it and do everything with it, you can't go back to a moment where you can't actually do that. And so I keep using the analogy of like holding a little toy in front of a baby and they can't reach it and they're just screaming about the fact that they want this thing. And that's kind of, I think, deep down what our inner child is saying when we don't have hands in VR.
[00:02:36.274] Kent Bye: And so what are some of the biggest insights that you've found about hand presence in VR?
[00:02:41.875] Devin Reimer: So one is that getting your hands, the visual representation as close to your real hands as possible is really good. So we have these glove hands and so they're just a little bit bigger than a normal hand. So it actually kind of feels like I'm wearing gloves instead of like a weird thing is like, oh, these are puppet hands or something like that. And so what we do is we do a lot of work to line up where we think the person's hand is on the controller for each different controller type so that you get this kind of hands presence. So when you like lift up the headset, it'll like line up with where your hands are.
[00:03:09.588] Alex Schwartz: Yeah. Definitely. Once you hit hand presence and you look down and you flip your hands over and you look at them and you go, oh, these are actually my hands versus me controlling some robot arms. A lot of things instantly click and come together. So we see that skills that you have in real life with your hands and dexterity and movement in 3D space all translate directly one to one in VR. So I've shown videos of me juggling because I can juggle in real life and therefore can juggle in VR. And just these moments that, you know, you don't have to be juggling to have these aha moments, but when you drop something on the floor and you bend over and you reach out with your hand and you make direct contact with the item and you pick it up, a lot of people when they first play Job Simulator and they reach over and pick up an egg that they just dropped, and then they stop for a second. And we could tell that it's that moment that they just realized, Cognitively, oh, I just bent over in VR picked up something with my hand and now I'm moving forward with whatever I was doing and holy crap the whole system just let me do that and So like if something's rolling away from you like a ball and then you reach over here and you grab the item just in time it's these moments where you realize everything is just right and when you've got your head and your hands all being tracked in the same one-to-one environment and new gameplay concepts just instantly emerge. Because we have a game, Job Simulator, you have some tasks that you could do, but it's also a full-world physics sandbox where you could ignore everything that's going on and say, hmm, how many of these tomatoes can I sink into that pot that's halfway across the room? I'm just going to take 10 of them and try to see how many free throws I can make. And so we still see people doing that kind of thing, like, I'll set up a bunch of bread and see how many dominoes I can knock over, or just things emerge out of the physics and the good hand tech that we never expected, and we see new things every time someone plays.
[00:04:45.415] Kent Bye: It seems like a bit of a challenging marketing to talk about going VR and simulating a job. We do that all day, every day. Why are we going to? So what is it that you generally tell people? Why is this compelling?
[00:04:59.085] Alex Schwartz: Well, just generally, people sound completely crazy when they're describing their experience to someone else who hasn't tried it. And it's exacerbated by Job Simulator being a tongue-in-cheek thing about doing jobs. But just in general, people come out of an experience and they go, oh, there was a whale in my living room, and then I threw a tomato. And it's like, you can just imagine them getting dragged off to the loony bin at that point. But you can't describe how it felt to be in a place. And then when you flatten it to a 2D image, or you put a video up of what something looks like, man, there's just no parallel to what you're seeing. So we go intentionally silly on our video representations of what Job Simulator is, just because it's funny and engaging to watch someone kind of like flipping things over their head and catching them behind their back and, you know, making sandwiches, speed stacking sandwiches or whatever it is that we're doing, drinking out of two mugs and then using staplers to knock things around. So the trailers we kind of go in that direction But man is such a big disconnect between the marketing content for VR and actually being there So really I think the only campaign that's true about VR is just to say you have to try it to believe it and Commercial spots should just be a black image with white text says try this. We'll wait, you know Like there's no commercial that could ever sell someone on VR. I
[00:06:12.334] Devin Reimer: We've noticed this thing on the internet, so comments are really terrible on the internet, just generally. But we're seeing things where people are like, job simulator, why would everyone want to play that? That sounds dumb. And the amount of people that come and defend job simulator are like, no, I tried it. You need to try it. It's really good. It's actually kind of interesting. It's kind of this whole thing on the internet of the people that have tried it and people that have not. And it is slowly changing over time, and it's very interesting.
[00:06:35.352] Kent Bye: Yeah, I was at Sundance and, you know, seeing all the different experiences and there was a bar where people go up and they have a choice to what to choose. And I think that people not being from the VR community, I don't think that necessarily job simulator was something that was striking to them as to do, but I kept telling people like, no, you got to really try it because it's like the one interaction that's here that actually uses like interaction and hand presence and you know people did it and they were just really loved it so there was something like it's just fun to watch people and I think it's something about like their personality can really come out in terms of like how they play and if you guys have noticed that as well.
[00:07:10.869] Alex Schwartz: Oh yeah, the personality thing is incredible. I feel like at this point we need a full-time staff psychologist to start analyzing what's going on here. There might be value to seeing all this, but we have people who... I feel like there's like an A-type and a B-type of player when they go into Job Simulator. I'll keep using the kitchen as the example, but let's say there's a bottle of Sriracha on the table and it gets knocked onto the floor. certain types of people will be very careful about everything. They'll pick up the item they just dropped and they'll put it slowly back on the counter exactly where it was located at the beginning of the game. And like they go to the cabinets in the new kitchen and then when they take something out they close the cabinets when they walk away. And all these things, it's this weird representation of how they're working in real life. We see people walk around broken glass because they broke a bottle directly on the middle of the room and then they step, they tiptoe around it. And we were saying, hey, I wonder if there would be different levels of that if you were actually wearing socks at the time that you're playing VR or barefoot versus wearing shoes. Would you walk over the glass more or things like that? And then so the B type is I go in and I realize there are no repercussions here. Let's go nuts. And so people are just flinging things over their shoulders and smashing things and throwing things at JobBot. And then my favorite is people who start out as A-type and then they realize, wait, what am I doing? Why am I following the rules of this game? Why am I doing everything in order? Screw this, I'm going to start rebelling. And so I like seeing people start to realize that just because someone's telling you, hey, you should try this, that you go, no, you know what? Screw this, I'm going to do this other thing. And yeah, it's fun to see those flips and to see the other good story was someone pulled a donut out of a trash can in the office. And then before they ate this trash donut, they went and they looked to the left and right to see if anyone was looking at them before they ate the donut. So it's just, when you really feel like you're there, there's so many human moments that come out. And watching people play is so awesome to see these weird mannerisms.
[00:09:10.517] Devin Reimer: Yeah, it's like the refrigerator. I had one person play the kitchen for a very extended, long period of time. And every time they opened the refrigerator, They put things back, they closed the refrigerator, and then I kind of giggled at one point. I guess they heard me, and then they're like, I just thought you should always close the refrigerator. And it's like, well, I guess that's what happens in real life. You don't leave a refrigerator open, so.
[00:09:30.069] Alex Schwartz: I feel like maybe we should play into that. And if you had a cabinet or something and you left it open, Job Bot could say, do we live in a barn? Or, you know, just really need to get that going.
[00:09:40.845] Kent Bye: Yeah, I've played Job Cementer three times now. The first time was just on a Vive, just the demo that was shown at GDC. I saw it at Oculus Connect, and then I saw it at Sundance. And I kind of noticed that there was a lot of magic the first time that I played it. And then the other subsequent times, it was still fun, but that original magic wasn't quite the same. And so I don't know if you found that as well as, like, this is an experience that people have that's extremely novel. But I wonder what happens once they start to play it over and over again.
[00:10:10.241] Alex Schwartz: Well, so first off, in early VR, in the first couple of years, we think that there's going to be a ton of wanting to show your friends and other people who don't necessarily have access to the hardware the experience. So we've been really cognizant of building a lot of content in the game and making it so there's a lot to do. you know, that the novelty doesn't run out after a few minutes, but also taking into account the fact that we know that people are going to be passing and playing and showing kind of like these VR parties form where when one person's playing, you have this audience form around them. So we found that you might have done one of the jobs before, but then when there's a crowd watching you, you do almost like you switch into kind of this performance mode where people are like either like yelling out suggestions or you're doing things extra silly so that the people in the room who are watching can have this different experience and I think there's different ways to play and we're seeing that when you go through a second or third time that there's interesting side effects of that.
[00:11:02.351] Devin Reimer: We're really excited for Job Simulator to finally come out, because we get to show these five-minute demos, right?
[00:11:07.152] Alex Schwartz: And they're just these five-minute little slices of the game, and we're excited for people to finally get to dig in the game and play, like, the long form of it, and so... That's actually a really good point, is we keep having to do... Yeah, so the first kitchen was a three-minute experience, and it gets cut off, as far as time, and then the office demo has a five-minute loop where it says, it's five o'clock, and so you get kicked out, right? Just because of the realities of a conference, you want a bunch of people to play it. But we've put in people into one job for over an hour, not telling them, hey, you have to stay here. But there's enough content for them to be in a single job for an hour at a time. And it's a completely different experience from the very limited experiences that you have at conferences. So yeah, I'm eager to have people play from home when there's less time pressures and can kind of explore. Because I think you play differently when people are watching you and you know that there's a line of 20 people waiting to play eagerly.
[00:11:58.933] Kent Bye: Yeah, this is something that actually was brought up to someone who went through Job Simulator and was in there for maybe 30, 35 minutes and came out and literally thought they were only in there for 5 or 10 minutes. And he said that you guys told him that, oh yeah, you just experienced time dilation.
[00:12:13.725] Devin Reimer: What is time dilation? So this is something I discovered a few months ago. So we were putting people through longer forms of Job Simulator. And I do this interview process after people come out. And the first questions that I ask are, how long did you think you were playing? And so the first time I ran somebody through, they played for just over 50 minutes. And the person said, I was roughly 10 minutes. And I laughed. And then they were like, what? And I'm like, well, no, like, seriously, how long do you think you played? And they're like, they thought for a moment. They're like, yeah, pretty close to 10 minutes. And I was like, whoa. And then I showed them the clock. And they're like, what? Like, what happened, right? And then I thought it was like, OK, this person just got wrapped up in it, whatever. And time after time, it was like people played for 45 minutes, think they played for 8. People played for an hour, thought they played for 15. There's like, this whole thing was like almost four times less at times when people are playing the game, which is just, it's crazy, but people kind of just get wrapped up into it and super excited and kind of like time just slips away.
[00:13:10.027] Alex Schwartz: Yeah, I don't know. I'd like to hear some more research done on this, but there might be some element of the whole, there's no windows or clocks in casinos so that they don't, you know, you don't know that you have obligations outside. Maybe there's something about kind of leaving behind some of the things that are keeping your brain going about, okay, where do I need to be next? And if you kind of remove that urgency, you can just kind of go into a flow state, but it's really, it's really, really interesting stuff.
[00:13:33.645] Kent Bye: I've been talking to people here and just trying to figure it out like what is it and my theory is that as we're standing around in this room we look around and we can kind of glance and then we have a mental model of where we're at and what we can expect the rules are in this world and that I don't need to keep doing that over and over again. But in VR, it totally violates all of our expectations of what reality really is. And so perhaps those same neurons in our brain that only fire maybe once or twice a minute are firing constantly, right? And so maybe there's something about that. But I would expect that to be the opposite, like you have an expansion of time, like you feel like you're in there for 10 minutes and maybe it was like two hours because there's so much new information.
[00:14:12.168] Alex Schwartz: But it seems to be opposite. over and over like your brain clock, but it's the opposite. It's something's happening that you are not having these moments of kind of checking in with how long it's been in your brain.
[00:14:23.752] Kent Bye: Or maybe it's just like working your brain in a new way that's totally different and unique to VR that the side effect is that you just feel like you're in there and you just sort of lose so much time. To me it's really interesting but also like a little bit scary quite frankly.
[00:14:38.145] Devin Reimer: Yeah, I think also it has to do with the fact that, like, with traditional games that people are used to, right? They're playing a game, looking at the screen, but, like, constantly looking away, right? So you get snapped back to reality every once in a while, and you don't get snapped back to reality. You're kind of just, like, in there all the time. And then the other thing is I use is, like, when I'm working, if I'm really having fun with what I'm working on, time will just kind of slip away. But if all of a sudden I'm working on something super terrible, like, time takes so long. And I think maybe that's also part of it. People seem to experience time going faster when they're having more fun.
[00:15:06.482] Alex Schwartz: So is that maybe how we should rate games? Since time flies when you're having fun, the amount of fun is how much time dilation you experience in VR.
[00:15:13.972] Kent Bye: I think, yeah, I mean, there's also like the flow state, you know, Mihaly Csikszentmihalyi wrote the book Flow, which I think that you're getting into that, where you get in such a flow where time just flies. So I think, yeah, there's something about that is actually a pretty good metric of how much time dilation. But yeah, at the same time, talking to Carl Krantz of SVVR, and he said that he was in VR for 12 hours, and he didn't mean to and expect it, and he was surprised that he was in 12 straight hours, yeah. So to me, no food, not going to the bathroom, just 12 straight hours. So to me, in talking to people, it just seems like this is just something that you've been seeing in the field. And what do you think is actually happening?
[00:15:55.571] Devin Reimer: People are just having fun, I think, and it's just like they assume that they're going to put something on and just play for a short period of time and they just kind of get wrapped up in things.
[00:16:04.938] Alex Schwartz: So maybe a good test would be to just build a quick VR experience where you have to, you know, go through a math course where you're sitting there sitting at a desk and having to learn some high level calculus or something and then see maybe do you get the reverse time dilation.
[00:16:21.318] Kent Bye: So you're here at the Vision Summit talking about launching on every platform. What are some of the big takeaways in terms of the differences between... It seems like you're just using the triggers, so you're not getting into any sophisticated control mechanisms. But just from the control schemes, what are some of the big takeaways for designing for everything?
[00:16:40.718] Devin Reimer: So you have one-to-one tracking with these controllers on all three platforms, and that is huge. I read on the internet all the time, people are like, I really wish there was more buttons on those controllers, and it's like, no, you don't. We could actually use less buttons on those controllers and still be fine, because once you have that one-to-one motion, you can move around, you can do so many things, and all three platforms give us that so that we leverage that as much as possible.
[00:17:03.594] Alex Schwartz: Yeah, I mean, I think we had a slide that was not in this presentation, but when you show just a tracking puck or whatever it is, just a little handle for a controller, and it's taking all of the inputs that were on, you know, the 27 buttons that are on an Xbox 360 controller or whatever, and merging them down to one, and it's just physical motion. And that's hard to think about designing and the joke we keep saying is people who have all this prior game knowledge and they start designing or even playing, they go like, oh, what button do I hit on this Vive controller to crouch? Where's the crouch button or the jump button? It's like, no, you just do that. And so motion controls have melded all the abstractions that used to be in traditional game design. down into a world position so you could press buttons in 3D space and you can use all these different kind of spatial interactions like pulling levers and pulling drawers and pressing buttons and twisting dials and all of those are just that one movement mechanic. So I'd love to, yeah, like you said, remove any and all buttons that could be removed. So yeah, we did kind of a analysis across three and saw that the commonality was you've got a good analog trigger, we've got the one-to-one motion, and then we've got one other, all we need is one other button for our getting back to the menus, and so we just kind of said okay here we're gonna just use those three types of inputs and build a game around that. And to exit you have to eat a burrito? That is correct, yes. When you hit the menu button, you get your physical menu, which is a briefcase that spawns. You open it up. You've got your burrito that says exit on it. And you take a bite of it. Inside of the burrito, it says, really? And then you take the second stage of eating, and you go back to the menu. So man, does it fall flat when describing it vocally. It's like, so there's a burrito, guys. Believe me, it's super cool.
[00:18:51.388] Kent Bye: Wow. So what have been some of your favorite personal experiences in VR so far?
[00:18:56.566] Devin Reimer: I mean, like, fantastic contraption, like, getting on the ground and, like, building things has just been fantastic. I recently played Budget Cuts. That is really great, being able to, like, teleport around and sneak around the robots and stuff like that. I mean, my favorite thing is how many different experiences there were. I was worried that when VR kicked off, there'd be, like, one genre and win. It'd be, like, first-person shooters. That's what almost all content is. But that's not what's fallen out at all. Like, people, like, This space is so large that they're just like, oh, I want to do this, I want to do this, and we get to play all these really cool things that are so different.
[00:19:30.488] Alex Schwartz: You took all my good examples there. Yeah, if we're talking about budget cuts, like there were moments playing that recently where I was once again impressed by what could be done in VR. I was laying on my stomach in the middle of a floor looking straight down and still being tracked and looking at a panel that was in the ground into another space and throwing a knife right into the carpet and killing one of the enemies and man, it just, the realization, when you pull back and go, Let's take a picture of what I'm doing in this room. And the fact that we're doing that is just absolutely incredible. So yeah, put a lot of time into budget cuts and fantastic contraption. Yeah, those are the ones that kind of come to mind top two right now.
[00:20:08.615] Kent Bye: And when you're thinking about like other experiences that you want to either create yourselves or experience, what do you what are you looking forward to experiencing?
[00:20:16.823] Devin Reimer: I think the thing is, I don't know. This space is so huge, and I don't know what I don't know yet. And I am confident that in the next three years, there's going to be so many different genres that pop up. That might be my new favorite genre of game that just doesn't even exist yet. So that's what's really exciting to me.
[00:20:34.193] Alex Schwartz: I think the question of what do you want to build next kind of exemplifies our design process in that we don't have that plan. We have a book of 30 game ideas that would be great, but it's just planning ahead in any way as far as what mechanics you think are going to be great is I think a fallacy of VR design. And so we found that you have to prototype and you have to figure out just in basic form, block out, here's what an interaction could be and to try it and feel it and say, okay, maybe we can go down this path now that I've tried that and go in this direction. And I think kind of like starting with a kernel and growing the design organically is kind of the, right now for us, the only way to go forward. I think if we sat down and just wrote out a bunch of ideas for how a game would work, it would just fall flat on its face.
[00:21:19.237] Kent Bye: And finally, what do you see as kind of the ultimate potential of virtual reality and what that might be able to enable?
[00:21:26.123] Devin Reimer: Wow. I tell people it's going to change every industry in the world. It is going to be truly disruptive. One example that I use that really excites me is that there's people in developing countries right now that in a very short period of time will be able to put on a virtual reality headset get the best education regardless of their location, and then get a job anywhere in the world from where they are. And that's going to unlock billions of people that haven't had an opportunity to do that before. And that's just one small slice of what VR is capable of.
[00:21:57.445] Alex Schwartz: Alchemy Labs used to be an all-remote company and recently we've kind of gotten, now that we're getting all these dev kits, we only get a couple dev kits from each platform and we have to kind of bring everybody into one room to do this development. We've been talking about how this is maybe a five-year stopgap, having to make sure that everyone's in the same physical space in the world to work together and collaborate properly, like standing at a whiteboard and just Coming up with ideas and being able to gesture and look at each other and and have you know subtle nonverbal communication and all that kind of interactions I think those will be able to be replicated properly within five years to the point where The concept of going on like a work retreat so that everyone could get together has very little value outside of the Almost as good personal interaction within VR So really excited about just people being able to communicate and collaborate across the world at any point
[00:22:50.484] Kent Bye: Is there anything else that's left unsaid that you guys would like to say?
[00:22:53.565] Alex Schwartz: We could promote tomato presence. Everyone should be talking about tomato presence as the terminology. I don't know if you... What is this tomato presence you speak of? So in our first kind of post-mortem of Job Simulator, we were talking about the fact that we hide our hand when you pick up an item. There are a number of reasons that led to that idea, but basically when you're moving your hand around, you have hand presence. And then when you pick up an object, even though you hide your hand, the item that's in your hand kind of transfers its presence. And so when you look at the tomato that you're holding, and now there's no hand next to it, you still feel like that's your hand. And so we kept using the term tomato presence to describe that just because the first thing on the original prototype on the counter directly in front of you was a slice of a tomato. And so other developers in the early VR space have been using the terminology tomato presence to refer to kind of the usage of items to represent like a stand-in hand. So we would be personally touched 20 years from now if someone used that in a in some interview It's like tomato presents. We're like, yes, we did that. That's our thing fly the flag Awesome.
[00:23:58.938] Kent Bye: Well, thank you so much. Thank you. Thank you And thank you for listening if you'd like to support the voices of VR podcast then please consider becoming a patron at patreon.com slash voices of VR
