#237: Technolust’s Cloudstep VR Locomotion & Overcoming the Uncanny Valley by Coding Social Behaviors

Blair-Renaud2Blair Renaud is the creator of Technolust, and he talks about his strategy for creating different experiences out of the Technolust universe for the Vive, Playstation VR, Oculus Rift, and Gear VR. Technolust has been one of the early VR adventure games, and so Blair has been on the forefront of trying to find a good VR locomotion solution that is comfortable for the majority of users. He explains their flexible dungeon crawler locomotion scheme that they’re calling “Cloudstep”, as well as some of their strategies for overcoming the uncanny valley with realistic social behaviors.

LISTEN TO THE VOICES OF VR PODCAST

In this video, Mark Schramm describes their solution for VR locomotion named “Cloudstep.” They took inspiration from a combination of the comfort mode for yaw turning as well as the step locomotion that’s found in dungeon crawlers like Crystal Rift. The innovation that Cloudstep makes is you can have more freedom to explore around because you can move in the four directions relative to your head position as opposed to the four fixed cardinal directions.

In this video, Mark describes how they’re using four colliders oriented around the head position to calculate whether there are additional processing that is required for moving up or down or if there is a restricted movement to prevent walking through walls. Cloud step eliminates vection that is suspected to cause simulator sickness, and therefore provides a more comfortable locomotion experience for people who get sick easily while moving around in VR.

Here’s a video of Cloudstep in action with footage from Technolust:

Blair has also been involved in helping to start up and advise a photogrammetry studio called Quantum Capture where they’re able to produce super high-resolution avatars. One of the potential issues of having high fidelity avatars is that it could easily fall into the pit of the uncanny valley, and so Blair told me about some of things that they’re doing in order to help give these NPCs a little bit more lifelike behaviors. They’ve been collaborating with the Coffee Without Words developer Tore Knabe who has been mining psychology research for different non-verbal behaviors that could be coded into Unity scripts and attached to the NPCs.

A couple of research papers that Tore has used include “Realistic Avatar Eye and Head Animation Using a Neurobiological Model of Visual Attention” and “Eyes Alive.” He’s written more about how he coded these behaviors into Unity in this blog post. Having these types of behavioral cues can help make these characters seem less creepy and help to climb up that steep incline of the uncanny valley of creepiness. Blair is very confident that their approach is going to make some advances in creating a coherent feeling presence with these characters.

Finally, Blair talks about winning the Best Art Direction and Best Sound Design honors at the Proto Awards, and his approach of adapting Unity Store Assets in order to create a cyberpunk world with a consistent look and feel that feels really nice.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Rough Transcript

[00:00:05.452] Kent Bye: The Voices of VR Podcast.

[00:00:12.196] Blair Renaud: I'm Blair Reneau, Iris VR, working on a lot of things, most notably probably Technolust for the launch of the Rift. Got a bunch of gear titles. Some stuff in the Technolust IP for Vive and Morpheus as well. There's probably other things I'm forgetting, but yeah.

[00:00:31.850] Kent Bye: Some stuff, some VR stuff. Yeah, didn't you do a thing with David Cronenberg as well?

[00:00:36.834] Blair Renaud: Yeah, I did a David Cronenberg experience for the Canadian Film Centre. We're actually doing another, like a more expanded version of that for the Vive and the Rift, assuming we get some No, I don't even think we need touch controllers for that. Yeah, what else? Partner in a photogrammetry studio called Quantum Capture, doing photorealistic digital humans for games. And Occupied VR, which is a VR studio in Toronto, doing mostly 360 video stuff, but also VR experiences, like client-side stuff.

[00:01:13.271] Kent Bye: Wow, so it sounds like you're keeping pretty busy. That's great. Let's start with Technolust and all the different platforms. And maybe you could kind of describe to me, first of all, the intent of Technolust, but also how it comes through in all the different mobile, desktop, and sort of room scale.

[00:01:29.299] Blair Renaud: Sure. Well, Technolust is kind of like, I'm trying to do like a Star Wars thing, right? Like build out the universe and then I can use that IP for multiple things. Saves me from doing some extra art every time. So for the Rift, the main experience is kind of like an intro to the world, an open world adventure style thing. And then for the mobile, we've got our Game Jam game called Thought Crimes that won bronze, which is like seventh place, I think, technically, which is OK. Anyway, it's kind of on hold right now until there's more of a market for the gear, which I think Oculus will hopefully be announcing this week. And then for the Vive and the Morpheus, I've got more of a standing experience. It's called Scanlines. You're basically like a TSA agent in the Technolust universe. We have our photo scan characters, all performance captured and everything. They come in and you're like a security guard. You scan them, interrogate them, plant evidence on them, get bribed by them. Kind of like a papers please, kind of moral dilemma thing. But with really intense social interaction, hopefully.

[00:02:46.387] Kent Bye: So maybe you could talk a bit about the core game mechanic that you see is driving these types of interactions. Is it about adventure and trying to kind of put puzzles together? Or what is the gameplay or the larger sort of intent? What type of experience do you want people to have? Sure.

[00:03:02.136] Blair Renaud: Well, Technolust is primarily a point-and-click adventure re-envisioned for VR. So there's a lot of finding objects and using them elsewhere. just really to forward a narrative but I think the fun of the game is more in the kind of exploration aspect and we've got like a new game plus sort of thing so once you complete the story you can start over from the beginning and because a lot of the aspects of the story have been revealed things will make more sense kind of like a like a fight club kind of vibe right like Once you watch it the second time, you realize how all the pieces fit together a lot better. So yeah, it's more story and exploration. And that's kind of a genre that's picking up, and I think it's really well-suited for VR. You know, things like Vanishing of Ethan Carter and Gone Home and things like that. So it's very similar to that. A little bit more old-school adventure style, I'm hoping. And we've actually incorporated a new control scheme for it that alleviates motion sickness, hopefully, that we'll be announcing soon, and probably a Cloudhead-esque parody video. We're gonna announce that but yeah, it actually gives it more of an adventure game style like feel as well because it's like a stepped movement sort of thing but it works really well in VR and I found that moving isn't really that important like you don't really miss smoothly moving around and rotating as long as you get to a place and you can explore it like Standing and like crouching and exploring the whole area. You don't you don't really care how you got there So it's a lot less of a big deal than I thought it was gonna be which is great So it sounds like you have like either waypoints or are you actually determining where you're teleporting to I guess I can just tell you now I'm working with Mark tram from VR bits and we've devised a Other people have actually demoed this. I think it was the Smash Hit Plunder people. The woman from that was doing a talk. It's like a discrete steps movement. So there's no acceleration and deceleration. And of course it's toggleable. So if you're fine with moving around smoothly in VR, that's great. But yeah, you kind of teleport ahead a step. Like you hold forward and it steps you forward like an old school dungeon crawler kind of thing, right? Like the scene just jumps towards you in like a comfort mode style rotation. And yeah, it works really well. You've still got the ability to sprint and your steps become further apart and shorter time interval between them. But yeah, I've been finding it works really well and people aren't bothered by it at all. You know, you'd think people are like, oh, I feel like I'm not walking or something or like this is jarring, but it's not. It's actually, It makes total sense. And it actually fits really well in the Technolose universe. If you've ever seen Blade Runner, the Esper machine that he uses to navigate through photographs in Blade Runner, kind of like dig deeper into the photograph, and it did that in like discrete steps, right? It was like enhance, enhance, enhance. It gives that kind of feel, which is, yeah, it works really well.

[00:06:14.227] Kent Bye: Does that mean you have to kind of design it as more of a grid-based, where when you're walking, you're walking either sort of either northeast, south, or west one unit?

[00:06:23.377] Blair Renaud: No, not at all, because your rotation can still be based on where your head is pointing, right? So most of the time you don't really use the right analog stick to turn. If you're standing you can just actually turn around. Or you can push the thumbstick in and it'll turn you a 180 directly. So you can just quickly flip. But Mark's designed it so if you get close to an object, it kind of sends out a feeler to know that there's something there that's going to block you and will put you as close as you can to that object. So there's not really a grid per se, it's just discrete kind of steps. Just like a person, like if you walk, your gait will change if you get too close to something, right?

[00:07:05.305] Kent Bye: Yeah, and because you are using objects in different situations and contexts, you sort of face the challenge as a designer as to how do you tell people what is important and what isn't important, and can you have an inventory system where you can pick up anything and do anything to anything else?

[00:07:21.681] Blair Renaud: Right, no, no, it's not really like that. I consider doing an inventory system, but I found that that actually pulls you out of the story a little bit more. Just kind of opening a new screen or having a bunch of objects come floating out in front of you, it's just doesn't really have a good feel. So it's sometimes a bit of an issue where people say, oh, I don't know what I can pick up and what I can use, but I'm hoping to kind of push that with the story, right? And anything else that you can interact with outside of the story that hasn't been made obvious to you, like, oh, I need to find this to do this, will be things like collectibles or just Easter eggs and stuff like that. But this happens with old school adventure games too. You're just sweeping the screen with the mouse looking for the grab icon to show up, right? So you might get situations like that where people are just running around a room mashing the X button, but hopefully that's not the case. I think people are just They'll go up to an object and examine it. And sure, they might hit the button once to try and pick it up. And if it doesn't pick up, then they'll just enjoy it for what it is. It's just a neat little knick-knack there or something.

[00:08:30.068] Kent Bye: Yeah, and in your Gear VR experience, I think you debuted some of these photogrammetry avatars. And so maybe talk about what you're doing with those.

[00:08:37.733] Blair Renaud: OK, yeah. So Quantum Capture, who I'm partnered with, basically their goal is to make humans, like digital humans, and have them interact properly with social interactions. We have a script that we just drag and drop onto them, basically, at this point. and they become alive, they become aware of their environment, they have points of interest around the room. Tore Nabe, who did Coffee Without Words and a bunch of VR experiments, actually built the main script that we're working with. So yeah, it gives you eye tracking. We're working on repulsion and attraction, so you could put a script on an object that says that it's repulsive to a certain AI, and when that object gets close to them, they'll pull away, they'll squint, and hopefully trigger animations, like facial animations. Not for Technolust, we're not doing any full facial performance capture, just because it's way too much money and effort and work, but for scanlines, definitely. Yeah, they look crazy. It's crazy sometimes. We had the actress, I don't know if people know Amity. She's the bald girl that's in the DK2 demo of Technolust sitting in the chair. We had her come in to do a reshoot of her scan with her full facial performance and everything. And we put her in the Vive in front of her scan with these scripts on it, so making eye contact. looking at her, and she was freaked out by it, of course, right? Like you would be. It's like looking in a mirror that isn't doing what you're doing, and it's very strange.

[00:10:16.679] Kent Bye: Have you scanned yourself and interacted with yourself in VR Experience?

[00:10:20.282] Blair Renaud: The first scan we did, actually, before we even had a full camera rig, we had, like, 12 cameras. We did my head, so I had to stand on a pedestal, and they'd rotate me by, like, 15 degrees, take, like, I don't know, 20 shots or something. So we put my head in as like a Zoltar arcade machine, like the fortune teller arcade machine from the big movies. And yeah, I put that in and I wanted it to do sort of a jerky robot movement back and forth, but I'd like misplaced a decimal point. And I got into the game and my head was just like spinning uncontrollably. And yeah, that freaked me out. It was like a Jacob's Ladder moment. It was very, very creepy on Candy Valley. So I assume later down the line, I'll have, yeah, why not? Like a nice full body avatar of myself. But for now, it's just frivolous, right? I'd rather get some game characters.

[00:11:15.633] Kent Bye: I guess that's the thing that I wonder about doing something that's so high-fidelity and high-resolution, just from what I've done a number of interviews with academics and different people looking at the issue of the Uncanny Valley, and it seems to be the issue comes down to expectations. And so when you see another human and it looks realistic, you have a certain amount of expectations for how they should behave. And if they don't meet that expectations, then that's when it starts to look creepy. And so I'm wondering, you're kind of pushing that edge of really, you know, trying to go out of the valley and up to the top of that super high fidelity. And I'm wondering how you felt, like, as you're watching and seeing some of these avatars with these scripts, if, you know, they feel creepy to you or, you know, you have to continually try to refine to kind of climb that steep hill of the uncanny valley.

[00:11:59.182] Blair Renaud: Yeah, well this is, we're kind of getting around the uncanny valley with Technolust in that most of the characters were either made to be sort of like give you a feeling of unease or creepiness. Like they don't, we don't have the full facial performance capture for them, right? So they have some bones in their face so I can move them around a bit but You know, for instance, that demo with Amity in the chair, like she's supposed to be like not really there, right? Like she's there's something wrong with her. I'm not going to spoil what what's actually happening. But yeah, so Technolabs, we're getting around that right now. But we've done some experiments with things and I'm pretty sure we've bridged the uncanny valley. There's characters, there's this one guy Thomas, he's the big black dude, you might have seen him in one of my demos. We scanned him, he had a little smirk on his face, just kind of looks happy. And just that alone seems to be like enough. He's blinking, he's smiling, he's looking at you, and there's no reason to get creeped out, you know? And I think that's what the uncanny valley is, when it's that not sure if they're dead or not. It's like seeing somebody that's moving but looks like they're not alive is what creeps you out. So as long as you They look happy and they're not doing anything weird, like their arm is twisted funny or their eyes are rolling or something. I think it's pretty easy to bridge with this kind of fidelity. And these kind of like social interaction scripts, like you said, there's certain things you expect, like micro saccades in the eyes, glancing down at the lips when they're looking at you, things like that. Points of interest around the room, like if someone's talking to you, they don't stare you right in the eyes the whole time. They'll glance away, they'll like, If somebody walks by, they'll look at them. We've got all that stuff in there. And I think that stuff really helps kind of bridge that sense of unease.

[00:13:52.286] Kent Bye: Yeah, and maybe say a little bit more about CoffeeWap without words, so people aren't familiar. Because that's sort of an exploration of some of those kind of nonverbal social behaviors to try to convince people of that more natural human behavior.

[00:14:04.038] Blair Renaud: Yeah, yeah. So Tore, I hope I'm pronouncing his name right. I've only ever read it. Yeah, he did a really great job. He basically used these numbers he got from psychology papers, like human social interaction with the eyes, and programmed them into a Unity script. So it's doing, yeah, like micro saccades are like when your eyes kind of shudder left to right, even if you're just staring at one eyeball. You look back and forth between the person's eyes. And all this stuff is easy to do in VR because we know exactly where the person's eyes are, right? Glancing down at the lips, that's kind of a rough approximation, but we assume everyone's lips are a couple inches below their eyes. Yeah, it's amazing. And we're hoping to get a lot more of that sort of thing in, like even more subtle things like nostril flare and like breathing. All of this stuff can be procedural. There's no reason why it shouldn't be, right? That's something that bothers me with a player avatar in VR is when I look down and I see somebody else's body hanging off of me. And even if I've got hand controls, that'll bother me because the IK isn't right and the elbows are facing the wrong direction. But subtle things like, it's not breathing. I'm looking down and I'm breathing, but this body hanging underneath me isn't breathing. Those things are really important and I think those are things we're missing. Quantum capture is hopefully gonna get all that stuff, right? It's the plan anyway

[00:15:38.055] Kent Bye: Nice. And, you know, congratulations. At the Proto Awards last night, you took away a couple of awards for sound design and art direction. Maybe you could say a few words about that. Sure.

[00:15:47.864] Blair Renaud: Yes. Thank you. I was up there way more times than I deserve to be, but it seemed to be for good comedic effect. Except it went for Mark Trum at VR Bits for Nighttime Terror as well. And the Vox Machina guys. What's their name? Space Bullet Dynamics Corporation? I love that name. That's fantastic. Yeah, so I accepted their award just because they weren't there. But no, it's really... I love the awards that I got. Sound design is very close to me. I worked at Rockstar a long time ago doing sound design for Grand Theft Auto London 1969, which actually won a BAFTA for sound, awarded to Rockstar, even though I was the sound guy. So I'm proud of that as well. And art direction. Direction is the right word because I'm not actually doing a lot of modeling and art for the game. It's a lot of kit bashing, taking Unity assets, tearing them apart and putting them back together. The great models from the Quantum Capture guys for the characters and stuff. But I think I have a good sense of aesthetic and lighting. So it's nice to be rewarded for that. Makes me feel good, because I spend a lot of time just sitting in these worlds going, this doesn't look right. I'm going to sit in here for hours until it looks right. So yeah, it was nice. The awards were fun. It's great. I've got these giant pieces of bronze I have to get back on the plane with me, which is going to be annoying and expensive. Maybe I can get them to ship them to me.

[00:17:20.945] Kent Bye: And finally, what do you see as kind of the ultimate potential as, you know, all these technologies that you see coming together and really starting to play with, you know, kind of bridging that uncanny valley? Where do you see all this going, you know, moving forward?

[00:17:34.076] Blair Renaud: Oh, everywhere. Come on, this is like the typical, like, where do you think VR is going question. Yeah, it's like you can get some truly presence inducing like I believe I'm here experiences which you can use to great effect and for any effect, you know, like from just like emotional responses to to just relaxation like you can you can do a lot and it's porn can't forget porn and Porn's gonna be crazy, I think. I don't know. We'll see. But yeah, the potential's insane. I don't know what else to say. It's like, you've seen the holodecks on Star Trek. I think given a couple years we can get to that sort of level. Hopefully we'll have some better ways to do tactile response and haptic feedback, but that'll come, I'm sure. Okay, great. Well, thank you. Yeah, no problem. Thanks.

[00:18:32.398] Kent Bye: And thank you for listening! If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash voicesofvr.

More from this show