Aaron Lemke of Unello Design talks about his VR development strategy where he’s prototyping and quickly iterating and refining a VR experience first for Google Cardboard, then updating it for Gear VR, and then increasing the graphical fidelity to create a release for the Oculus Rift.
The tagline for Unello Design is “Transcendence through Virtual Reality”, and so a lot of Aaron’s experiences are very relaxing and meditative. He’s probably most well known for Eden River, where you simply float down a river within VR and lean side to side in order to stear. He adds his own musical compositions, and then uses VR as a sort of music video and immersive experience to give people a feeling that goes beyond what a 2D medium can accomplish.
Some of his other experiences include Zen Zone, which uses the concept of mirror neurons to be able to watch a digital avatar in front of you and then focus on the highlighted areas of your body. Other experiences include Opera Nova, Nebuland, and Lunadroid 237. Aaron has also been exploring the possibility of using some of his experiences within a medical context for pain management.
Aaron also recently participated in the Austin Vive Jam, where he created a snake game that is kind of like operation for a room-scale maze where you’re trying to avoid touching virtual walls. Given that a Lighthouse tracked room is only 12 feet by 9 feet, then their design intention was to create an experience that turned people around and disoriented them so much that they forget where they’re located in the real physical room. By all accounts, they were able to achieve that.
Aaron also talks about some of the other highlights from the the Vive jam as well as from the SVVRCon.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:11.955] Aaron Lemke: Hi, this is Aaron Lemke. I'm a VR developer and musician from Austin, Texas. My latest release was Eden River on the Gear VR. Actually, that's not true. I have a meditation experience on the Google Cardboard called Zen Zone. And then I have a psychedelic music ambient puzzle experience on Cardboard called Nebuland. And those will likely be coming to Gear in the coming months. My next project is going to be a suite of meditation experiences for the Gear. And I'm targeting short 5-7 minute little experiences where you can buy individual ones for 99 cents and sort of pick whatever you want.
[00:00:56.752] Kent Bye: Interesting, yeah. What has it been like to go from, you know, starting with the DK1, DK2, then the Gear, and then the Google Cardboard? Do you feel like in order to start projects now, you have to kind of go the most bare minimalist approach with the Google Cardboard and then upgrade to the Samsung, rather than to start with the Samsung and then go backwards to the Cardboard?
[00:01:19.846] Aaron Lemke: I like starting out on cardboard. It's kind of the min spec, if you will. And also it just feels a lot more casual, like the market and the feedback that you get is usually pretty honest. On the gear it's a little bit more formal and I guess I like to fine tune things before I put them up there. But yeah, I mean, part of it is, I mean, my approach now is if you design for mobile first and get it running on a phone, then for sure it's going to work on whatever crazy 90 hertz headset. Yeah, that's been my approach.
[00:01:52.737] Kent Bye: And so yeah, what type of special considerations were you doing on your two cardboard games? I mean, you can expand a little bit about, you know, what the experience is like and what's sort of special about them.
[00:02:02.402] Aaron Lemke: Yeah, well, one of them is a guided meditation, full body relaxation. There's a classic technique where the guide will say, okay, now picture a warm glow spreading up from your feet up into your ankles, slowly up into your calves. And so I'm visualizing that glow in an avatar in front of you. So it's sort of like this particle system that's spreading up in tandem with the voiceover. And it's really powerful. It really increases the tactileness of it. Because before, you know, you're kind of visualizing it anyways. So this just makes it a little bit more concrete. Yeah, and just an interesting side note there. I tried it first person first. So like you look down and the glow is spreading up through your own body, but it was kind of confusing. Like everything is sort of stacked on top of each other. It's hard to distinguish where your limbs are when you're looking straight down. But if you put an avatar out in front of you, you still can relate to it. It's something to do with mirror neurons. Like we can still empathize with that other body and sort of get inside of it and feel what it's feeling. Yeah, as far as specific considerations for cardboard, I just try to optimize the shit out of everything. I mean, nothing special.
[00:03:18.322] Kent Bye: And so yeah, for this mirror effect, how did you experience that? Did you feel like once you were kind of like almost looking at yourself in the mirror and seeing this kind of glowing part, it actually made it easier for you to focus on that part of your own body?
[00:03:30.614] Aaron Lemke: Yeah, it was just a test, just a little experiment. I mean, that's what VR is all about right now. And I kind of knew a little bit about mirror neurons and I was like, well, maybe this will work and it totally works.
[00:03:40.904] Kent Bye: Wow, that's pretty cool. And so this whole other, you listed a lot of things about like ambient music, meditation, puzzle, game. What is this other experience that you've created?
[00:03:50.173] Aaron Lemke: Another one is very much inspired by Fantasia. You're in a dark space. You start out, there's like a floating galaxy in front of you and basically you're solving puzzles that just have really nice music and particle effects and they're puzzles that are kind of like VR inspired so that you have to move around and look around to solve them. So I tried to use the head tracking in a cool way. Yeah, it's maybe five minutes long, charging 99 cents for it. It's kind of a cool, psychedelic experience. I think I made it in like three days, but it's got good music.
[00:04:26.252] Kent Bye: At this point in the market, do you feel like there's a trade-off between just getting it out there and get your name built up versus like actually getting enough people to actually buy it? Do you feel like there's enough volume of people willing to spend a dollar for it to make it worth your time to sort of financially or I'm just kind of curious of this trade-off right now, at this point, of getting stuff out there, getting your name out there, versus just charging for stuff already.
[00:04:50.272] Aaron Lemke: I mean, yeah, I think about that a lot. And on Gear, nobody's really buying anything because there's a bunch of free apps still. So it's making it when you have a $2 app, or you could download 15 cool free apps. There's enough to try where people are probably aren't going to buy your app. But on Google Cardboard, there's less free stuff. So it's easier to get people to buy your thing. I don't know, I mean like one thing about mobile is I think you can get away with making shorter, cheaper experiences and that's more towards the end of the spectrum of like spreading your name out there. I mean I guess I could give it away for free but enough people are seeing it on cardboard to where I think it's worth it. I think mobile is like a good balance between the two. Yeah, like Eden River HD was a premium $10 experience, and some people bought it, but most people didn't buy it. But yeah, I feel like with mobile, the reach is a little bit broader, because it's less of an investment.
[00:05:47.757] Kent Bye: So going forward, do you find yourself creating more of these iterative experiments, and then maybe not even bother creating a higher resolution DK2 version of some of these things then?
[00:05:59.593] Aaron Lemke: Maybe just taking the mobile versions of things and sort of adding more effects and cranking up the shaders and the lightings and then porting them over to room scale. Although I did the Vive Jam a couple weeks ago in Austin and it's just so freaking fun to play around in that thing. Like, all I want to do is just go experiment and build cool room-scale puzzles and physics games and things like that. So, I don't know. Like, I want to target mobile first. I think that's the smart thing to do, but it's so alluring. You know, these room-scale systems, it's... I don't know. We'll see how focused I can stay.
[00:06:39.773] Kent Bye: Yeah, talk a bit about that experience of doing the HTC Vive. Did you get a chance to do the demos at GDC, or was this the first time you got to experience it? And maybe talk about your experience with both the technology and what the jam was like.
[00:06:51.456] Aaron Lemke: Yeah, I didn't get into the GDC demo, but my friend Alex Schwartz of Alchemy Labs, he is pretty close with Valve, and he made some of the demo content for GDC. So he's had a Vive dev kit for a while, and I got to go over to his house and try it a few months ago, and it's the real deal. There's some really cool demos. His demo is Job Simulator. It's this really goofy, like, physics-based, like, follow-a-recipe, and, you know, it's wacky and fun. That one was cool. Valve has an insanely cool Portal demo. It's all the Valve polish, and it's the Portal universe again, and some of the characters. Really cool. I think my favorite was called Tilt Brush which is a 3D drawing tool and it's just magical. Like one of their paint brushes, you can select a bunch of different paint brushes. They have a really great UI that pops up out of your hand and so you can select different paint brushes and one of them is a light brush. And so you're literally painting light and it's just like hangs there in space and glows and lights up all the other things that you've painted. It's just amazing. And I guess they got bought by Google recently, which is so awesome. Yeah, as far as the jam, I mean, I didn't really know what to expect. I showed up and they had three vibes set up kind of in the upper level where the jam was happening. Everybody was working downstairs and then the vibes were upstairs. And at first it was, you know, sign up for a time slot and come test your experience. And after about an hour they were like, just come up here whenever you want and just put it on a USB and load it on the computer and you can try it. So they were so open and they gave us so much freedom. It was really incredible. Like the Valve guys were there kind of making sure we didn't break stuff. But really they just let us do our own thing. I mean, there was never more than a five-minute wait to try something on the Vive. A lot of people were worried about that, because, you know, it was 70 or 80 people, and there were three Vives. So I was worried we were going to have to be waiting in line the whole time. But it was, I mean, at the end, when people were cramming, then the line got bigger. But, you know, most of the time, there was no wait. It was really fantastic.
[00:09:02.826] Kent Bye: So what were some of the experiences that you saw come out of that jam?
[00:09:06.187] Aaron Lemke: A lot of them were just really cool experiments, like with locomotion, for instance. Or, I guess, how do you expand the room beyond the room? Like, how do you make it bigger? Because, I don't know, a lot of the demos I tried, like Tilt Brush and Job Simulator, you're still very aware of the physical space that you're in, because the room is static the whole time, and you know, like, okay, that corner is that corner, and nothing really shifts. One cool thing a group did was they made a jet pack, So it's like a standing jetpack in the middle of the room and you have to walk over to it and like... get in it. And then you take off and you can fly around this little world and then you go land somewhere and you get out and you can walk around that site for a little bit. And then you take off and go explore some other site. And it was a really cool way of, like I said, expanding the room beyond the room. Some of the guys were experimenting with a locomotion system where you kind of like clamp onto the ground and pull yourself forwards through the world. So you're standing in the same place but you're kind of like swiping through the world almost. And that was kind of cool. I think some people got sick doing that. The smartest solution was just put an elevator in the corner of your room. So you're on the ground floor, you do some interaction. You go stand in the elevator, the doors close, you can't see, so you don't get motion sickness because you're going up. And then they open, and then you're in a new room. And they did that two or three times. It was really cool. It's a really great way to do things. That was cool. A couple people made Godzilla simulators, so you're walking around smashing stuff. Those were super fun. Somebody made a tower defense game. That was pretty neat. Oh, Jono has one of my favorites. I think it's called VRMT, and it's a world-building app inside VR. And so you've got a little UI that pops out of your wrist that's got a number of different meshes on it, like trees and bushes, and you can just, like, click on one and drag it out to the scene. You can scale it, you can rotate it. So you're, like, building worlds from the inside out. It's really, really cool. Like, he needs to go work for Unity or something. Because that's the dream. It's, like, a Unity editor where you can build the worlds from the inside out. My team ended up building kind of a 3d operation type game 3d maze where you can't touch the walls and We had little sort of like I don't know 4 foot by 4 foot 3d mazes that you'd have to navigate Without hitting the walls and then once you finish one maze that one disappears and it spawns the next one behind you and so you'd have to like turn around and go find it and That had a really nice effect. I think morphing the room and changing people's static reference points and things like that, it makes you forget where you are in the room. And a lot of people, when they got out of it, they were like, well, I feel like I just walked in a line for 20 minutes, you know? So we really got people lost in a good way. And it's not quite redirected walking, but it's sort of inspired by that. So that was like a really cool thing that we came up with. I mean, I would love to keep developing that. I think it could be really fun. Sort of like collecting Easter eggs, navigating mazes, and, you know, sound-based cues. So I like the idea of having to hunt around using your ears. Like you hear some sound that you have to collect, and you go like, where is that? Oh, I have to go over here. And then, oh, there it is. You find it. So yeah, it was one of the coolest things I've ever done. It was just so fun.
[00:12:33.122] Kent Bye: So does that mean moving forward, you're going to try to get an actual dev kit for the HTC Vive then? And what's that process like for you?
[00:12:39.852] Aaron Lemke: Yeah, I mean, the process is just you fill out the form on Valve's website and cross your fingers. I mean, hopefully they liked what we were doing with the Snake game and they'll want us to keep exploring those ideas and send us one. But, I don't know, I think it's kind of just up to Valve or HTC, who they believe in and who they think can come up with cool new techniques.
[00:13:05.677] Kent Bye: Actually, I did an interview with Joe Lugwud, and he said he was really interested in the snake game from the jam. Really?
[00:13:11.499] Aaron Lemke: That's good news. That's really good news. OK. That's awesome.
[00:13:16.441] Kent Bye: Just made my day. But yeah, so exploring these sort of innovative locomotion techniques, I think, is really interesting in terms of what you can do to kind of distract the user or kind of turn them around and ways of making it comfortable to be in a constrained space but yet make it feel like they're not. Because there's things that you can do in terms of turning that you can start to play with, turning them more than they are actually turning.
[00:13:40.191] Aaron Lemke: If we'd had more time, I would have tried something like that. You got a puzzle, and then the puzzle would spawn behind you, and so you'd have to turn around. So we had them turning around 180 pretty regularly. So I was thinking when they're doing that we could sense that and either over-rotate them or under-rotate them to really kind of mess with their sense of space. But yeah, we didn't get around to it. I also wanted to do something where like there's just a hole floating in space. Maybe there's a wall with a hole floating like in the middle of the room and you walk in and it's kind of a pipe and so you can like go into the pipe. But then you keep going and it turns and it becomes Kind of like an impossible space so it goes all around the room and you didn't see it previously But you have to go inside of it to see it.
[00:14:23.375] Kent Bye: That would be so cool Well, I could just sort of see the impact of using two-handed interactions within VR of how that sort of spurred a lot of Brainstorming here for you.
[00:14:32.566] Aaron Lemke: Yeah, I just want to get back in there And people towards the end of the jam figured out, oh, we can load our Unity project up on this computer and really iterate a lot quicker. Because otherwise, it was just build, go up there, test it, go back down, iterate. But man, if you had your own Vive and you hooked it up to the Unity editor, that's just the dream. You could do so much cool stuff. Yeah, I can't wait. Great. So yeah, what's next for you? Next is Zen Zone, a suite of meditation apps for the gear. And some of them, I'm trying to do more interactive meditation, you know, things that we can't do in real life. So visualizing the breath, visualizing the body, like I was talking about. I want to build a mini Zen garden. Like, you know, some people have those little sand gardens on the edge of their desk, and they'll rake them, and it's kind of calming. So something like that I think could be cool. If it's like a miniature garden right in front of you, kind of in that sweet spot, high detail, miniature zone in VR, that's like really cool to look at stuff. I'd like to do that. I was thinking about doing a rock skipping game, just like super ambient, nighttime, swipe to skip rocks. But then I was playing one of the Gear Jam entries and somebody, it's not entirely about that, but it's a feature in one of their levels, so I was like, Man, they already did it. But I guess I could do it differently. I don't know, I might still do it. That's kind of a Zen thing though, you know, it's like... sort of therapeutic.
[00:15:57.973] Kent Bye: Yeah, there's a Snowfight game that came out of Portland by Damon from Portland, which actually is super compelling. I think it's Snowfight, I'm not sure of the exact title, but yeah, it's basically from the outside it looks like not as compelling as it is actually when you're in there. That motion of actually swiping and throwing I think is pretty fun, so... But yeah, did you see anything here at SVVRCon in terms of demos, like the Morpheus demos or the Crescent Bay GDC demos or anything that sort of sticks out?
[00:16:27.933] Aaron Lemke: Morpheus was cool. I hadn't tried the new one. That was really, really nice. Last time I tried it was a year ago at SVVR. Yeah, cover shooter with move controllers in the Morpheus. It's just going to be so freaking fun. Like the London heist one, where at the end you're like ducking and shooting. It's just going to be too much fun. I met Fruxius, which was really cool. He showed me Neos VR, which is amazing. The guy is just ridiculous. He's so smart. Neo VR is kind of a tech demo, but also like power of 10, history of the universe kind of thing. It's just so cool, and afterwards I was talking to him and he's pointing out all this crazy stuff. He's like, yeah, we wrote this crazy shader that does all this stuff. And he's like, yeah, this one object, we had to use light fields because it was too much geometry. So we just put a light field in there. And he's like, man, you are so smart. But yeah, the demo is just really compelling, and that was cool. I didn't do Crescent Bay because I think I've seen, they were just doing the Unreal demo.
[00:17:31.803] Kent Bye: Yeah, the same from the GDC is basically the same demo reel. And a couple of things that I saw that really stuck out was the Sony demo. The London Heist actually gave me a really weird feeling to stand up and be shooting people. Because I don't necessarily do a lot of gaming and it actually was a big break in presence. I had to be like, this is not real. But it was sort of like I felt my body like, oh my god, I just shot a guy. And seeing your hand in there, and it was like, wow, this is intense. But I sort of stepped back and just told my brain, this is not real. Then I was just like, OK, just kind of going through the motions of doing everything. But yeah, it was kind of intense.
[00:18:08.075] Aaron Lemke: You know what you mean. I kind of think with shooters, we should make them abstract. Because if they get really realistic, people aren't going to want to play them. They're going to be traumatizing. You know, I mean you look at some of the really gory games that are out there now, it's like, I don't want to be seeing that when I actually feel like I'm next to the zombie or whatever that's bleeding on me. So yeah, I think abstract shooters are the way to go.
[00:18:31.125] Kent Bye: Yeah, I wonder, I especially think that games like that are gonna be rated higher in immersive VR than they are gonna be in a 2D. Like, a 2D would be fine, it wouldn't be that big of a deal, but, you know, when you feel like you're actually there, it just has an extra weight, I guess. And that was sort of interesting for me to note, but... But the other thing is that the Sony demos with all the robots, they had this sort of panorama of like this dollhouse with each level and you could just like lean in and each of these little kind of dancing robot things would do something. They would play out some scene and it was just so rich. I felt like I could have spent hours in that. And there's something about, you know, that sweet spot of VR that Paul Bettner talks about, like that they discovered with Lucky's Tale. You just stick out your hand and the distance between your fingertips to your eyes is that whole range that is right in there where you can like lean in and have this kind of tabletop feel and have things like dancing around. I think it's just that's some of the most compelling stuff that I've ever seen in VR before.
[00:19:26.223] Aaron Lemke: Yeah, I totally agree. That demo had really great audio too. I was super impressed. Sony has some awesome audio going on. Yeah.
[00:19:35.230] Kent Bye: Yeah, and I don't know if you got a chance to see the sub pack, which is like, you know, put a pack on your body and it just does the low bass just completely rocking out. It felt like I was in a standing next to a subwoofer and a giant like dance club when I was like listening and just it kind of put my body into a completely another place of immersion. It was just audio wasn't even anything visual. It was just incredible.
[00:19:57.403] Aaron Lemke: I'm a huge fan of the SubPAC. I think it's one of the coolest devices. I went to their, I think after Oculus Connect, I went to their office and tried it for the first time in Los Angeles. At the time I had Nebulan, the ambient puzzle space game, on my gear and I just plugged it in to the SubPAC. You take the audio out from your headphone, plug it into the SubPAC, plug your headphones into the SubPAC. And without doing anything, it instantly made the experience 10 times cooler. It was like, you don't have to do it, you know, I guess you have to, I didn't, wasn't conscious of it, but I guess I mixed enough low end into the soundtrack, in the sound effects to where it worked with the sub pack. But like, I didn't do anything, I just plugged it in and the experience was so much better. I think like for the money, that's the best device for making your experience better without having to do anything.
[00:20:44.586] Kent Bye: Yeah, I think I was really sold on terms of the potential of using just like low frequency bass as a way of giving that really visceral haptic feedback just for the audio. And yeah, like you said, it seems like pretty simple to integrate if you just like plug the audio out, that's great. And I can imagine how having sort of rhythmic beats rippling through your body and changing that dynamically with whatever you're seeing visually, I think that's going to be sort of next level immersion there.
[00:21:10.070] Aaron Lemke: Yeah, I'm interested. I ordered one a couple weeks ago. And so I'm interested in exploring how do we use it for meditation and sort of like, yeah, soothing the body in a physical way.
[00:21:20.189] Kent Bye: not only soothing, but I've done, you know, shamanic breath work, which is a more cathartic breath work. So when you're breathing, you have quite a lot of really vibrant music. And when that driving drums or bass is going through your body, it could be a pretty cathartic experience as well. So I'm not sure how well that would translate necessarily to a VR experience, but I know that it could also not just be meditative, but actually cathartic of helping move a lot of trauma through your body. Yeah, I love it. Great. So finally, what do you see as the next, now that you've seen the Vive, I guess you probably see a little bit more of the potential of VR. But what do you see as the ultimate potential now that you've seen room-scale, two-handed interactions in VR?
[00:22:04.112] Aaron Lemke: I mean, I still think virtual field trips are gonna be so cool. Especially in the Vive, if we can do multiple people at one time. And seeing Fructious' thing too, honestly, it's like, really makes education compelling. Yeah, I've said this before, but I think it would be smart to put really good VR in schools first. Because, you know, if kids get to the point where they have awesome VR at their house, and they have like, okay VR at school, they're gonna go and be like, eh, whatever. Like, I'd rather be playing Call of Duty in the Vive or whatever, but if they have, like, the first time they experience VR is in a school, and they're learning about something, like, they're gonna remember that for the rest of their life, and it's gonna have a huge impact on them. You know, I'm hoping the powers that be are considering that, but who knows.
[00:22:50.071] Kent Bye: Yeah, I think there's been in the past perhaps an over-reliance on throwing computers in school, thinking that those are going to be solving a lot of educational issues. And I think that to some extent that could be true for VR. Like, I don't think it's necessarily a good idea at this point to throw virtual reality, HMD, into a school without the ecosystem of content. So I think the content has to come there first. And I think that the gaming is going to probably more likely have the revenue stream to be supporting it. So it's more likely for them to have it at home. But something that actually has been told to me through an interview is that you think about a lot of educational experiences, there's not a lot of adult education experiences. A lot of the educational experiences are really geared towards children who are still in school or, you know, elementary or middle school or high school. And so, just to think about, like, the adult education opportunities there for people to learn about things. And so, if you can make something that's compelling for adults, then perhaps it could also be compelling for children as well.
[00:23:43.596] Aaron Lemke: Yeah, I recently reconnected with my high school art history teacher, and she's one of the best teachers I've ever had. And she uses projectors, so she's got a giant projector in her room. And most of what she's teaching is 2D art, paintings, but she teaches a lot of architecture too. And she always complains it's hard to give kids a sense of what a building feels like. It's not as compelling to see a picture of it. So I brought her over and I showed her the gear photo spheres of some of the architectural stuff. And it just blew her away. And so we started talking about making kind of adult education, sort of guided tour experiences of classic architecture spots. And yeah, like you said, if it's compelling for adults, then probably going to be compelling for kids too. But I like the idea of selling an education app just on the gear store and being like, hey, this is an alternative. You can try this out. You know, because some people are going to be really into that. Awesome.
[00:24:40.267] Kent Bye: Anything else that's left unsaid?
[00:24:42.530] Aaron Lemke: I'm looking at medical now, medical VR. I think there's a lot to be done there. Yeah, I guess with the Vive, like physical therapy is really interesting. So what if you can take what little movement someone does have in a limb and amplify it in VR? So you create this kind of false positive feedback loop. Who knows? I don't even know what that would do, but that might be a really interesting thing. I mean, they use this thing called the mirror box now to treat phantom limb pain, and that's basically virtual reality. It tricks your brain into thinking the limb that you're missing is there, and that's how they cure phantom limb pain. So, yeah, stuff like that, just like weird neurological effects that we don't really understand yet. For physical therapy, you know, pain is the one I'm looking at right now, distraction therapy, and I guess it's not really anything high-level going on there, it's just if you can get the patient to not focus on the pain and focus on a really stimulating VR experience, then it's gonna be less painful for them. Yeah, there's just so much there's so much fun stuff. I don't know. It's gonna be a fun couple of years I hope we get to do this every year so we can go back and be like she had no idea what he's talking about back then Awesome well, thanks so much.
[00:25:55.368] Kent Bye: Thanks Kent and thank you for listening if you'd like to support the voices of VR podcast then please consider becoming a patron at patreon.com slash voices of VR
