#397: Biometric Feedback with Guided Meditation VR + Augmented Reality on Gear VR

josh-farkasJosh Farkas has given over 6000 VR demos over the last couple of years, and he’s been in the position of having had to try to explain the potential of VR to many skeptical businesses. That’s in part because his Cubicle Ninjas was primarily a web development and creative agency before becoming an early adopter of VR. They’ve released two virtual reality applications so far including Guided Meditation VR and the augmented reality filter app Spectacle. I had a chance to catch up with Josh at SXSW in March where we talked about using the Gear VR to detect heart rate and provide biometric feedback, releasing the first augmented reality application for Gear VR called, and some stories from the frontlines of evangelizing virtual reality.

LISTEN TO THE VOICES OF VR PODCAST

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

Rough Transcript

[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to The Voices of VR Podcast. On today's episode, I have Josh Varkis, who is the founder of Cubicle Ninjas, which is a web design shop that also is doing a lot of immersive experiences. So I first met Josh back at the very first Silicon Valley Virtual Reality Conference and did an interview with him back in 2014. And he was working on a meditation experience. And just within the last couple of months, that guided meditation VR experience has released on the Gear VR, as well as the first augmented reality experience for the Gear VR called Spectacle. So we'll be talking about those two experiences as well as some of his experiences of traveling around the country over the last couple of years evangelizing virtual reality at different conferences and businesses. So that's what we'll be covering on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. Today's episode is brought to you by the Mind and Body VR Hackathon. The Mind and Body VR Hackathon is a 48-hour competition that's happening on July 22nd to 24th in Los Angeles. It's being sponsored by HTC and Quest Nutrition and has over $20,000 in gifts and prizes. Some of the areas of focus are mindfulness, brain training, education, as well as fitness, nutrition, and medicine. It's starting the Friday before SIGGRAPH, and we'll have a community showcase on Sunday night. So for more information and to register, go to mindbodyvrhack.com. So this interview with Josh happened at the South by Southwest Festival's VRAR track back on March 17th in Austin, Texas. So with that, let's go ahead and dive right in.

[00:01:49.742] Josh Farkas: My name is Josh Farkas, and I'm the CEO of a company called Cubicle Ninjas. And we're a design and development agency, but we also have created a few apps in the VR space. So one of the ones that we're showing here today at South by Southwest is Guided Meditation VR, which is a virtual relaxation app. And we're also showing off another one called Spectacle, which is kind of like Instagram, but for reality. It's the first augmented reality app for the Gear VR.

[00:02:11.241] Kent Bye: Right, so you have a number of different filters that you're seeing. So what was the intention for creating this app?

[00:02:18.102] Josh Farkas: We actually played a lot with pass-through stuff when we were originally working with guided meditation. And so we were thinking, I'm obsessed with the concept of bridging, bridging realities. And so moving from a real, regular, normal, traditional, safe space to the world of a virtual space. And so we were originally doing that of starting off meditation in your room and then slowly transitioning to this virtual space. We didn't find it that compelling, to be perfectly honest, right now. Maybe down the line it'll get more interesting. But one of the things that, while we were playing with that technology, we stumbled upon the fact that, oh wow, filters, living in a sketch world, living in an 80s movie, those filters really felt hilarious and fun. You can be on a train, you can be in a car, anywhere, all of a sudden it becomes hilarious. Everything becomes so much more interesting and fresh. So that was really, for us, the turning point where we realized, hey, this might be an interesting product into itself.

[00:03:05.373] Kent Bye: I was listening to this author talk about like metaphors for time and she was saying like you could think about it like a lens where you have different lenses and like if you put a wide-angle lens onto a camera it distorts your sense of space and time. And then if you have a macro lens, you have a different experience of time. And so there's a lot of times that we don't necessarily even question the lens of reality that we have because it's not being altered at all. And so to me, I think it's really interesting some of the implications of starting to put these different lenses over our eyes and experience the world in a new way. And I'm just curious of the early days of playing around with this, what you've found.

[00:03:43.513] Josh Farkas: Yeah, that's my passion for where VR is. It's kind of where the human mind meets this technology and kind of where it might be interesting to kind of play around. So there's a lot of weird ones that, for me, were kind of mind-blowing. So we have different categories of filters that you can apply to reality, and one of them is called Animal. And so these are all how animals actually view the world. And so for us, it was really neat to see, like, what's the point of view of a dog, right? Like, we know it's black and white and a little blue and yellow, but what does that actually feel like? What does it look like? And if I could be a bumblebee, what would that look like? And so that's for me the interesting part. So that was one that was cool and that actually plays out into you starting to kind of feel a little bit more empathy for what it must be like to live in the world this way. The other one that I thought was a lot of fun was night vision. So we have a series of night vision filters. And so I would walk through the office and completely pitch black and just to test it out to see if it worked. Again, to just augment our senses in this little minor way like was really cool because it was like all this data is there. We just need to filter out the bad stuff. The same thing with we have there's a filter that's a faux thermal filter and it works great. And so it's like, you know, it's these are the things that are just all this hidden data is around us. And we kind of just we filter it out and we live our daily lives. But for me, that's the exciting part of like bringing that to light and letting me like kind of bathe in the joy of all the stuff that's around me that I don't even pay attention to.

[00:05:01.037] Kent Bye: That's really interesting, the concept of turning your Gear VR into a night vision goggles. And I think that we haven't really spent a lot of time just observing the world in night vision goggles. So who knows what we can't see? So what have been some of your favorite memories of using this spectacle app?

[00:05:19.085] Josh Farkas: You know, I think so far, like for me, it's just that, you know, I think there's so much joy to be had in simple modifications of reality. So, for example, seeing the world as a sketch blueprint is like probably my favorite. I grew up as like, I was obsessed with comics and I did comic art for like 15 years. So now I can apply a filter to reality and live in a comic book world. And that, for me, is just so delightful. And I would do that all the time if I could. It's really, it's just joyful. But the fun thing for showing it at shows is to see young people kind of have fun and play with reality. And so I've enjoyed here at SoftBuy having little kids try it, having young adults try it, and just kind of run around and see, like, my hands are huge, and just get so excited again about what is effectively a convention center.

[00:06:07.582] Kent Bye: So from your perspective, as someone who's running a design agency, getting it as kind of an early adopter of VR, I know that you're at the very first SVVRCon as an exhibitor and kind of putting yourself out there as doing these VR experiences for different companies. What has it been like to see this industry grow over the last couple of years?

[00:06:25.336] Josh Farkas: It's surreal. I've never seen anything like this in my life. And it's been a pleasure to be part of. It's really been a joy. And so, you know, at the SVVRCon, I remember I got up and I was right after Oculus and I was so nervous. And I got up on the little pitch stage and was like, we're going to make VR experiences for companies. And everybody was like, huh? You know, that's a thing? And so, like now, two years later, it's so commonplace, like it's an everyday sort of occurrence. And so, one that has been cool to see that transformation of being laughed out of conference rooms. You know, I would fly across the country and, you know, go into LA and they wouldn't put it on because they didn't want to mess up their hair. And so, in the middle of a boardroom, and me just getting rage-filled, like, and here we are two years later, those folks call us up and go, yeah, you know what, you were right. And so, it's been fun to also get in front of clients and show them, hey, maybe we can help you in some ways you didn't imagine. So that's very, very cool.

[00:07:20.491] Kent Bye: What have been some of your favorite experiences that you've created over the last couple of years then?

[00:07:24.934] Josh Farkas: For customers, we've created some really fun things. We created some for a health care company. It's a modification of what we're doing, using some of the knowledge of meditation in virtual spaces. And so that's coming out pretty soon. That one I really love. We've also built another one for a neuro training company, which for me was a lot of fun in terms of like, being able to increase the ability for you to track items all at once and that that's just something that your brain gets better at with practice and so for me that was like engaging on that level but for the ones that we make I mean I'm really excited about guided meditation VR as like our kind of core app I think really it's all about kind of bringing people to a safe space that they can select and modify to make their own and so I grew up, when I was a kid, that thing didn't exist. It was like the ability for you to have a breather just never occurred because I was poor, I had a single mom, we lived in a one-bedroom apartment. Stress was our lives. And so we try to build tools that hopefully for people who may not be able to afford the big crazy VR stuff, but that at least can take a break from reality in an affordable way at a mass scale. That's our hope.

[00:08:33.024] Kent Bye: What have you been hearing back as feedback in terms of maybe some stories that people have been implementing this guided meditation VR into their lives?

[00:08:42.322] Josh Farkas: There's a few stories. The one that really gets me every time is there's a school in New Zealand which started implementing it for anxiety and for behavior disorder. And so originally the principal reached out and said, is it okay? And we were like, absolutely. We were so excited to have you use it. And then a couple of days later, we had a child from that class who was dealing with anxiety for an upcoming surgery, write us a letter that every time they get nervous, you know, they just think back about that space, about the Japanese garden. you know they feel a little bit better and so for me that's really touching because to be able to teach for kids dealing with anxiety is our thing and it's kind of this weird emotional so to give them some sort of space to kind of center back on I think it's so exciting. Another memory that I think back on a lot was that We had a young man come up and he suffered from a condition where he was unable to move from the waist down and barely his arms. So he was in a wheelchair and confined and he used to be a pro Halo player. So it was a situation where he now needed to do meditations but he hated it because it was boring. you know, he had a chance to try it and he came out and he started moving his arms towards me and I asked his mom, you know, what does this mean? And she goes, it means he wants to hug you. And so, like, you know, here I am hugging this young man who really was able to experience something that maybe he didn't get to very often in his life of kind of flying around and just being a little bit free again, so.

[00:10:04.939] Kent Bye: Yeah, to me, I just am really struck with the power of VR to change lives. And I'm curious from you moving forward, how else do you want to help push this forward? Or what do you see happening with VR as a modality for healing?

[00:10:17.454] Josh Farkas: Yeah, so I think when we originally were at kind of launching the app, Our focus has been feedback loop. Without a feedback loop, all this is kind of useless. And so it's playing at you and that's great, but it needs to react to you. And so with guided meditation VR for the gear VR, we've started doing that. So we now have the ability to track heart rate and we're tracking breath as well. And so for us, that's a huge breakthrough because now we can do so many things based on that. So I think continue to grow that feedback loop, continuing to grow the ability for us to understand how to slightly modify these things so that they can have the maximum benefit. What we're also curious about is that there's not a lot of data out there, really, about what relaxes people in terms of spaces. We know some from environmental psychology, but when you read a lot of it, it's still pretty limited in terms of the depth of how that impacts us. My intention is, you know, I feel like in six months to a year, we'll probably have the largest data set in terms of about how spaces impact people's ability to relax and what kind of elements and triggers are there. And so once we get some of that data, we can start to actually figure out and make it more effective. And so maybe there is a kind of a entry level and a pro kind of meditation or relaxation that allows us to go to that state so much faster in terms of pain relief or just dealing with anxiety or other challenges.

[00:11:37.795] Kent Bye: How are you getting this biofeedback into the app?

[00:11:56.652] Josh Farkas: We then log that in the beginning and the end of the experience. And what we're showing users is that, one, that this is something that you're in control of, but also that there's a marked improvement in A to B. And so they can see that there's a slightly gamifying effect that, hey, this has a benefit to you. We can also then gather that data and then recommend in the future experiences that work for them. So we call it a relaxation concierge. And so we can kind of proactively recommend things for their uses.

[00:12:24.245] Kent Bye: Yeah, I know the first time that I talked to you back in May of 2014 at SVVRCon, I mentioned HeartMath, which is like a way of, you know, putting a thing on your ear to be able to measure your heart rate. And the thing that they were looking at was actually heart rate variability, showing that, like, as your heart is varying over time, that's actually kind of giving a really good indication as to how coherent your whole system is working. And so it sounds like you're just doing, like, heart rate at the beginning and to notice a difference, but Well, first of all, I think in the long run it would be great to be able to get that to the level of detail to the heart rate variability, but making do with what you're already capturing with the heart rate, how did you come across that? I mean, is there anything else doing like measuring the heart rate from a phone?

[00:13:06.712] Josh Farkas: Yeah, yeah. So it's really awesome. So we had a talented engineer who literally appeared on our doorstep one day and said, I just quit Zynga, and I want to change the world with VR. And I said, you're crazy. And so he said, OK, let me prove it. And so he sat down, and we've been solving very crazy problems since then, one of which he's been working on for four years, which is how to deal with heart rate. So it was kind of, you know, I feel really kind of blessed in that way to have stumbled upon Matthew, who's just simply brilliant. But yeah, so you're right. The heart rate variability is where we're going to next. And so all built in the phone, no additions, and we're excited because we think that that steers VR one step closer to be considered a medical device.

[00:13:47.552] Kent Bye: You said put your thumb on the back. Can you use any, like your index finger, or why the thumb?

[00:13:51.856] Josh Farkas: Oh, no, yeah. So, sorry if I misspoke. Any finger, usually the index is usually the best, and the hardest thing that we're finding right now is just training people where to put their finger in 3D space, because it's like, even people have problems finding the size of their head with the Gear VR on, so, like, getting some random point in space has been a fun training challenge for us. of how to use tutorial. But yeah, you put it on there. We capture about 30 to 45 seconds worth of data, and we've compared it versus very expensive blood cuff monitors, and we're within one beat. So it is precise.

[00:14:22.034] Kent Bye: Wow, that's pretty mind-blowing. Is there any other biofeedback that you're getting from the phone?

[00:14:26.113] Josh Farkas: Yeah, so we're also using the microphone and we're able to kind of pay attention to breath. The only challenge with that though is that right now we're showing at South by Southwest with the VR AR. It's a very noisy room so that data is totally useless and so it's only good if you're in a quiet space. And so we use it for multiple things. We use it for both capturing the breathing rate but we also are able to use it to affect the world. So we kind of pay attention to that and we have like ambient items around you will actually slightly blow when you breathe. So things like that we think are kind of subtle interactivity really matters.

[00:15:00.399] Kent Bye: And what type of experiences do you want to have in VR then?

[00:15:02.841] Josh Farkas: Yeah. It's funny. I think probably the ones that I'm most in love with at the moment and continue to be curious about are the creation things. Like anything using the ability to build worlds within VR, the ability for you to kind of get lost in a creation. And that for me is where my mind's at, where my heart's at.

[00:15:27.411] Kent Bye: And finally, what do you see as kind of the ultimate potential of virtual reality and what it might be able to enable?

[00:15:35.983] Josh Farkas: That's a hard one. But I think realistically, I think most people still tend to think about virtual reality in terms of entertainment. And I don't think there's anything wrong with that. But I think virtual reality is going to change the way we view and deal with our daily lives, whether it's five years or 10 years or 20 years, but it will happen. It's an eventuality. So my hope, my thoughts are a lot placed on healthcare and in terms of How can we change the lives of millions of people and provide a more relaxing, more comfortable, more safe and reactive world? I think in the longer term though, I think the sky's the limit. There isn't a day, we've been in VR for two years now, every single day someone brings up a completely new idea that I never thought of. I thought that would have stopped by now, but it hasn't. And so I think, you know, as long as there are smart people who are interested and curious and willing to spend those late nights to build something, that's, for me, is what gets me excited. Because I don't think we can run out of ideas. I think we're going to keep building really powerful, interesting stuff that amazes for years to come.

[00:16:36.226] Kent Bye: Is there anything else that's left unsaid that you'd like to say?

[00:16:39.368] Josh Farkas: I'm just grateful for you and this podcast and the community. This has been the pleasure of my life. So, thank you.

[00:16:45.373] Kent Bye: Awesome. Well, thank you. Thanks. So that was Josh Farkas. He's the founder of Cubicle Ninjas, which is a web design shop that also is doing a lot of virtual reality experiences, including guided meditation VR and spectacle VR. So there's a number of different takeaways that I have from this interview. First of all, I just want to say that I really respect Josh and a lot of work that he's doing in VR. If you're not following him on Twitter, you should definitely check him out because I think he's got a lot of really insightful analysis and reflections on what's happening within the VR community. So check out his Twitter, at Josh Farkas. Also, just the innovation that he's been doing with trying to incorporate different biofeedback metrics within these VR experiences. I think it's really interesting and something that we'll see a lot more of that in terms of being able to actually have concrete quantitative metrics to be able to track our progress over time and to be able to see what impact some of this is having. And I think the other thing that was really striking was just hearing more about Josh's background of just growing up poor and just needing these types of technologies that are able to allow us to take a break, take a step back, and really de-stress ourselves by meditating and breathing and with the help with some of these technologies. It's really amazing to hear some of those initial stories of feedback that he's been getting both from conferences but also from different classrooms. Just imagine being able to have access to these types of technologies as a teacher and trying to cultivate the skills of mindfulness to people who are growing up. These are going to be extremely valuable skills to be able to be centered and be present in this world that is just overwhelming and flooded with information all the time. I was also really struck by some of Josh's stories of going in and being laughed out of different meetings and some people not even wanting to put on the headset because it would mess up their hair. And Josh and Cubicle Ninjas have really been in the trenches of exhibiting and doing lots of demos, getting lots of feedback and promoting their products to lots of people, even going to E3 a couple of times, which doing a meditation app at E3, that within itself is, pretty remarkable that he's out there doing that. You know, when a lot of people think about virtual reality technology, they see it as very isolating and escapist in a lot of different ways. And while that could certainly be true in certain cases, I think in the case of something like meditation VR, I think it actually can connect us more to being fully present in the moment and in our bodies and more connected to reality. to be able to have an escape from all the stress, but to be able to take a break to be able to nurture ourselves. And if more and more people start using VR technology in this way, then I think overall that can just make the world a better place. So fully support what Josh is doing here, and looking forward to helping judge at the MindBody VR Hackathon that's coming up here just before SIGGRAPH on July 22nd to 24th. So with that, I wanted to thank you for listening. And if you enjoy the podcast, then please help spread the word. Share it with your friends on Twitter or leave a review on iTunes. Or if you want to help financially, then you can go to patreon.com slash Voices of VR.

More from this show