The VR hardware ecosystem in China is fairly self-contained, and Nolo VR ran a successful Kickstarter campaign to bring 6DoF hand tracking to mobile VR headsets, but also giving access to your SteamVR library through your mobile phone.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Support Voices of VR
- Subscribe on iTunes
- Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. So at CES 2017, I had a chance to kind of walk the showroom floor and see what the vibe was like. And what you tend to see is a lot of like, I guess, middle tier startup type of companies, a lot of them from China. you don't tend to see a lot of the major players that are showing any of their latest hardware because they have kind of their own events, their own developer events and their own kind of ecosystem that's already pretty well developed and filled out. And so HTC was there with their own booth and showing their latest technologies, but it's not something that you see either Oculus or Daydream or PlayStation VR. You know, these are the kind of like the top tier. So I tended to see a lot of hardware that was kind of like, well, I wonder where this is going to end up going. Well, I ended up doing an interview with NoLoVR and what they're doing is kind of interesting because they're taking this ultrasonic device and putting on top of a mobile VR and be able to translate mobile VR into six degree of freedom. experiences, both mobile VR experiences, but also be able to broadcast to your headset, like SteamVR. So to be able to see wirelessly from your PC to your SteamVR and be able to broadcast to a mobile VR headset, which was kind of an interesting kind of link that they were making here. And so this was right before their Kickstarter. And they did have a successful Kickstarter that ended up raising almost five times their goal. They had $231,000 out of their $50,000 goal. And I just went to Qingdao, China, and I saw the latest iteration of the Nolo VR. So they're still around. They're still making these sensors to be able to add in six degree of freedom capabilities onto mobile VR headset. Now, of course, within the next year or so, we're going to have the Santa Cruz and we're going to have each of the different companies, I think, are going to have their own solution. But The thing about the Chinese ecosystem is that they're kind of self-contained. So it'll be interesting to see what ends up happening with Nolo VR as they move forward and how they'll be integrated and adapted into perhaps the larger content ecosystem within China. So I had a chance to talk to one of the co-founders of Li Robotics, which produces Nolo VR, Lisa Zhao. And I had a chance to talk to her on Thursday, January 5, 2017 at the Consumer Electronics Show in Las Vegas, Nevada. So, with that, let's go ahead and dive right in.
[00:02:38.082] Lisa Zhou: My name is Lisa Zhao, and I'm the COO and co-founder of Li Robotics. And we are behind the product NOLO, which is a motion tracking system that provides six degrees of freedom motion tracking for both the headset and the controllers. So the whole product consists of one base station, one headset marker, and two controllers. So with our self-developed technology, millimeter level of accuracy and low latency could be achieved with high affordability, high portability, and really ease of use to set up.
[00:03:10.536] Kent Bye: So the thing that I found really interesting about the demo that you're showing here is that you're actually finding a way to get SteamVR content on a mobile headset with your system. And so tell me a bit about how you were able to do that, and then your approach of having a lack of content for fully room-scale mobile VR. A lot of it is on these high-end systems, but you're trying to, in some way, provide access to people with mobile phones.
[00:03:35.501] Lisa Zhou: Right, so our logic behind it is that our vision is to make mobile VR a really great ecosystem so that VR can be a really accessible thing to the masses. However, there's definitely a chicken and egg problem between the developers and users, right? So if there are not many users, developers don't want to develop for it, but if there are not as many games, then users don't want to buy it. So from our point of view, in order to solve that problem, by providing the access to SteamVR, so users can already access a bunch of existing games with their mobile VR headsets, and then once there's a user base, the developers for mobile VR, room scale, more interactive games will be more willing to step into this field as well. In terms of the technical side, how we are able to do that, so thanks to SteamVR's OpenVR Drive, we can actually input our data into the system and to work with all the games. And also we're partnering with RiftCat for the screen casting, so that the computer's image for SteamVR can be casted on your mobile phone as well. And so playing SteamVR games on a mobile VR system with no LOS, the input system, would be possible.
[00:04:47.996] Kent Bye: So maybe you could tell me a bit about the base station and how you're actually kind of logistically doing this six degree of freedom tracking.
[00:04:55.863] Lisa Zhou: Awesome. So in terms of the technical side, the base station emits out signals, and that includes a fusion of optical, acoustic, and radio signals. And then the markers serve merely as the emitter, and then the headset markers are receiving the encoded signals they need and calculate on board for the position. And then the controller transmits back to the headset marker, and that goes into your phone or headset. if it's Gear VR.
[00:05:26.836] Kent Bye: So how are you using sound? This is the first I've heard people using any sort of acoustic mechanism to be able to do positional tracking, and I didn't necessarily hear anything.
[00:05:36.282] Lisa Zhou: Right, so we use ultrasonic and our tech director is a PhD in the acoustic field and so he kind of came up with this idea because it's a really brilliant way to track like dimensional data and so in addition to the optical part of it. Also more importantly the addition of acoustic part we don't have to use multiple markers for tracking so for one marker is sufficient for the three axis positional data.
[00:06:06.129] Kent Bye: And so coming from using a lot of SteamVR with the lasers and Lighthouse, I know the stability that that positional tracking provides. And using this system, I did loose tracking a few times and there was a little bit of latency that I think from people who are used to using SteamVR would be pretty unacceptable level of latency, I think. If you're just getting into VR for the first time, there may be some level of tolerance that people may be able to tolerate for that level of latency. But yet, too much of that latency could actually get them sick. And so maybe you could talk a bit about, like, you're doing a number of different things there. You're both screencasting onto the mobile phone, so there's latency introduced there, as well as the additional latency of the communication. So maybe you could talk a bit about how you're addressing that, and how you're measuring in, what kind of metrics you're trying to aim for that.
[00:06:55.235] Lisa Zhou: Right. In terms of the loss of tracking, so there is a kind of unique situation here at the CES because there are just so many different devices. It's a very complicated environment. So our transmission using radio signal kind of gets interrupted at times. And so that, for one, it's not a problem when you actually buy it at home and play, and two, we're definitely trying to solve that problem as well. And then in terms of latency, the SteamVR part It's a little bit more complicated. So one, like the screen casting has a fair amount of latency as well. And so our positioning latency is less than 20 milliseconds because our frequency is at 60 hertz. And we're currently trying to increase that frequency as well by fusion of IMU data. So that part will be improved probably in the near future. But in terms of how much it will be improved, I'll follow up on that because I can't say for sure since we haven't finished doing that. And then in terms of the transmission of data from our device to the PC, there is some latency there right now as well. That we're probably going to solve very soon because that's an easier problem to solve. In terms of the screen costing latency, we're definitely trying to work with RiftCat and improve that as well. But since we're using their software, that's not something very much in our control. I know RiftCat has a pretty big following as well, and the users are having a good experience considering how much money they're paying. And I think that's a great starter kit for people who are interested in VR and want to do more with it.
[00:08:34.081] Kent Bye: Yeah, one of the things that I was trying to do with your controllers was to push what I would ordinarily see as the SteamVR button to go into the menu system. And so, do you plan on doing a one-to-one mapping to what the default SteamVR controllers are able to do with the trigger? And there's also grip buttons, but they also have, you know, two buttons as well as kind of a trackpad as well. So, do you plan on making the initial release of this fully compatible with, like, the existing SteamVR Lighthouse controllers?
[00:09:03.517] Lisa Zhou: Right, so the answer is yes. Since we're making it compatible with SteamVR and a lot of users are using it to play SteamVR games, then for the buttons we're definitely trying to make sure that they have the full experience. So the answer is yes for the buttons. As for the future, in terms of like if it's a fully mobile VR experience, we'll probably start thinking more about what is actually necessary if it's a more portable and a little easier experience I guess. Because right now with the controllers I feel like some of the Steam VR games can get quite complicated that people who are not hardcore gamers are having a hard time to get like the hanging outfit. So right now, for this version that we're releasing, we're definitely going to have the same experiences in terms of controller. But for the future, we'll see what the users want.
[00:09:53.494] Kent Bye: And talking to HTC's president of China, he was telling me about a lot of the initiatives from the government of China to be able to really support a lot of these immersive technologies. And maybe you could talk a bit about that ecosystem and environment for startups there in China and what you've personally experienced with NOLO.
[00:10:13.045] Lisa Zhou: Right. So I think the government in general is definitely pushing for innovation right now, since China's economy is in this turn from more manufacturing to more innovative ideas and creative companies. So I think the government is having a great initiative over there. I mean, in terms of how effective it is, because creative ideas can't be cultivated by government money so much. It could help, but at the end of the day, it's really about what game changers are out there, and we're definitely trying to be one, but it's not that easy. So, in terms of what we've experienced, we've got funding from the government, but it's not significant, but it's good that we're getting the additional money for free.
[00:10:56.853] Kent Bye: And what are some of the big companies that are in the VR space that are from China that are on your radar screen that you look to? Because, you know, coming from the United States, it's hard for me to really track what's really happening in China. And what are the ones that you see are kind of really making an impact in China?
[00:11:13.402] Lisa Zhou: Let me think. I think there are a lot of Chinese companies that are making impact, especially in terms of the virtual reality scene. A couple of them are doing great. I mean, in my personal opinion, there is a company called, I don't know how they pronounce it, but D-L-O, D-L-O, and they have this virtual reality glasses that actually look like a glass. In my opinion, if you are gonna make a standalone all-in-one headset, and it looks exactly like the PC one or the mobile one, it kind of defeats the purpose. So I think their way of going, making it actually very comfortable and it looks great, it's a great way to go about it. So I think something more creative on that end and more consideration of user experience could be a great step.
[00:12:03.170] Kent Bye: And how did you get started into virtual reality? What was your inspiration to co-found this company?
[00:12:09.365] Lisa Zhou: Right. So our company started out more as like a more technology focused company. So we wanted to do more like 3D positioning and tracking technology. That's how we started. And then getting into the virtual reality world is actually when we bought the Oculus and used our positioning technology to work with it and we realized how great it is. And then I think VR is such an interesting thing that it's really hard to market. As in like, if people don't try it themselves, it's hard for them to get the idea of VR and get the fun of it. So yeah, since the first time we tried it on Oculus, especially with our system, we were like, wait, so you can do this in a 3D world, and we actually created a 3D mouse. So we kind of got into this field by accident, but it's great once we got it. We think it's very interesting.
[00:12:57.790] Kent Bye: Yeah, I think it's really interesting to see Tilt Brush in a Gear VR because Tilt Brush is one of those experiences that can really be a profound experience for a lot of people to be able to paint and draw in 3D. And maybe you could talk a bit about some of the reactions that you've seen of being able to provide access to Tilt Brush to people using your system for the first time.
[00:13:17.280] Lisa Zhou: Yeah, like you can see people like passing by who probably doesn't know much about VR at the CES and they're like, wait, they are like painting in 3D world and they are just like very excited about the experience. So I think when people are trying it and they'll be like, wait, your device is $99. So for $99, I can actually get this whole experience and people are very excited about the potential and like how many users potentially they're going to be for developers as well.
[00:13:44.834] Kent Bye: Great. And finally, what do you see as the ultimate potential of virtual reality and what it might be able to enable?
[00:13:52.217] Lisa Zhou: I think, in my opinion, I don't necessarily, I don't know, people may not agree with me, but I don't necessarily think the virtual reality is going to be as big as phones as the next platform. I think virtual reality is a really interesting, provides a really interesting experience that is for gaming entertainment and also some other fields for simulation, like education side of it. So I think I always summarize it for the customers, it's more of the game and entertainment, and for the business side, everything is education or training, if you summarize it. Because whatever you use it for, whether for medical or education or military, everything is basically to give you a virtual environment for you to train for certain movements or certain kind of goals. So I think those are the two major things for VR.
[00:14:41.937] Kent Bye: And what's next with no load? I think I heard you mention that you're going to have a Kickstarter.
[00:14:47.055] Lisa Zhou: Yes, so our Kickstarter campaign is going to start end of January. Our perk is going to be $99 for Nodo, plus a RiftCat membership, plus a cardboard. So if you haven't done any VR stuff before and only have like a Steam gaming experience, but you have a laptop, you can totally just get $99 pack and then start playing Steam VR games and that's all inclusive. Also, another target audience now for our Kickstarter campaign is the mobile VR developers. We really hope that we can build the mobile VR ecosystem together with the developers. And the ultimate goal is to make mobile VR fun. And there are so many contents that we can actually push forward for the whole ecosystem.
[00:15:33.265] Kent Bye: TIMOTHY JORDAN-PETERSENE Awesome. Anything else that's left unsaid that you'd like to say?
[00:15:38.145] Lisa Zhou: Sign up at our website right now, www.nolovr.com. And if you sign up now, you get a chance to get priority access for shipping, and also you can have a day before you launch us to be the first 100 to get the discounted price at $89.
[00:15:54.856] Kent Bye: Awesome. Well, thank you so much. Thank you. So that was Lisa Xiao. She's one of the co-founders and CEO of Li Robotics, which produces NoloVR. So I have a number of different takeaways about this interview is that first of all, I just had a chance to try the latest demos of Nodal VR there in Kingdown, China, and it's certainly getting better. They keep iterating and getting it better and better. I wouldn't say that it's necessarily on par of having like a tilt brush type of experience within, you know, HTC Vive. There's still a little bit of latency. There's still, you know, a little bit of precision. It's not completely like always a sub-millimeter accuracy, although it was really good and good enough. And I think that when it comes to like just a tilt brush type of experience, it was great. They were showing demos on a video of playing Beat Saber. And I think that would be a great test to be able to actually play Beat Saber and see how well you could play Beat Saber on the mobile VR headset and how much latency it has. because Beat Saber you're basically moving the controllers around pretty rapidly and potentially even moving your head around quite a bit so it would actually be a really great stress test but they just had kind of a painting application so I wasn't able to see how it was able to actually perform. So it's a really interesting idea though to take something that you have maybe a middle tier PC you're able to then broadcast onto a mobile VR headset and maybe it'll be cheaper for people to have access to maybe they already have some of these Phones that work on something like the gear vr or the daydream and to be able to do something like they were showing the demos on the oculus go or the xiaomi version there in in china where you could actually start to Screencast onto these headsets from a pc and to be able to have access to some of these six degree of freedom games because it is developed for mobile VR. A lot of the mobile VR experiences that are out there don't necessarily have six degree of freedom interactions. And so it kind of makes more sense for you to have access to some of these other experiences that maybe you have more advanced like gameplay from, you know, these steam VR games where you actually have these six degree of freedom. Or you have social VR. I could see how this would be a perfectly great application for social VR. But I think something like this actually could have a lot of applications when it comes to like training or education, especially when it comes to embodied cognition. Like I wouldn't underestimate the impact of being able to put your full body into an experience and the training impact of being able to be immersed within an environment and to be able to make choices and take action and to actually move your body in a way that is going to be more similar to what you're actually trying to train someone to be. It's like whenever you're able to connect all those dots, then it actually creates this really rich training experience. And so I think they're starting with gaming, but I think there could be a lot of really interesting training applications with some of this technology. And like I said, the hardware ecosystem is completely different in China, and there may be some of the location-based entertainment places that are adopting this type of solution in order to do kind of like a more affordable ways of doing six degree of freedom experiences. So it'll be interesting to see whether or not something like this Nolo VR types of headsets are able to find real use cases when it comes to the long term, especially as more and more of the hardware companies within the next year or two are going to start to implement their own versions of these six degree of freedom controllers. But they were able to have their successful Kickstarter and be able to send out a lot of units for $89 to $99. So it's a cheap and affordable solution to be able to put on six degree of freedom onto your mobile phone and your mobile VR experiences. The biggest question is, what is the content that you're actually going to be able to use? And are there hooks into some of these mobile VR experiences that are able to then allow you to take full advantage of having your full embodiment within these mobile VR experiences? So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member to the Patreon. This is a listener-supported podcast, and so I do rely upon your donations and your membership in order to continue to bring you this coverage. Just $5 a month makes a huge difference. And if you want to donate today, you can do that at patreon.com slash Voices of VR. Thanks for listening.
