Qualcomm believes that the future of XR is mobile. TechCrunch published rumors published that the Oculus Rift 2 headset was being cancelled, which catalyzed a larger discussion about the degree to which Facebook/Oculus is committed to VR on the personal computer. Facebook denied that the second iteration of the Rift was being cancelled saying, “we do have future plans, and can confirm that we are planning for a future version of Rift.” They’re also not commenting on the timing of their internal product roadmap, and so it seems likely that if there was an internal delay of the timing of the second Rift, then they’re not commenting on those private plans. But it seems reasonable that Facebook is putting all of their focus and energy on the launch and promotion of the Oculus Quest, which hopes to be an iPhone turning point moment that shows that standalone VR is good enough and affordable for mass consumption that will then catalyze a critical mass of adoption for VR technologies to take the VR ecosystem to the next level.
Qualcomm is committed to this vision of mobile and tetherless VR and AR since their Snapdragon™ 820 powers the Oculus Go, and their Snapdragon™ 835 powers the Oculus Quest, Lenovo Mirage, and Vive Focus. Qualcomm also announced a XR1 dedicated chip during AWE that will be used in the next generation of self-contained XR devices that will likely start being announced in 2019. At GDC, I had a chance to catch up with Qualcomm’s Head of XR Hugo Swart as well with Hiren Bhinde, XR Product Management and Strategic Partnerships. We talked about the Qualcomm latest Snapdragon™ 845 system on chip and development kit features that were announced at GDC and the Mobile World Congress, some of the future display technologies they’re optimizing for, as well as Qualcomm’s four-pillared strategy for supporting the XR ecosystem.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
Here’s the announcement video for the Snapdragon XR1, which is the world’s first dedicated XR platform
Here’s Hugo Swart AWE presentation: The Making of an Ambient World: The Future of XR is Now
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So earlier this week, there was a report from TechCrunch's Lucas Matney saying that Brendan Arabay was leaving Oculus and Oculus Rift 2 was canceled. Now, Oculus and Facebook deny this report saying that there was never any specific roadmap for the Oculus Rift 2, that they're still completely committed to PC VR, and that my sense is from what Lucas is saying is that there was a potential delay of the second release of the headset. and that they're really focusing and doubling down on Oculus Quest as the future of VR being mobile. Now, this is a report that caused a lot of rumors, a lot of controversy within the VR community. Is there a delay? There's been no announcement as to when Oculus Rift 2 was even going to come out. So it's kind of an internal decision. Even if they did delay it, then is that really a new story? So my sense is that there likely is a bit of a delay just because as Mark Zuckerberg said at Oculus Connect 4, that their goal is to get a billion people in VR and that at Oculus Connect 5, he said there, about 1% of the way there, and they have a long way to go. And so if that's true, then PC VR is maybe a part of the ecosystem, but it's not going to be the catalyst to be able to really cross the chasm into the mainstream. So at GDC this year, I actually did a number of different interviews, one with Qualcomm and one with Intel. Now Qualcomm says that the future of XR is mobile and that XR is the next major computing platform. And that they're really kind of going all in and supporting this ecosystem. And in a lot of ways, what they're building with this system on the chip, Snapdragon 821, 835, 845. And after this interview, they announced an XR1 platform, which is going to be a dedicated XR chip to be able to focus on different things that are very unique to XR, whether it's doing 4K video, 60 frames per second, 3D overlays, popular graphics APIs, OpenGL, OpenCL, Vulkan. These are some of the things that may not necessarily be within the phones. So a lot of Qualcomm's chips end up going into the latest phones that are releasing like this year in 2018. So we have something like Samsung Galaxy S9, S9+, the Galaxy Note 9, and then the Pixel 3 all have the Qualcomm 845, which had a developer kit that was announced back at GDC. It was originally announced at the Mobile World Congress. And so you start to see like these mobile chipsets get into the phones right away, but then there's a bit of a delay. There's like a year delay on some of the standalone VR devices. So for example, the Oculus Go uses the Qualcomm 821, which is like two generations back from whatever the latest is. And that the Oculus Quest is going to be using the Qualcomm 835 with some special fans, but also the Lenovo Mirage has a Qualcomm 835 as well as the Vive Focus is also using the 835. And that the 845 that we talk about here in this interview, we haven't really even seen any devices with that yet that are like the standalone, either XR or AR devices. But that after this interview, just about a month and a half later at the Augmented World Expo, they announced the XR1. So we're going to dive into this interview where they talk about the 845, which is starting to be used in some of the standalone devices probably coming in 2019. But also as CES is coming up, it'll be interesting to see what new devices are going to be out there that are using this chipset that they're talking about here in this interview. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Hugo Swart and Hiram Binde happened on Tuesday, March 30th, 2018 at the Game Developers Conference in San Francisco, California. So with that, let's go ahead and dive right in.
[00:03:51.555] Hugo Swart: Okay. Hi everyone. I'm Hugo Swart. I run our XR business at Qualcomm. At Qualcomm, we are referring to VR and AR as XR. So we have a dedicated team, business, engineering, totally focused on our strategy to enable mobile XR to the masses.
[00:04:12.265] Hiren Bhinde: Hello, everyone. My name is Hiren Binde, and I lead the technology product management at Qualcomm for XR. And I help with the product planning for XR for the next chipsets and coming up with the technology and software features and the roadmap, as well as enabling technology partnerships for next generation dev kits.
[00:04:31.390] Kent Bye: Great. So why don't you first talk a bit about the announcements that you're making here at GDC 2018? Sure.
[00:04:38.193] Hugo Swart: Actually, it's a two-phase announcement that we're making. The first phase was actually last month, where at MWC we announced our reference design, our standalone HMD reference design, based on the Snapdragon 845. So, the Snapdragon 845 and its reference design, we are showing new functionalities like 6DOF with SLAM, eye tracking, foveated rendering with our branded Adreno foveation. The chip enables twice the display bandwidth, supporting up to 2K x 2K x 2. So it brings a lot of new goodies for our VR and AR customers. And at MWC we announced now we have the reference design, working with our ODM partner Vortac, and we made it available to OEMs. And then from the MWC announcement, that brings us to today at GDC where we use that reference design now as a dev kit. And I'm going to pass it back to Hiren to talk about the dev kit.
[00:05:47.783] Hiren Bhinde: So Hugo just shared about the Snapdragon 845 chipset and the reference design that was built on that chipset. Now we've taken this reference design and built a SDK on top of it and we're packaging this as a complete VRDK as a virtual reality development kit which includes the hardware and the software for developers. So what we're doing here is there are three key offerings. So first is the hardware itself, where as a part of the hardware, we are enabling SLAM, the room scale 6DOF. We are enabling eye tracking, and the hardware has these cameras that help with these features. So there are two external facing cameras that help with SLAM. There are two internal facing cameras and IR LEDs that help with the eye tracking. The second aspect is the SDK itself. The SDK has APIs which enables end developers to not only develop applications for eye tracking and room scale sixth off, but also make sure that they use power and thermal management APIs so that the device doesn't get heated up too much. So, we have this SDK rich set of APIs which could help with any of these application development tools from like profilers to developing cool applications for eye tracking as well. The third aspect is the partnerships. The partnership, the first collaboration that we're announcing is with Tobii for eye tracking. So Tobii and Qualcomm collaborated together in agreeing on a set of hardware that's required for eye tracking. And we've installed that hardware, integrated that as a part of the dev kits that are being shipped. Along with that, once an end developer gets the dev kit, he or she would be able to download Tobii's SDK and use Tobii's iCore algorithms to develop eye tracking use cases. The other thing is the dev kits, when it will be shipped for the later part of this year, it would ship with HTC Vive's Wave SDK. And that's something that we are really excited for as well. So how does the Wave SDK help us is something that, again, Hugo can share more about this.
[00:07:54.184] Hugo Swart: So actually, let me just take a step back and share with you and the audience our overall strategy for XR. So actually, we have what we call four-pillar strategy. The first pillar is what we are known for, right, our chip, the silicon. You know, whenever we do a new chip, premium chip, right now we looked at what do we need for VR and AR? What kind of acceleration do we need to put in hardware for VR and AR? What are the architecture, SoC architecture that we need to do for VR and AR? And we started it, I would say, about four to five years ago. When we showed the Snapdragon 820, that was the first chip that we considered meeting the minimum criteria for immersion. So 820 was the first chip, then came 835. 835, we introduced new features like the 6DoF with an older algorithm, VIO. So we, of course, it was an incremental step in performance versus the 820. And then comes 845. which we announced end of last year as a chip, and then now the reference design and the dev kit that we are talking today. So that's the first pillar. The second pillar is our software. So what we believe is that to make available all this hardware functionality that we put on the Snapdragon chip, we need to create APIs that Hirman was alluding to. So that's the second pillar, building these algorithms, some of them in-house, like 6DOF, right? So we have years of R&D in computer vision, both on the VIO and SLAM variants, that we spend a lot of energy to make them suitable for a VR and AR experience. So that's the second pillar. The third pillar is what we call the HMD accelerator program. And in the accelerator program, what it does is take the first and the second pillar, put them together, the chip and the software, and put them together in a reference design. But it's not just a reference design. We have qualified components. that we select to work perfectly with what we are delivering with our software. So if you are doing 6DOF, you know, with cameras and you need, you know, a dual camera and you need a certain field of view for the lens in the camera. So all that, you know, is something that we qualify and we, you know, we work with the selected partners to make included in the reference design. And once we build the reference design, We define, again, a few ODM partners so that we're lowering the bar for our OEM customers to quickly come to market with the Pillar 1 and the Pillar 2, with our hardware and our software. And then the fourth pillar is working with the ecosystem, working with the platform partners. So one interesting thing that you may look at from our announcement at MWC, we have the endorsement from Google, from Oculus, and from Vive. So basically, what we are trying to do is with the first three pillars, really enable that to the platform providers so that we have a full you know, end-to-end ecosystem enabling, you know, mobile VR to come to the masses, right? That's, I think, everyone's objective, right? It's Qualcomm's objective, it's the platform guy's objective, the OEMs and the content ecosystem in general. So just thought it was useful to kind of provide this big picture on the four pillars that we're working on. And I think so far we are seeing a good amount of success with the product launches and so forth.
[00:12:00.432] Kent Bye: Yeah, so the SoC, the system-on-chip, is in some ways replicating both the CPU and GPU analog within a PC and able to add all sorts of other things as well into creating this custom silicon system to be able to do these immersive experiences. I know that there's a lot of Daydream phones and Gear VR, as well as the phones that are driving that, but also some of these things like Vive Focus, as well as the Oculus Go and Oculus Santa Cruz. And so I'm just curious, like where all of this, either Qualcomm 835 or 845, those chips, like where are they showing up and what are the major platforms where this is really driving?
[00:12:40.260] Hiren Bhinde: So, from the smartphone side, we introduced premium VR experience on smartphones beginning with Snapdragon 820 and then with Snapdragon 835. And we worked closely with Google on some of the Daydream smartphones. That's where you see the Snapdragon 820 and 835 because Snapdragon 820 has the minimum amount of performance that's required for premium VR experience. One of the key things that Qualcomm has worked closely with an ecosystem partner like Google in enabling dating-based smartphones is a key concept called as motion to photon latency, which when you're looking in the virtual world, let's say you're looking at a building and you look on the right and there's a dog. In the real world, if you're looking at a building and you look on the right and you look at the dog, it's right there. And you expect the same kind of experience when you see that in the virtual world. Now, that needs to happen in less than 20 milliseconds. Your eyes need to, as soon as your head moves, the device needs to show exactly what it needs to show in less than 20 milliseconds. Otherwise, the end user is going to feel extremely dizzy. Qualcomm has worked on such technologies very closely with Google in ensuring that the motion to photon latency is much less than 20 milliseconds. Now, for that, there's a certain kind of graphics bandwidth and graphics processing capabilities and memory bandwidth that's required on the SoC itself. And that's what 820 provides as a minimum bar. Now, that's on smartphones. Once you bring the VR experience to a standalone device, the capabilities or the graphics capabilities required can be higher based on the content that you're running on that. That's where, like, Oculus Go is based on Snapdragon 821, whereas Y-Focus and the Lenovo Mirage by Daydream are based on Snapdragon 835. So, from Qualcomm's perspective, to meet the premium VR experience bar, there's certain kind of graphics capabilities and DSP compute capabilities as well as memory bandwidth that's required to have that experience ported on a mobile SoC for a truly standalone device, which not only has the right compute power, but more importantly, running that without the device getting heated up. Because sometimes it's easy to have a lot of applications run on a marginally okay compute device, but then the device could get heated up in 15 minutes and then the device might shut down and you might require fans. and it could go all over the place. So that's something that we've looked at extremely seriously from a technology standpoint. As far as the business and the overall ecosystem acceptance of these chipsets in the VR space, I'll let Hugo add more.
[00:15:19.863] Hugo Swart: Yeah, and just complementing a little bit about what Hiren said about, you know, the value of a Snapdragon SoC when compared to maybe PC architectures, right? You referenced, yeah, in our SoC we have the CPU, we have the GPU, but we also have ISPs to handle, you know, all this camera frames that are coming from in the latest reference design for cameras. You have a hardware engine for video decode, right, that does, you know, video decode in low power. You have a DSP where you can accelerate all these computer vision algorithms that are required for virtual reality on the head tracking, hand tracking, eye tracking. We have connectivity, right, so Wi-Fi as an example, Bluetooth as an example, all in the same SOC. So then, I mean, how do you compare that with a PC where, you know, you have all these typically different chips? So when you want to do a standalone product, well, you don't have that freedom of space and power that you can dissipate, right? You cannot dissipate 20 watts. You cannot even be at 100 watts or something on a desktop. You need to do it at very low power. So that's the differentiation. Can you do good PC VR? Of course you can. but at a certain price, at a certain power envelope, and that may not have the reach where the mobile ecosystem has to take it to the masses. And so by having everything on SOC, you can make it more affordable. and easier to use, if you will, because it's smaller, right, it's something portable, and many of the same functionalities that you have on a PC setup. So, you know, having the 6DoF tracking without the need of any accessory or any setup, that's a big differentiation. So that's what, you know, Snapdragon, I think, brings to the VR when compared to PC. As far as commercial traction, we have, you know, more than 20 devices that were already shipped with Snapdragon across standalone VR, standalone AR, smartphone VR, smartphone AR. So there are many devices already launched and at least another 20 on the pipeline. So we're very happy with the traction. We know that, you know, it's going to take a few years until, you know, we can substitute our smartphones, you know, for a, you know, a lightweight glass that does, you know, all my compute need. But we are on the right track. I think our four-pillar strategy certainly envision getting there, you know, with a super light, you know, all the use cases that you can imagine on a glass. But we understand, you know, it's year over year, you make these incremental improvements that look at it at five years or a decade. I mean, wow, what a change, right, you can do. Even if you're just looking to the first, compare the 845 versus the 820, it's already a significant difference in terms of performance delta that we can provide. You know, put another three years on top of this, we're going to make something really groundbreaking.
[00:18:44.858] Kent Bye: Yeah, and when I was at CES last year, I had a chance to try the ODG glasses, which I think at that point was the first device that had the 835 Snapdragon from Qualcomm. And so has the 845 been launched in any actual products yet? Or when is the first time we're going to start to have some of those products be seen with an 845 processor?
[00:19:04.939] Hugo Swart: Yeah, I mean the next few months I think we'll be able to show early devices. Our strategy is always to start with a reference design because then that helps OEMs and even our own engineering to clear up all the main development steps with our reference design and then going to OEMs who will leverage that and create their own. One interesting thing is that we actually see AR customers, you know, leveraging what we build for VR and using the same kind of sensors and components that we chose in the reference design also on AR glasses. If you look at from a trajectory from reference design to commercial products, well, you know, last year we announced the A35 reference design And dev kit. Last year was actually interesting. They coincided, right? GDC and MWC were on the same week. So we had both the reference design and the dev kit done together. This year, they were a few weeks apart. But, you know, we had that done beginning of the year. You know, at Google I.O., you know, Google announced Daydream on it. Later in November, Vive announced their development conference. They were using our reference design and, you know, shortly after, we see products coming to market. So I think the reference design is the starting point. Give it a few more months and you start seeing the first products with it.
[00:20:37.718] Kent Bye: Yeah, being in the VR industry, one thing that's difficult to see is when you're in the middle of a emerging technology that is growing, maybe exponentially, or maybe not. There's different indicators that the VR industry is looking at in terms of virtual reality headset sales and how fast it's growing. In my mind, I'm sort of all in in virtual reality, and it's more of like, I don't think that either VR or AR are gonna go away. It's a matter of time before we get to that point of crossing the chasm into the mainstream, but I think that as people who are developers trying to figure out what is happening, some people say, oh, VR is dead, and I'm like, that's crazy, that's not going away. When I talk to people at Unity, they say, look, this is a technology adoption curve that has exponential doubling year over year, we see that doubling. You said that you're going from 20 to 40, that is one sense of a doubling, but yet what kind of metrics does Qualcomm see to be so dedicated to this industry and to see that this is a path that is, you know, in some ways taken root in terms of the growth of the future? Like, what are the indicators that you have that are able to say that this is a real thing. Because often when I talk to people, I don't have those numbers. I don't have the sense of being like, yeah, no, but this, this, and this. But you're operating at such a low level of the infrastructure stack of creating these chips that in some ways you've got a very good insight into why virtual reality and augmented reality are going to continue to grow and evolve into larger and larger ecosystems. So I'm just curious to hear, what is it that you look at that gives you confidence to be pushing forward into this?
[00:22:17.308] Hiren Bhinde: So why don't we answer this question in two parts. One, we'll give you an insight into why we believe in this technology and what we're doing to kind of push the technology envelope. And then I'll let Hugo discuss kind of like then from the numbers perspective. So if you think about the history of our company as well, we're an integral part of the mobile revolution. 11, 12 years ago, Qualcomm was the first company that worked with Google and HTC and launched the G1 and the G2 and all the Android phones. At that point, the culmination of a device with the right kind of touch display and the mobile ecosystem like Android at that point had not fallen in place, and that was something Qualcomm totally believed in and was a part of. If you see the history of just how technology has changed, going from mainframes to PCs to mobile phones, we do strongly believe that XR is the next mobile computing platform. Now, are we there today? Of course not. For an end consumer to actually start using this device, there are three things that need to fall into place. There's mobility, there's price, and there's content. So from the content perspective, big companies like Oculus, Google Daydream, and Vive are doing their bit. From the price perspective, OEMs are trying to do all sorts of subsidiaries and other efforts to make sure that the end consumer gets excited. From the mobility perspective is where Qualcomm's doing its bit in ensuring that it keeps pushing the technology envelope to bring this to a modular device that could be used anywhere, just not in a home bound by wires, but it could be used by like a mass consumer device. Now from a technology perspective though, just like 13-14 years ago, we were still using those big brick smartphones, but they kept pushing the technology envelope in terms of having bigger displays and making video stream working and cameras at 1.3 megapixel at that time, but it kept pushing the technology envelope. We're keeping the same attitude where we're making sure that we keep pushing the technology envelope. As Hugo mentioned, 820, it was just only you could see the world around you. In two revisions of chipsets, you could actually move in the world around you, and you would be able to do that in a room of 20 by 20 feet that we are announcing with our dev kits, and also have the right kind of security system and 3D audio. So the technology is being pushed forward. What we think is that the developers should see this as an opportunity, and it's excellent that we're actually crossing the chasm. 2016-17, yes, there was a lot of hype. And now we're kind of crossing that, we're slowly getting out of the chasm, and that's where 2018 is going to be a year. Just think about it from a history perspective. As of today when we are talking, none of the standalone VR devices have actually launched. The VR community, the developers, the technology enthusiasts have been talking about this for the last two years. The devices are all going to launch this year. So in my opinion, as a technology enthusiast, as well as somebody who loves VR, it's too early to pull the plug.
[00:25:17.340] Hugo Swart: Yeah. And, you know, you, you mentioned about metrics, you know, well, how do we measure this? Well, you know, we, of course we have the, let's say the advantage of being able to talk to multiple OEMs, to talk with multiple platform companies, to have access to content studios, game developers. And we strategize around that, right? So how much momentum are we saying? And, you know, I can even quote public data, you know, when at Oculus Connect last year, Mike Zuckerberg said, we're going to reach, you know, 1 billion users in VR. Well, you know, that tells something. You know, it's not Qualcomm trying to drive VR. I mean, it's really many heavyweights in the industry going in a similar trajectory. So we keep aligning ourselves, you know, with all the key industry players and making sure that what we put in our chips are the things that are going to be used two years from now, you know, when the chips that we're designing now come to market. So it's a combination of getting the inputs from the various industry sourcers, partners, customers, and our vision, right? Our vision. I mean, if it wasn't for our vision, you know, when we started CDMA, 3G, and not only from an air interface, but also, you know, adding GPUs into our silicon. All of those things were following a similar process that we're doing now. Is the market fully baked? No. But we invest in advance of the market because the market for the market to materialize, you need the chips. So we are always like one step ahead. We're one step ahead. So in going back to your earlier question, when you can start seeing devices with 845, well, We're one step ahead. We are putting the reference design, and next step, you're going to start seeing the OEMs. So I think I echo what Irwin said in terms of the excitement. The excitement with what's happening, we know, with our customers and in overall industry. We're true believers on XR as the next generation mobile compute platform. And then, you know, we adjust our investment, our development based on what customers tell us, but, you know, following our vision on where we see the mobile compute going forward.
[00:27:56.764] Kent Bye: So I'd imagine that when you are working with the chips and the technology, there's a little bit of a chicken and egg problem where there's going to be future displays, digital light fields, holographic displays, very focal displays, where there's technology that is possible in the lab, but yet in order to miniaturize it and actually put it into a device, there needs to be either special architectures on that system of the chip to be able to deal with those digital light fields or those new methods of displays, maybe new paradigms. I'm just curious as we're looking forward in the next two to five years, what are those displays that you see that are exciting enough to be able to start to have to build the architectures at this chip level that we're going to be ready for looking to the future?
[00:28:41.221] Hugo Swart: So, and that's part of the big magic that we do at Qualcomm of not only listening to customers, platform partners, but the whole supply chain, right? So we do have technology experts, display technology experts, you know, at Qualcomm, camera technology experts. that are talking to each of these companies, you know, in their field, understanding where it is, and then mapping to what the chip will require, you know, two years from now. And you're right that, I mean, one of the key challenges that we have for lighter, smaller, all-day wearable is the display. But we see in VR already improvements. In AR, there's a combination of technologies that are being discussed about micro-OLEDs, DLPs with waveguides, and LCOs. And we are looking to all those, which technology will eventually win, and how small, and how soon Yeah, we have our own predictions, but probably a little hard to put it on on public, but for sure we're planning for that.
[00:30:00.407] Hiren Bhinde: I think in addition to what Hugo said, some key factors that we definitely look at while shortlisting some of these technology providers. What you're saying is right, there's a lot of interesting demos out there today in the labs, or like people have stuff working in the labs, but is this something that could be manufactured for tens of thousands and hundreds of thousands and eventually millions of devices? because we know that this is going to be a mass-consumer device. That's number one. Second, is this going to be how price-sensitive this market is? For an enterprise-based AR glass or for a consumer-based AR glass, the solutions are going to be different. Could an end consumer live with a product which has like $300 a lens or do you need something that needs to be really low? And of course then there's the technology aspect where the most important criteria, like specifically for displays is not only the brightness and like the nits per value and all those factors, but one of the key things that Qualcomm always looks at, and it's extremely crucial in an XR device, is the power consumption of those particular lenses, specifically in this case. I keep going back to the smartphone analogy, but in the early smartphones that had come up, there was always this concern of the phones getting heated up in the pocket, irrespective of the brand and the manufacturer. And that's something that folks had expressed concern for a device that would stay in your pocket. Now the devices are going to be right in front of your eyes. So we take that extremely seriously. And we want to make sure that any technology vendor that we end up working with, alluding to the HAP program that Hugo just mentioned, that's one of the key criteria that we see. So manufacturing abilities, cost, technology, excellence, and of course, the power consumption. So if there's somebody who's listening to your podcast and wants to come and talk to Qualcomm to be a future vendor, I'd say, hey, make sure that your power consumption is low.
[00:31:50.504] Kent Bye: Great. And finally, what do you each think is the ultimate potential of virtual and augmented reality and what it might be able to enable?
[00:32:00.769] Hugo Swart: Well, I think we all have our own dreams on where this is going to be in a decade or even a few years more. But, you know, for sure, we also, we see our vision is this glass, right, that you wear the whole day. It's always connected, which is something we haven't really touched during this conversation, but always connected will be key. and how 5G will be a key enabler to get there. But the vision is it's always connected. I can be anywhere in the world and have the telepresence or teleportation kind of experience anywhere I'm at, where I'm talking to someone. I have my digital avatar, my photorealistic digital avatar talking to another photorealistic digital avatar. And being able to do that while having technology transparent to the user, right? So having, again, something light, everything encompassing, always connected, and then having these experiences where The users don't know how many cameras I have, how many microphones are there, how am I tracking my hand, my eye. They even forget all that and just have a seamless digital experience that you sort of get to be confused about reality and virtuality. Yeah.
[00:33:34.950] Hiren Bhinde: It's extremely exciting. I think the possibilities are endless and I think Hugo alluded to the always connected aspect. I think the devices are also going to become more intelligent with the convergence of always being connected, the machine learning algorithms for all the partner algorithms that run on the system. And then of course the use cases where you see AR and VR kind of converging into these lightweight glasses where it would be extremely difficult at one point to see what's photorealistic and what's actually realistic. So those kind of use cases are going to be there. But for now, if I am able to wear glasses along with my kids and if I'm able to show them a solar system right in the middle of a room and walk with them to every planet just within the room and show them with extreme clarity the rings around Saturn or the moon revolving around the Earth, I'd be very happy to begin with that kind of experience and then take this one step at a time.
[00:34:39.555] Kent Bye: Great, and is there anything else that's left unsaid that you guys would like to say?
[00:34:44.103] Hugo Swart: Well, maybe I couldn't end this without saying that the future of XR is mobile, right? So I think we are strong believers on that. I mean, I don't think it will come from the PC world. It will be pretty much a continuity from the smartphone mobile trend, be it because of the 5G connectivity, be it because of the pace of innovation that the mobile industry enabled. with billions of users having access to smartphones, that's what we will enable next on this mobile history.
[00:35:23.858] Hiren Bhinde: Yeah, I think Hugo kind of nailed it. The future of XR is mobile, and XR is the next computing platform. And for people and developers who are still not believing in this completely, I'd say come see the different devices. And if you look at the history of technology, be it televisions, be it smartphones, be it the mobile apps ecosystem, everything has gone through the hype cycle and the trough, and eventually it has all crossed the chasm. So XR is definitely going to cross the chasm.
[00:35:55.494] Kent Bye: Awesome. Well, Hugo and Hiran, thank you so much for joining me today on the podcast.
[00:36:00.055] Hiren Bhinde: Thank you very much. Nice being here.
[00:36:01.615] Hugo Swart: Thank you.
[00:36:02.655] Kent Bye: So that was Hugo Swart. He's the head of XR at Qualcomm and Hiran Binde, the XR product management and strategic partnerships. So I have a number of different takeaways about this interview is that first of all, well, that statement that the future of XR is mobile is something that really stuck with me because Yes, while I think that there is going to be a part of the XR ecosystem that is going to be totally mobile, I do think that PC VR as well as location-based entertainment are never going to go away. So I know that John Carmack talked about the potential for the Quest to be able to do streaming. So maybe it's also going to be possible to do these wireless headsets and be able to have these higher-end experiences that have the content that's being rendered on a PC and streamed into the Quest. So that is also something that may be there as well. If we look at what is happening in the video game industry, you basically have the PC gaming, console gaming, and mobile gaming. And some arcades, but the arcades have really kind of fallen by the wayside in terms of like the types of gaming experiences you could get on PC and console is pretty much equivalent to what you may be able to get within an arcade. Aside from some of the haptic and more laser tag social collaborative things like that's still there But I think actually in virtual reality We're gonna see a lot more of location based entertainment arcades that does really push the limits of fidelity of what type of experiences that you can have but also using space and embodiment and haptics and being able to do things that you could never do in your home just because it would not be feasible to be able to have that expensive equipment within your home and So I think there's going to be a similar type of breakdown within VR where we're going to still have this mobile VR, console VR, PC VR, as well as location-based entertainment. Now we're starting to see a little bit of this fusion of the console, like standalones, like the Quest and the Go with something that is like the mobile. Those are kind of verging together and it'll be interesting to see what ends up happening with like something Like, the PlayStation VR is maybe still going to hold down the console, which is somewhere in between the power of a PC and much more powerful than a mobile VR. Although, over time, with some of these chipsets, you're maybe going to start to see more of a parity and equivalence around that. So it's a big open question I think as to how this is going to play out and whether or not some of these Oculus Quest experiences are going to be at the same level of immersion and embodiment or if there's going to be limitations of, for example, like because there's front-facing cameras within the Oculus Quest, I'm really curious to see like how is this going to handle a game like Beat Saber where you have a lot of your body moving and you can understand your proprioception of where your hands are without having like an external sensor, are those cameras that are in the front of your face going to be able to always track at the same level of fidelity as something like Beat Saber on an Oculus Rift or the HTC Vive? That is the level of embodiment that you're just completely not thinking about the limitations of embodiment. But once you start to have a front facing camera, then you're going to have to always have your hands within the eyesight of your head. And so you're not going to be able to like turn your head around and still be able to do things with your hands when it's completely out of the field of view. So that's one of the open questions, I think, like how often is that? How good is that tracking going to be? I think that's going to be a big differentiating factor as to, you know, how much some of this mobile Oculus Quest gaming is going to really take off and be able to be at parity with people who have been used to having the types of experiences on the PC VR. that said i think that pcvr is not going to go away it's still going to be there and while it's probably likely that there is a de-emphasis at oculus on some of these pcvr i mean do we really need a second iteration of oculus rift i mean i think that the first version of rift is probably going to hold us over until we have this big quantum leap of the technological improvements that are going to come from both the display technology, the very focal display, the eye tracking technologies, you know, who knows if there's going to be strain sensors of being able to detect different dimensions of your face. So I think that we could hold off for another year or two. And that, um, even if there was plans to announce the Oculus Rift 2 in 2019, I think with the quest getting out there, there's going to be enough quality and fidelity in the first Oculus Rift that it's functionally not going to make that huge of a difference in that. even if there was an internal shift that the VR ecosystem is still growing out to different markets that are going to really sustain it, whether it's enterprise or education or training or architecture, engineering, and design. So this chipset of the Qualcomm 845, I think we're going to start to see some of those devices at CES and into 2019, like what people have been working on. And so they also announced XR1, which came after I did this interview at AWE. That's going to be the express purpose of delivering high quality VR and AR experiences. So I'm going to be really curious to see what type of devices are coming out within the next year to see what's going to be in the XR1. It's rumored that maybe HoloLens is going to be using XR1 and be releasing at CES 2019. I've also heard rumors that it's not going to be until 2019. So we may start to see something at CES this year from Microsoft, but I think it's going to be a bit of a wait and see. These types of announcements with the Qualcomm, they're really, you know, got this year lag before we start to really see some of these devices actually launch. Like I said, the 835 is going to be powering the Lenovo Mirage, the Oculus Quest, as well as the Vive Focus. And this 845 is not going to be into this next year to see like, okay, what new devices are going to be coming out with that. And that Qualcomm really has a four-pillar strategy for XR. It's the silicon and the system on the chip, and then fleshing out the APIs and algorithms, things like 6DoF SLAM and eye tracking. And then they have this HMD accelerated program where they create this reference design, where they define what's possible for the OAM partners. I had a chance to actually try that out at GDC where there was eye tracking and 6DOF and you know it was pretty good. I wouldn't say it was as good as Oculus Quest in terms of just the fidelity of the 6DOF but you know it was a reference design and I think that as things get out it's going to be much improved once the custom integrations happen. And then finally they're just working with the ecosystem and platform partners to be able to support these different devices that are going to be coming out. And that, you know, Hiron said that for consumers, they really need three things. They need mobility, price, and content. And it does seem like this year that the launch of the Oculus Quest is going to be a bit of a turning point. And I think it's going to be easier for you to be able to just hop into VR. And talking to a number of different developers, like NextVR, I talked to Dave Cole, and he said that the difference between the Gear VR and Oculus Go is absolutely huge for being able to just quickly jump into VR and to not have to do the extra steps and that they're seeing much higher engagement with the experiences like the next VR in Oculus Go and that he expects that with Oculus Quest we're going to continue to see like this huge jump of being able to just throw on the VR headset and be anywhere that you are and to be able to start to have a immersive VR experience and that having a dedicated VR room there's a bit of friction that I think sometimes happens especially if you want to be able to do everything from watching a movie to sit down in the most comfortable places and also like in your living room you may be exercising so I'm really excited to see what happens with the Oculus Quest and it does seem like that Qualcomm is at the heart of being a partner of a lot of these key technologies everything from the HoloLens to the mobile phones to a lot of these standalone devices like the Oculus Quest, the Vive Focus, as well as the Lenovo Mirage. So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon your donations in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voices of VR. Thanks for listening
