The Khronos Group announced on Tuesday that they have a critical mass of major VR players who are collaborating on a VR open standard. This VR open standard will have a software and hardware component that will enable VR application portability across VR platforms, but also minimize the cost for hardware integrations across different VR platforms. The Khronos Group has been able to get public support for this initiative from VR headset manufacturers including Valve, Oculus, Google, & OSVR as well as the major GPU and CPU players of AMD, NVIDIA, Intel, and ARM.
LISTEN TO THE VOICES OF VR PODCAST
Back in March 2015, Neil told me that the VR industry needed a period of innovation before trying to pin down an open standard. But now that the major VR headsets have now launched, there’s not enough significant differences between the major VR SDKs to warrant third-party hardware peripherals to have to create custom integrations. The benefits for standardization outweigh the costs of having a fractured ecosystem, because this VR open standard will enable smaller VR companies to write a single driver that allows them to interface with all of the major VR headsets.
I had a chance to catch up with Khronos Group President Neil Trevett to talk about this call for participation on this VR open standard. There’s not a preliminary specification that’s been developed yet, but they wanted to go ahead and make this announcement in order to let the industry know that there’s enough consensus for this to happen, and to encourage participation from other upstart VR companies.
Neil estimated that this standardization process usually takes around 18 months to finalize, but it may move quicker depending upon how motivated the major companies are to lock it down and remove barriers to innovation. Designing an open standard is more of an art than a science, and Neil said that they would try to limit the scope of the standard, but also allow for extensions. You can read more information about this initiative here with a number of quotes including Oculus, Valve, Google, OSVR, and Epic Games.
Donate to the Voices of VR Podcast Patreon
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to The Voices of VR Podcast. So back in GDC of 2015, I had a chance to sit down with Neil Trevitt of the Kronos Group. And at that point, it was way too early to be able to start to create an open standard within the VR community. The HTC Vive had just been announced. Oculus still had their DK2 and the innovator edition of the Gear VR, but Daydream hadn't even been announced yet. And it was just too early to be able to start to look at the different SDKs between all of the desktop and mobile VR systems. But that has changed now. On December 6, the Kronos Group announced that there is a consortium of major players within the VR industry that are working towards defining a virtual reality open standard. So some of the big players that are on board so far are Oculus and Valve, Google, OSVR, which includes Razer and Sensex, as well as the game developer Epic Games, some hardware manufacturers with AMD, NVIDIA, Intel, ARM, and Verisilicon, as well as the eye-tracking company of Tobii and the 3D graphics driver developers of Lunar-G. So there's already a critical mass of companies that are coming together and starting the process of trying to define a virtual reality open standard to be able to interface VR applications with the required technology. So I had a chance to talk to Neil Chevet of the Kronos Group to talk about this announcement, which is a call for participation for an open standard for VR and what it means for leveling the playing field for smaller players within the VR industry. So, that's what we'll be covering on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. Today's episode is brought to you by the Voices of VR Patreon campaign. The Voices of VR podcast started as a passion project, but now it's my livelihood. And so, if you're enjoying the content on the Voices of VR podcast, then consider it a service to you and the wider community and send me a tip. Just a couple of dollars a month makes a huge difference, especially if everybody contributes. So, donate today at patreon.com slash Voices of VR. So, this interview was recorded on Monday, December 5th, while Neil was at the SIGGRAPH Asia. So, with that, let's go ahead and dive right in.
[00:02:35.640] Neil Trevett: I'm Neil Trevors. I am president of the Kronos Group, and the Kronos Group today is announcing a virtual reality initiative to help some of the fragmentation issues around the virtual reality industry. So, the announcement we have today It's not the announcement of a completed specification. It's something even more exciting. It's the announcement of a call for participation in an initiative to create an open standard for virtual reality. It's a chance for the industry to get involved and have a voice and a vote in how this important standard evolves. But it's significant because we already have a pretty interesting consensus and quorum of the key virtual reality players supporting this initiative, including Oculus and Valve and Google and others. So we really feel that this initiative already has the momentum to make a difference.
[00:03:43.342] Kent Bye: Can you elaborate a little bit as to what specifically you mean by the fragmentation of the VR ecosystem? Maybe talk a bit about the problem that you see that's happening right now in the VR industry and what this open standard helps to try to solve and achieve.
[00:04:00.512] Neil Trevett: Yes. So at the moment we have fragmentation from two different directions, the hardware direction and the software direction. So from the software direction, It's hard to write an application right now that is portable across all of the different VR platforms. So you often see applications being launched today that are either for the HTC Vive or for the Oculus, or even more typically, it runs on PC, but not on mobile. And that's because the virtual reality SDKs for each of these different platforms, VR platforms, is different. And so it takes time and effort to port applications across these different VR environments. So it's possible to do, but it's definitely a friction in the industry that is putting a burden on the application developers. And it means that the consumers are being robbed of the best selection of virtual reality applications because the application they want might not run on the VR device that they have. So that's one aspect. The other aspect is perhaps a little more subtle and will become more important as the industry evolves. is that it's becoming harder as the number of VR environments begins to proliferate. It becomes harder to integrate new hardware into all of the VR environments. So if you have a new eye tracker or something that you want to integrate into a VR environment, the hardware vendor has the problem that they have to integrate into multiple VR SDKs. That is also a cost and a real piece of friction in the industry that prevents innovation from the hardware and sensor space and the rendering space too for the GPU guys into virtual reality environments to be delivered to consumers. So what we're trying to do with this standard is have a standard in the middle that in traditional Kronos fashion, we're connecting software to hardware, have a standard in the middle. Applications can be coded portably, and the different VR SDKs will support these standard APIs. So an application can be written once and it can run anywhere. But also solving the fragmentation problem from the hardware point of view. So right now, the hardware vendors, if they have a new piece of hardware, perhaps a new sensor or a new display, they have to integrate it into these multiple VR SDKs. So that's introducing cost and inefficiencies for the hardware community too. So by having a single standard, we can solve the problem for the hardware guys. So we're defining a driver-level API. So if a hardware vendor has a new sensor or a new display or some other piece of VR hardware, they will be able to write a driver that will integrate into the open standard. And so all of the different VR environments will be able to take advantage of that new hardware which is going to lead to a lot more innovation in the hardware space as well as portability in the software space.
[00:07:10.090] Kent Bye: So I know that in looking at comparing what Valve is doing with SteamVR and their SDK versus what Oculus has been doing with their SDK, we kind of have two different philosophies with SteamVR's already kind of taken an open approach where if you design something for using SteamVR and you launch it on Steam, you can play it on the Vive, but you can also just plug in the hardware from Oculus and be able to actually play that same experience using the Oculus Rift. Now, the same is not true for the Oculus SDK, for what they've launched on the Oculus Home, that at this point, there's no way to take a game off Oculus Home and be able to play it with an HTC Vive. And so, I'm just trying to get a sense of, like, this seems like something that Valve has already been doing with their approach with SteamVR SDK, and I'm just curious what you can tell me in terms of whether or not that's changed for Oculus and whether or not they plan on taken a similar approach to be able to open up and make some of their programs, maybe not all of their programs with the, you know, exclusive titles, but at least some of them being a little bit more agnostic to be able to run on either their hardware versus other hardware.
[00:08:21.083] Neil Trevett: Yeah. So, and of course, all these VR companies are still going to have their own business models. And he's just saying that they might strike exclusive deals. But I think that the reason why this is significant, it's the key players in the VR industry are coming together and we have consensus that a lot of the differences between the different VR SDKs right now are not intentional and they're not adding value. And so I think, yes, it is a movement in general towards being able to have content that will run across multiple VR environments. So now you're right. In the PC space particularly, there are different degrees of Portability already built into some of the solutions that are out there, as you say, SteamVR can support multiple hardware. And of course, OSVR has been open source from the get-go to try and support things openly. Oculus has taken a different approach in a couple of places. But no, we have the Prime movers and shakers from those three platforms, and now Google is joining in, too, on the mobile side, to try and figure out together how we can create a common platform that does provide more portability for applications. So we're not preventing portability across VR platforms by accident. So I think the realization is content portability is going to be a good thing for the industry. And I think it is significant that we have these key players coming together, agreeing on that, and working to create a standard that is going to provide in practice, application portability for applications running across multiple devices.
[00:10:00.842] Kent Bye: And if we look at the differences between the HTC Vive's Lighthouse controller versus the Oculus Touch, there's actually like a couple of different buttons, but they're just configured differently such that if you were to try to push the same buttons, it would just be difficult to do a similar kind of button sequence on both the Oculus Touch and the HTC Vive. And so I think there's gonna be nuances and differences in button mappings as well as there's opening up of the HTC controllers so that we're going to see a whole new variety of different peripherals that are out there that have perhaps different interaction models. And so how do you do that translation and mapping between the different controllers and account for all the new controllers that are coming in?
[00:10:45.475] Neil Trevett: Yeah, and you're right, the world is a messy place. The goal isn't necessarily to mean that there's actually no changes as you go from platform to platform, but we want to minimize the cost and the effort. And again, I'll say designers just started, so we don't have to find answers to everything. But of course, as with any API standard, the key is to find the right level of abstraction. So in the end, it shouldn't matter whether you're using a certain sequence of buttons or whether you're using gesture or hand gestures. the API should be encoding the intent of the user up to the application, really regardless of how the hardware was used to understand that intent. That is the right way to get portability for the application without stifling innovation in input devices. Now, is it going to be trivial in all cases to abstract every single nuance? Probably not, no. But I think it will be possible to have a core of commonly understood input hardware and how that translates into user intent to the application. And the other thing perhaps it's worth saying at this point is because obviously there is a lot of innovation still coming in the VR industry. We're not pretending that everything is fixed. We're just starting, in fact, on the innovation curve in terms of displays and input devices. Just like any Khronos API, this standard will be extensible. So we'll be building a core specification that covers everything that is shipping today and makes sense to standardize between the vendors. But the extension mechanism will always be there for hardware vendors or software vendors to use as they wish if they want to innovate ahead of the curve or outside the commonly standardized platform. So I think you'll see in the VR space a lot of user input innovation perhaps being delivered as extensions first and then as it becomes more widely available and common across the platforms it can be brought into core.
[00:12:54.870] Kent Bye: Yeah that's one of the things that I didn't see is any companies that were representing hand tracking. I think right now if you look at the major VR companies you know just the HTC Vive and you know there's Sony that's also out there as well. I didn't see them on the list yet but Just in terms of PC, we have the Vive and Oculus, but with something like hand tracking, is that something that would be an extension to the existing VR standard? Or how do you account for the innovation that is still perhaps happening with some of these different input controls that haven't really have a widely adopted consumer launch yet, but are still kind of emerging as a potential solution that could be out there. And, you know, is the risk of. setting the standard, going to lock out some of those smaller companies that haven't got a widespread adoption yet.
[00:13:47.101] Neil Trevett: Right. Yeah. And that's absolutely the thing that we're trying to avoid. The whole point of an open standard that it enables and encourages innovation doesn't stifle innovation. You know, we've talked before, but to me, that is the sign of whether a standard is going to succeed and whether it's being well-designed that encourages people to innovate and removes barriers to innovation. And this is a great example where we need to get this right. And again, I will say the design of this has just started, you know, so we don't have solutions on the plates right now, but it's just the normal design philosophy that goes into any successful standard is you don't want to have too large a scope because if you try and do too much, it's going to take too long to deliver and you'll be standardizing things that aren't ready to be standardized. They're still in the innovation phase. But I think really the core reason why we have this number of companies and soon we hope we'll have more coming together is there's enough that is common between different platforms that it makes sense to standardize that. So I would say probably when the design discussions progress, we're going to agree what is common and what it does make sense. And the first approximation is the stuff that's shipping today. there's probably enough commonality there that we can add value with some commonly agreed APIs to access that. Stuff that is still a little bit fuzzy, stuff that's not quite shipping broadly yet and people are still experimenting, that is the perfect place where people can use extensions. I think that extension mechanism, it does enable innovation because we're finding that some of the smaller device sensor vendors are actually being locked out right now, not because people want to lock them out, it's because the just amount of engineering work that a smaller company has to do to support all these different devices is getting quite large. Now there's a multiplicity of VR environments out there and a new device coming onto the market, now they have suddenly had to support all these different environments. That can be kind of a barrier to entry. What we're trying to achieve with the standard is that a new device, new innovative device would just write a single driver that can integrate into this connecting standard and then suddenly all of the different VR environments and applications can make use of it. So we're hoping that this is going to enable new players into the market in a very productive and synergistic way for the industry.
[00:16:18.555] Kent Bye: One of the things that I saw that was a little surprising was to see the combination of mobile and desktop PC. So for example, there's the desktop PC of the Oculus as well as the Valve's HTC Vive, but yet the Daydream platform as well as the Gear VR seem to me to be pretty different in terms of both input controls with the Gear VR with the trackpad control, but also the Daydream has a 3DOF controller, there's no sixth off sensing to do positional tracking on both of the mobile right now. So do you foresee the standard being kind of like a progressive enhancement where there's one standard that you are able to then add on the positional tracking and add on six degree of freedom controllers, or do you see it kind of split between the mobile and desktop?
[00:17:08.825] Neil Trevett: So you're right. And this has already been an interesting topic of discussion in the group that, of course, the difference between desktop and mobile platforms, there are more differences there than between the different desktop platforms. So that creates more challenges. And there's going to be, I'm sure, a lot of discussion in the group as to how we handle that and to what extent it makes sense to abstract the differences between those two different types of platform. So we don't know the answer yet. So it could well be that we have profiles or perhaps use default terminology, no feature sets. We don't want to encumber every platform to support everything if it doesn't make sense in that market. So some kinds of input devices simply aren't going to make sense in the mobile space. So we want to make sure that we slice and dice the standard appropriately. So someone shipping a mobile, they can just ship the feature set that makes sense to them. Balancing, of course, as soon as you have feature sets, there's danger of fragmentation. And this is why creating standards is an art form, not a science. But the good news is that at the core, the reason why this is a significant announcement today is we have the right players in the group. I mean, Oculus themselves, of course, working with Samsung, straddling both PC and mobile, so they have experience. We have PC SDK vendors, and Google have expressed their support in figuring out what this means for Daydream. So I think we have the right players in the group, and there are going to be some subtleties here, and there probably are going to be some feature set type things that span different types of devices by necessity but again i think we have the right people in the room to come up with a good solution that may not solve every problem in the universe but it's also enough of a problem to be useful to the industry.
[00:19:08.790] Kent Bye: Yeah, one of the parallels that I see with this standardization effort, it's somewhat similar to what's happening within the web development community with WebVR. They came together for a WebVR workshop in mid-October. And one of the things that I found really interesting with the WebVR specification is that Microsoft actually came in and had some changes that they wanted to make for the HoloLens and to make the WebVR specification more compatible with augmented reality. And so I'm curious if there's a similar parallel here. We're focusing very much on the virtual reality companies right now, but is this something that you think could be expanded at some point to include augmented reality? Or do you feel like the problem sets of AR are significantly different enough that you don't want to slow down any type of standardization process by waiting for what their concerns might be?
[00:22:17.942] Kent Bye: Great. So, you know, you've got some big players that have given some support and make statements. I'm just curious to kind of hear what you see as the next steps in terms of like, if there are other companies that you want to get involved and then, you know, what is the timeline in terms of actually kind of locking down the specification and then from there, what happens?
[00:22:40.846] Neil Trevett: So, yeah, that's a great question. So again, coming back to the core of this announcement, the reason where putting out this announcement so early in the process. I mean, some people might be thinking, wait, they don't have anything yet. Why are they making an announcement? The reason is very clear, is that we don't want to surprise the industry or to present a fair complaint to the industry. The whole point of a meaningful open standard, of course, is it has the support and the participation of the industry and companies in the VR space have the opportunity if they wish to join Kronos to have a voice and a vote in how this design is initiated and the direction that it goes into. So the whole point of what we're doing here today is to make sure that the right people have the chance to be involved. So, of course, we want more people to get involved. I think we do have critical mass. The quorum that we have already means that this will be a significant standard. But of course, people with PR platforms will be warmly welcome to join. And I think that the GPU community is already pretty strongly behind this because they see that this is a chance for us to increase our value with reduced cost and friction in the industry. I think sensor vendors of various shapes and sizes, vision-based sensors, motion-based sensors, eye-tracking type stuff, all the innovation in the sensor thread that we would love to get those companies involved so we can make sure that this standard is giving them the power that they need for clean integration of their innovations into this whole VR initiative. In terms of timeline, again, we haven't decided the scope yet. So it's kind of meaningless to say, how long is it going to take? Because we don't know what it is we're going to do. But I will say that there is urgency in the group. The group wants to move quickly. because the fragmentation that we have, we talked about earlier, it's not helping anyone right now. So the sooner we can solve it, the better. But of course, this is going to take time to do the design work to reach the consensus that feeds into that design work, you know, inevitably takes time. Just from previous experience, an 18 month ish timeframe is kind of typical for this kind of thing in our previous experience. But as soon as we have given the chance for people to come in and influence the design and have a better handle on the scope, which will take not that long, a few months, and we'll be in a much better position to give some guidance to the industry as to how long we think it's going to take. But it's going to be of the order of 18 months-ish, I would imagine.
[00:25:20.045] Kent Bye: Awesome. And finally, what do you see as the ultimate potential of virtual reality and what it might be able to enable?
[00:25:27.670] Neil Trevett: Well, I think that's a big question. Virtual reality and augmented reality as its close cousin, I think are set to change the way that humanity interacts with its storytelling and its computing resources. I mean, PR is a way of delivering experiences in a way that's much more compelling and in a way that we haven't been able to before. I think the VR industry is maturing where We're not fixated on, well soon, hopefully we won't be fixated and hopefully the standard will help. We won't be fixated on the nuts and bolts on how we do it. And we can shift more of the emphasis onto enabling the innovators in the content and the storytelling and experience creation and letting those content innovators ship easily across the different platforms so that they can make good business and that will in turn fuel more innovation in content. I think the VR industry is making great progress but the time has come for some standards to help push that transition from how do we do this to what do we want to do and I hope this standard will help. I think AI following on from VR will just open up a universe of information to us as we need it, where we need it, And it shares many of the same problems, is that we're going to be initially solving for VR. Now, as we're saying, I think that AR will be able to build on top of this standard and all that VR innovation in a very synergistic way.
[00:27:14.273] Kent Bye: Awesome. Well, thank you so much, Neil. So that was Neil Trevitt of the Kronos Group, and he was talking about the call for participation for an open standard for VR. So I have a number of different takeaways about this interview is that, first of all, when I think about this open standard, I think that, first of all, it's going to mean that as a VR developer, they're going to be able to potentially write things once and be able to distribute across many different platforms. And that's the dream. I think the reality is that there's always going to be a little bit of nuances between the controllers and the input devices, especially if you start to think about the differences between mobile and desktop. I would be actually really surprised if there was an open standard that covers both of the mobile and desktop. But that's something that is going to have to be sorted out. In terms of the other specific questions that I was talking about, Oculus versus Valve, I think that this open standard seems to be for the content creators such that they'll be able to write their application once and potentially be able to have it be compatible with both the Vive as well as the Oculus Rift. Now, in terms of whether or not you'd be able to buy an application through Oculus Home and be able to run it with an HTC Vive, that's something that you cannot do at this point, whereas the opposite is true, that you can run a game that was written for the Vive on Steam and then potentially still use your Oculus Rift headsets and have it work. So that's the type of cross-compatible dream that we're moving towards, but there still may be a decision by Oculus to not enable that or not to allow that. I think that's going to be a decision that's up to each of the individual companies. I don't think that this announcement is necessarily changing that fact. I think that if anything, it's trying to come up with an open standard that is allowing the software applications to be able to talk to hardware peripherals such that they'll be able to write hardware drivers. And this is really important for peripherals. Let's say, you know, the one that comes to mind is Tobii. They do eye tracking. If the Tobii eye tracker wanted to be able to allow their eye tracking to be used both in the Rift and as well in the Vive, then they would potentially just write one driver matched to whatever specification is defined by the Kronos group and then they'd be able to essentially enable eye tracking with either the Oculus Rift or the HTC Vive. I think that's the problem that a lot of the companies are coming together and seeing that needs to be solved and the little subtle nuances between their SDKs. It sounds like those subtle nuances aren't necessarily big enough to really justify the engineering effort for these smaller startup companies to be able to have to build custom code, to be able to get their hardware, to be able to work across all these different VR HMDs that are out there. At this point, the major VR HMDs are the Oculus Rift and HTC Vive. There's nothing that was mentioned about the Sony PlayStation VR. That's something that's kind of a self-contained hardware ecosystem. And perhaps there'll be this dynamic where there'll be the open standard that is used on PC as well as the mobile phones. But maybe there'll be these other self-contained units that aren't necessarily compatible with this open standard, perhaps like the Sony PlayStation VR just because it's a completely vertically integrated technology stack from top to bottom. So Sony PlayStation VR is the first one that's out there right now, but perhaps there'll be other ones, like maybe Apple will come out with a augmented or virtual reality headset that's completely self-contained, or the HoloLens. So to me, I think the first step is going to be just with these major players that are out there just looking at virtual reality, perhaps the augmented reality vendors will start to get into this discussion and maybe they'll be able to come up with some universal APIs between VR and AR and be able to Take technologies like perhaps eye tracking that would be able to be compatible with both AR and VR and kind of be agnostic as to which level on the mixed reality spectrum it was landing and They'd be able to just write a driver that would be able to be compatible with whatever immersive technology that you throw at it So it sounds like Neil put a timeline of about 18 months is about the time that it usually takes for this type of process to happen. There is a lot of high motivation for these big players to get this done because I think they're getting a little bit of roadblocks between some of these newer technologies and peripherals. It sounds like eye tracking is one of the first ones. I know that there's other technologies that are going to be out there that are either haptics or EEGs or biofeedback mechanisms and other peripherals that are going to want to be really tied in. And I think at CES this year, since Valve has already kind of opened up their lighthouse system, I think we're going to see a lot of those new types of peripherals that are going to be coming into the market. that new technology is going to then enable more and more software that's going to be developed. And then I think, you know, the dream is to be able to create these killer applications that people see this integrated solution and they say, oh my God, I have to have this virtual reality or augmented reality system in my life right now. I'm going to go ahead and invest into all of this technology stack because the benefits of this application just have crossed this threshold that are making me want to dive into VR. So I think that's part of the dynamic to be able to have this level of collaboration right now, just for the whole sake of the success of virtual reality, to really succeed as a medium. I think it's a really good sign that there's a lot of the biggest players that are cooperating on this type of initiative. And like Neil said, there's other initiatives that are out there like WebVR, but WebVR isn't talking about the low-level hardware interfaces. That's basically just looking at the browser and how do you get a browser to be able to communicate with some of these VR headset so it's a lot higher level up onto the technology stack and what Neil was saying is that this VR open standard by the Kronos Group is going to be a lot lower down and kind of looking at the hardware level of how do you get all the GPUs and processors to be able to communicate with each other to be able to enable a lot of these VR applications higher up on the tech stack. So again, the different companies that are involved, we have all of the major virtual reality headset creators with Valve and Oculus, Google, OSVR, which is a consortium between Razer and Sensex. And the only biggest one that's missing right here is Sony and perhaps some of the other kind of upstart mobile VR companies. And Epic Games is the only game engine company that's listed here. The biggest missing is Unity, and perhaps they'll jump on board onto this as well. And then in terms of the GPUs and processing companies, there's AMD, NVIDIA, Intel, ARM, and then there's this silicon platform as a service called VeriSilicon. And then again, there's Tobii, they do eye tracking and Lunargy, they do 3D graphics driver development. And so there's a lot of other smaller VR companies that are out there and they probably have the desire to be able to have full integration into these VR tech stacks. So I expect with this call for participation, we're going to see a lot more of these smaller companies start to participate. We're going to start to look at the SDKs that are out there, see how similar the SteamVR is with the DaydreamVR is with the Oculus SDK, as well as the OSVR SDK. So all of these are perhaps going to have some level of standardization. It sounds like there's a dual layer here, both in terms of the software and being able to use the software to talk to all the VR peripherals, but also at the lower level of being able to have the hardware being able to talk to that VR application. So to me, this is a very exciting announcement. I think it's a big announcement and, uh, it's something to look forward to this level of cooperation within larger VR industry, to be able to come up with some standards that are able to encourage some level of innovation within the VR industry. So that's all that I have for today. I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends and become a donor to the podcast. Just a couple of dollars a month makes a huge difference. And so consider this podcast as a service to you and the wider VR community and send me a tip at patreon.com slash Voices of VR.