I’m digging into my unpublished interview archives backlog to publish this conversation with Khronos Group President Neil Trevett marking the 1.0 release of OpenXR that happened during SIGGRAPH in July 2019. We talk about the intention of OpenXR as a specification, and I figured that it’s worth looking back at the evolution of the specification as it has become quite a prominent open standard within the XR industry. Be sure to also see episodes #1382 that I just published from GDC 2018, #507 from GDC 2017, and #1385 with a current update on OpenXR 1.1.
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the future of spatial computing. You can support the podcast at patreon.com. So continuing on my series of looking at OpenXR over the years, last episode was from 2018, where we're just starting to announce some of the initial workings on the specification. And then at SIGGRAPH in July of 2019, they were releasing the 1.0 specification for OpenXR. And I had a chance to sit down with the Khronos Group president, Neil Trevit, at SIGGRAPH that year to unpack some of the major announcements that they were doing and the work that they're able to do. At that point, they had pretty broad buy-in from a lot of the different major headset manufacturers at the time. And OpenXR has continued to go on and be one of the most successful open standards that Chronos Group has ever published, just in terms of the broad adoption curve that they've been able to accomplish. And we'll be covering that a lot more in the following episode, which is giving an update on the 1.1 release, which is releasing today on April 15th, 2024. But I figured I'd go back into my backlog and dig into some of these unreleased conversations about OpenXR just to help trace the evolution of the specification, to hear some of the intentions from the very beginning and how they've been able to really pull that off and make a huge difference in what's possible with XR. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Neil Trevitt happened on Wednesday, July 31st, 2019 at SIGGRAPH. That was happening in Los Angeles, California. So, with that, let's go ahead and dive right in.
[00:01:40.156] Neil Trevett: So, hey guys, I'm Neil Trevett, so by day I work for NVIDIA, by night I run the Kernels Group, and here at SIGGRAPH we've just released OpenXR 1.0.
[00:01:52.360] Kent Bye: That's a big. So what's in 1.0?
[00:01:55.062] Neil Trevett: So it's the result of over 20 companies working for over two and a half years designing a cross-platform portable API to let applications get to a whole range of different XR hardware both virtual reality and augmented reality. And it's an open, royalty-free standard, just like all the standards we do at the Kronos Group. And not just the spec has been announced on Monday. We actually have the first shipping implementations from Microsoft and Calabra. So for developers, now's the time to jump into OpenXR. The spec is stable, there's going to be no more changes, everything is going to be backwards compatible and you can go download the drivers, check it out, actually run real software.
[00:02:46.217] Kent Bye: Yeah, I remember talking to you at GDC 2015 and talking about the process of standardization. And at that point in the evolution of virtual and augmented reality, it was a little bit too early to start to dive in. And so now it's 2019. So it sounds like back in 2016, somewhere in the middle of the year, starting into looking in the process of the standardization. And so what is this going to give? What does OpenXR give to either companies or the wider developer community?
[00:03:13.156] Neil Trevett: So you're right. In the end, I think we hit the timing pretty well back in 2015. It was definitely too early. The timeline we've worked on is the end of 2016. We started to discuss and we found consensus that we should do something. It's been through 2017 and 2018, the bulk of the design work is done in the polishing. in 2019. We did a provisional release in March this year at GDC 2019, where we thought we'd almost finished, but we wanted the community feedback. And I want to take this opportunity to thank the XR community. We actually got some awesome feedback on that provisional spec in March. So we spent the last few months taking that input, polishing the final version 1.0, which is the spec that we have today. I think for the impact of OpenXR in the industry, it's a win-win-win, which is I think why it's getting a lot of positive vibes and a lot of support. There are no losers here. The hardware vendors win because now they have the opportunity to expose their runtime through an open standard cross-platform API, which means they will get more content coming onto their platforms. The content vendors win because now they just write once and they can run across many different hardware devices, so they get better market reach for less engineering effort. But of course the most important winner, as always, is the end user. Because I think we're in the VHS versus Betamax stage of the market, where there's still end user, consumer confusion over what hardware runs what applications. and that is a big friction point in the growth of the market. OpenXR isn't magic, it's not going to solve every problem, but it can help solve that fragmentation problem, because applications that are coded to OpenXR will run across a whole bunch of different hardware, so it will be more like a real market with software that can run everywhere, and that gives end users a good choice.
[00:05:20.460] Kent Bye: Yeah, so when I had a chance to talk to Chris Pruitt, he was the head of Oculus ecosystem. And I asked him, is the Oculus Quest going to be implementing OpenXR? And he legitimately just didn't know. And so I'm wondering if something like this trend towards moving towards these mobile, self-contained, tetherless platforms, if OpenXR gives those headsets anything or if the OpenXR standard and spec is more about peripherals attaching if there's no way to add a peripheral to a self-contained unit then I'm just trying to get a sense of like you see a future if things do continue to get into the mobile market if the OpenXR is going to be agnostic to the PC versus mobile or if you see that there's specific considerations there.
[00:06:02.855] Neil Trevett: Yeah, from the get-go OpenXR has been aimed at both the tethered and the standalone market and both AR and VR, so it's that kind of quad chart thing going on. From the get-go we've had the Google and the Samsung's mobile-based AR and VR participating in the group. I think the most urgent need is in the higher end, the VIVEs and the higher end Oculus devices. But I think it will be pervasive because it doesn't really matter whether you're standalone or tethered in the end, you still need access to the software. And the software doesn't want to write to lots of different platforms. It doesn't really change the equation. And the thing that's actually positively surprised us is how vigorously the AR community has jumped in. I think when we were talking Back in the day, we were expecting the VR to be the pretty exclusive VR in version 1.0, and then we do AR. But that has not actually been the case. The AR guys, most notably Magic Leap and the HoloLens team from Microsoft, they've actually been some of the most active participants. They actually see the need for cross-platform APIs just as much as anyone else, if not more. One of the implementations that's shipping on day zero, the spec went out on Monday, HoloLens 2 has an OpenXR driver that you can download today, as well as the Microsoft Mixed Reality headsets. So even in one company like Microsoft, it's spanning both VR and AR. You mentioned the device integration interface, and I remember some of the previous presentations we've done on OpenXR. We do have these two interfaces. One is the application-facing interface that lets the applications be portable. The other one is the device integration interface that lets devices plug into runtimes. We have found most demand for the application-facing interface. So the version 1.0 of the spec has that. That's what we're shipping today. The device integration interface is going to be in a future release. Not too long, but we wanted to get the application API out there first. People are knocking down the door for that.
[00:08:13.057] Kent Bye: And as I hear this, I'm trying to get a sense between the theory and the practice, because there's the technological spec that is theoretically making it possible to take applications, develop it once, and then be able to put it on all these different devices. And then there's a whole other. In practice, there's all these terms of service and business practices and making sure, just as an example, like you can't upload a Steam DLL into Lockless Home, and so they have different policies that's specific to each platform. And so it feels like you still have to abide by those individual platforms in terms of service and the different NDAs and whatnot. But I guess the goal is that if all of these technology companies implement OpenXR, then the theory would be that you'd be able to write the software once and use the same APIs to do all the user interaction and then be able to deliver it on all the different platforms.
[00:09:03.305] Neil Trevett: OpenXR is not magic, and we're not solving every problem. We're solving one problem, which is the fragmentation in the APIs for software to access the hardware. And that doesn't affect at all how people want to run their app stores, any exclusive bundles they want to have on a commercial basis, between platforms and application developers. OpenXR doesn't directly affect any of that. That's not our business. Khronos' business is open standard APIs. I think it obviously changes the dynamic. Some people might be exclusive to a platform, not voluntarily right now, but simply because they can't afford the engineering effort to take their application onto a different platform, and then another set of effort to take it onto yet another platform. OpenXR will help that. I think applications will be, from the engineering point of view, will be more mobile and more portable across the hardware reach for each developer will be greater because of the OpenXR cross-platform portable API. But the whole business thing, now that's a separate domain and that's none of our business. But yeah, the API fragmentation is one of the problems that we need to solve as an industry and we're happy to be a part of that.
[00:10:13.607] Kent Bye: Well to me it seems like that with this launch of OpenXR 1.0 that it is in some sense putting a flag into the ground saying that XR with both virtual reality and augmented reality could actually be a viable computing platform that is an open standard almost like how a lot of the personal computing PCs have different levels of which that it's a computing device that has all these different standards. Is that kind of the idea is that with this OpenXR now, it's saying that both virtual and augmented reality, whatever company, if they're following the spec of the OpenXR 1.0, that they're on the team of trying to make this a viable computing platform.
[00:10:49.197] Neil Trevett: Yeah, exactly. One analogy. is if you look at the GPU industry that many of us have been in for a while. Before OpenXR, remember all the way back in 3DFX, they had their own APIs, now Glide, and all the chip vendors were basically building their own APIs for people to access the 3D graphics in their GPU. That phase didn't last very long because, of course, it was terrible for growing the market. The developers couldn't move from GPU to GPU. Engineering friction was slowing everything down. That is the phase we're in right now, before OpenXR. And that's the problem we're trying to help solve with OpenXR. Now, all the hardware vendors have the choice of shipping an open standard API, probably alongside their existing APIs, I mean nothing is going to break. But now they have the choice of playing in a much larger market opportunity and I think this will help raise the tide of the size of the market that will lift all boats.
[00:11:49.705] Kent Bye: And within the next 10 minutes or so, you're going to have a whole birds of a feather with OpenXR. So what do you expect to talk about and discuss or announce today?
[00:11:57.930] Neil Trevett: So the main message is to the software developers. It's been a wait. The 1.0 specs always take a while, but it's here now. And it's the result of the best minds from the top companies in the XR space. soul and effort and intellectual capital into this API. We hope the developers like it. We want them to try it and let us know what they think. The time is right now to give it a shot and tell us what you think. It's taken six months longer than we initially hoped, but I think there's a good reason for that. Sometimes when you get a group of companies coming together and they're trying to agree on a common API, sometimes you end up casting away all the bits that people don't want to do and you end up with a kind of a boring lowest common denominator. OpenXR took a little bit longer because of a very healthy dynamic. All the companies came to the table with their first version API learning experiences, wanting to not just create a portable API that they could all agree on, but a conversion to API that they'd learned from the first gen experience and actually build a better framework for the application developers to use across any XR hardware. So I think it's going to be worth the wait.
[00:13:20.545] Kent Bye: And so you're also having some other gatherings here at SIGGRAPH with Khronos Group. What are some of the things that you've either been announcing or discussing here?
[00:13:28.530] Neil Trevett: Yeah, no, SIGGRAPH is always a busy time for us. So this morning we've had the GLTF BoF, so that's of relevance to many folks in the XR space because how are you going to get your 3D assets and models down into your applications? So GLTF has a lot of momentum. We just had WebGL BoF too, so delivering XR experiences in your browser, and then we'll call down to WebGL. So that's a foundational part of the ecosystem too. This afternoon, we're diving into Vulkan. And we're finding out what's new in the Vulkan world, including all the functionality we're adding for high-performance, low-latency 3D rendering.
[00:14:10.581] Kent Bye: And for you in the Kronos group, what are some of the either biggest open questions you're trying to answer or open problems you're trying to solve?
[00:14:19.572] Neil Trevett: It's interesting. That's a good question. The biggest, not necessarily directly related to XR, but it's definitely overlapping, is the largest opportunity that needs hardware acceleration is machine learning and inferencing. How people are going to deliver that. I mean, there's been a lot of good progress in how you train models in the cloud, but still a lot of opportunity, I think, in how people deliver inferencing in end-user consumer devices. and what shape and size of the different standards we need to make that come true is a very interesting ongoing debate. I think more closely related to home and virtual and augmented reality, the hot topic here at the whole show is ray tracing, and ray tracing is in Vulkan, and we're interested to bring that through Vulkan into the XR and AR community.
[00:15:11.271] Kent Bye: Yeah, and it sounds like that the OpenXR is focusing on the application layer. And then the next phase, it sounds like it's going to be looking at the interface between the devices. And I was actually on a panel discussion, moderating a panel with Taylor Beck from Magic Leap. And he was talking a lot about Magic Leap's privacy-first architecture. And so it sounds like there's specific hardware architectural decisions that, if they're designing for privacy first, may have different hardware configurations that may try to actually separate things. Is that part of the open questions of the next phase of how to either deal with the different back-end architectures that are trying to deal with things like privacy?
[00:15:49.010] Neil Trevett: Yeah, we've talked about that before and you've educated me on the importance of privacy in this domain. It may affect things. I think we will find out. I think the XR architecture is going to be okay. But yes, time will tell. We're certainly sensitized to that. No one's raised any problems with it so far. But we definitely have work to do. I mean, we have to finish the conformance tests, and that's going to be in the next few months. We're going to have that done, and that's going to be an important part of guaranteeing reliable operation across different vendors. Microsoft really pushing forward with advanced AR. That's my term. I know that's not an official term. but AR in the headset rather than on the phone. They've said they want to have a set of OpenXR extensions for things like foveal tracking, hand tracking, spatial anchors. Some of the things that are there in ARKit and ARCore, they are awesome APIs, but they're mobile phones and they're proprietary. I think Microsoft is leading the charge to use OpenXR as a platform to get that in a cross-platform way, in a way that suits the higher-end AR hardware as well. And they're saying they want to have some draft extensions for that out before the end of the year, which is awesome. I love it when people have an urgency because it really helps the whole group move forward.
[00:17:11.823] Kent Bye: Great. And finally, what do you think the ultimate potential of immersive technologies are and what they might be able to enable?
[00:17:21.228] Neil Trevett: I am a big believer in both augmented reality and virtual reality. I hate the hype. It keeps confusing people. But I think both AR and VR, you can see how they're growing and how they're going to evolve. It has a beachhead in the commercial space, and I think when the resolution gets higher, I think the high-end gamers will start to come on as well. If you look at the VR industry, it's steadily growing. It's not growing like the exponential hockey stick, like the hype says, but that's okay. It's growing. It's going to be here forever. I think AR is the one that's going to change the world eventually But it's going to take longer than people think to have a true consumer device that everyone wears every day and replaces the mobile phone I think that will happen, but it's going to take longer. It's like any revolution, right? It takes longer than you think but once it happens it has a bigger impact than you can possibly imagine and is one of those. But in the meantime, of course, again, in the enterprise space, AR is shipping in significant volumes, saving big companies a lot of money. You know, it's a real market today. We don't have to wait for the science fiction dream to come true. The AR is shipping out there in the market right now, today.
[00:18:33.332] Kent Bye: Great. Is there anything else that's left unsaid that you'd like to say to the immersive community? Please use OpenXR and tell us what you think. OK, great. Well, thank you so much for joining me today on the podcast. Thank you. It was a pleasure. Thank you. So that was Neil Trevitt. He's the president of the Khronos Group. And he was talking about the OpenXR 1.0 specification that was being released back in July 2019. In the following episode, I'm going to be diving into some of the latest updates with OpenXR with the 1.1 minor release that's coming out today on April 15, 2024. But yeah, just very interesting to go back and listen to some of the original intentions. And at that point, the Oculus Quest had just come out just a few months earlier in May of 2019. And so there was still a pretty distinct split between PC-based VR and mobile-based VR. And since then, I think mobile-based VR has been one of the more prominent ways that has been growing in the context of the VR industry. And so yeah, quite a lot has happened over the last five years in terms of how much OpenXR has been adopted across the industry, which we'll be diving into the next episode, talking about all the different ways that it's been adopted. So that's all I have for today. And I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a lesson support podcast. And so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voicethevr. Thanks for listening.
