Some of the earliest experiments of making VR a first-class citizen on the web originated at Mozilla in 2014. Then the WebVR spec was drafted in a collaboration between Mozilla and Google’s implementations. There’s been a lot of excitement and momentum building around WebVR over the last couple of months first with the WebVR announcements by Oculus at Oculus Connect 3, and then with over 140 WebVR developers meeting for a W3C workshop on WebVR that happened on October 19th & 20th.
I had a chance to stop by Mozilla’s offices and catch up with two WebVR developers Diego Marcos and Chris Van Wiemeersch who talked about the big takeaways that happened at the recent W3C WebVR Workshop. There were some commitments made for a publicly available WebVR-enabled browser from Google in Q1 2017 and a pilot program from Mozilla also in Q1. Diego talks about Mozilla’s new experimental, high-performance web browser Servo implementing the WebVR APIs, and Chris talks the unprecedented momentum and public support that WebVR is seeing across the VR industry. We also talk about some of the open web concepts like progressive web apps, launching WebGL games as native apps using Electron, emerging technologies enabled by the blockchain like the IPFS distributed web, and some of the next steps for WebVR.
LISTEN TO THE VOICES OF VR PODCAST
— A-Frame (@aframevr) September 20, 2016
Hyperlinks are coming to WebVR! https://t.co/pfMgQDI2pf
— Mozilla Reality (@mozillareality) November 2, 2016
WebVR Resource Links
- Sign up for WebVR Slack channel
- IsWebVRReady.org current status of WebVR API implementations
- WebVR Coming to Servo: Part 1
- 3-month WebVR Hackathon starting now
- WebVR: Developing for the Immersive Web VRDC presentation by Tony Parisi
- W3C WebVR Workshop
- GLAM – GL and Markup
- React VR
- WebVR Polyfill
- glTF interview with Neill Trevett
- IPFS distributed web
- Service workers
- HTTP/2 Server Push
- Progressive Web App articles by Ada Rose Edwards
- Coming Soon WebVR.rocks
Donate to the Voices of VR Podcast Patreon
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to the Voices of VR Podcast. So over the last couple of months, there's been a lot of really huge movement within web VR. So web VR, for anybody that's listening to the Voices of VR Podcast, is a topic that I've been covering since the very beginning. Back in 2014, when I did an interview with Vlad Vesovich talking about some of the early experiments that he was doing with being able to drive a VR experience with a web browser. He was at Mozilla with Diego Marcos and Josh Carpenter. And a few months after that is when the WebVR spec had come out after they had talked to Brandon Jones of Google. and started to actually seriously think about how could you create an API that would allow you to go onto the web and be able to see a VR experience. The reason why I think this is important is because we all have this vision of the metaverse and this kind of app ecosystem doesn't really necessarily create a viable way to be able to scale out and interconnect and interlink these kind of closed walled garden silos that are out there right now. WebVR kind of represents this vision of the metaverse that you could link to any other experience and be able to do what Philip Rosedale talks about, one of my interviews that I did with him, talking about Metcalfe's Law, which is that the value of the network is increasing by the square of the nodes connected to that network. So, in other words, it's the interconnections and the links and the portals between one world to the next that really makes the metaverse and the World Wide Web so valuable. So, today I'm going to be talking to Mozilla, where WebVR was really born, and I'm talking to one of the original co-creators of it, Diego Marcos, as well as Chris Van Weemrich, who's also been, more recently, working on a lot of the web standards in WebVR, and helping to push this specification forward. So back on October 19th and 20th was when there was like 140 different people from all the major companies that are doing VR and web browsers and have something that's pretty unprecedented within the web standards community, which is that they just had like a draft spec that a lot of people are already publicly committing to. So we talk about the state of web VR and where it's going, and some of the larger concepts of what it's going to take to really create this decentralized, open metaverse, and some of the other underlying technologies to be able to do that. So that's what we'll be covering on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. Today's episode is brought to you by the Voices of VR Patreon supporters. Do you enjoy all the insights that you gather from the Voices of VR podcast? Then become a donor because just a few dollars a month makes a huge difference in being able to sustain and grow this podcast. I just want to continue to bring you the most in-depth insights and analysis and news from the virtual reality community of where it's been and where it's all going. So, donate today at patreon.com slash Voices of VR. So this interview with Diego and Chris happened at the Mozilla offices in San Francisco on October 28th. So, with that, let's go ahead and dive right in.
[00:03:22.443] Diego Marcos: So I'm Diego Marcos. I'm a developer at Mozilla VR. So I've been doing VR for two years and a half right now. I started at the very beginning at the Mozilla VR team with Josh Carpenter and Vladimir Vukievic. And we started it all with VR. Nice.
[00:03:39.260] Chris Van Wiemeersch: So I'm Chris Van Weemersh. I had previously worked on add-ons and games initiatives here at Mozilla. I joined a year ago to help with the Mozilla VR team, and my first project was A-Frame. And I've been helping out with the WebVR API and kind of the community efforts since then.
[00:03:54.614] Kent Bye: Great. So back in May of 2014, I had a chance to talk to Vlad Vesovich and, you know, also Tony Parisi. And that was before WebVR had even formally been created. And so maybe you could walk me from what happened from the very beginning for how this came about and then where it's at now.
[00:04:12.109] Diego Marcos: Yeah, so I was part of the Firefox OS team doing the now canceled mobile smartphone operating system based on Gecko, Firefox. And I was tinkering with VR on the side and I was doing like JS, using Node.js module packets that used to communicate with the Oculus DK1 SDK via WebSockets. And one day Vladimir sent an email around, I was thinking that maybe we could hook up the browser with the DK1. And we said, hmm, sounds interesting. Yes, go ahead. And he started to implement a first very rough version of the APIs. And he sent out a build of Gecko, experimental build. And we started to play with it. And then Vladimir, at some point, he reached out to Brandon Jones from Google and said, oh, we have this. spec here to connect a VR headset with a browser. Would you like to implement it? Are you interested? And say, yeah, of course. And he jumped in and he also implemented the same draft that Vladimir started and everything went from there. Like a lot of momentum, people jumping in, people tinkering, experimenting, and then Oculus and Microsoft and Intel and NVIDIA and Samsung, they're all on board. So
[00:05:29.505] Kent Bye: Yeah, I think the thing that I've seen is that at the very beginning, there was not really the performance that you really needed with WebVR. And then I think that was before there had been any other headsets out there, just kind of basing the API on the DK1. And then now that there's been other headsets, you could start to make a more robust API for the WebVR. Not only that, now that we have the consumer launch of VR, then kind of at the 11th hour, it sounds like Microsoft came in and said, hey, wait a minute, we've got all these needs that we need for augmented reality as well. And so this sort of standardized platform for putting content on the web is going to be for both VR and AR, and that there was just a pretty huge gathering that happened just a week ago or so. with all these major companies coming together to really hash out the future of WebVR. So maybe you could kind of update us as to what happened at that event and then where we're going from here.
[00:06:24.118] Chris Van Wiemeersch: It wasn't until March that we really took the WebVR API spec seriously and said, what do we need to do to get to 90 frames a second? Because I know when I demoed some stuff with you at GDC, performance in neither Chrome nor Firefox was there yet. So with our platform engineer Kip Gilbert and Brandon Jones on the Chromium team, we redesigned the API basically from scratch and figured out how we could optimize fast paths and make a good developer economics interface. And so with that new API, in Firefox and in Chrome, we have a special VR compositor. And this allows us to bypass all the layout, all the traditional overhead that we had with the old API. So now we're seeing 90 frames a second. And the demos have been fantastic. And the community has exploded. We launched A-Frame in mid-December of last year. And the whole point was to catalyze content and get people excited. And we've seen that. We have over 2,000 active members in Slack. We have hundreds of scenes. But how do we get to the workshop? A lot of it was A-Frame, and a lot of it was the WebVR spec. And so we started seeing Microsoft having interest. We started seeing Samsung. We started seeing Intel. And so you can check iswebvrready.org to see the platform status of all the different browser vendors. We've also seen the introduction of new browsers. So you have Carmel by Facebook, Oculus, and you have the Gear VR browser that was a fork of Chromium being maintained. And how do we get to the conference? Well, so the W3C had pitched that we, you know, do a workshop to kind of forward the standardization process. And right now, WebVR is just a community group. So anyone can go to the w3.org, sign up for the community group, free admission, and you can be part of the conversations. Now, what is community group versus a working group versus standards body? Well, community group is the first step. And the next step will be to standardize things in a working group and then eventually standardize the API. So today, this is unprecedented. An API that has not gone beyond editor's draft is being publicly committed to by various browser vendors almost unilaterally. And so that's really exciting. So how did we get to the conference, as I keep saying? A lot of it was Edge. Edge's public adoption and commitment to WebVR was huge. They added to the platform tracker. And then Oculus keynote at OC3 and the WebVR sessions. Carmack, you know, sat on stage. You know, we believe in WebVR. It's the future. I mean, this is too big to ignore. Last year, that wasn't necessarily the case because we hadn't proved the performance. We hadn't proved the content. That has changed. We don't need to convince people of this anymore. So at the workshop, we had about 140 people, practitioners from VR and web industries, every single browser vendor, including Opera, including Apple. No public commitments from those two companies, but we're excited at the possibility. But then we had industry practitioners, we had folks from Dolby, we had folks from NASA, we had folks from every company you can imagine that's involved in VR. NVIDIA, AMD, Oculus, Valve. And so the energy there was just fantastic. It was electric. I mean, it was so respectful. It was a learning experience for, I think, everyone in the room. We got, like, firm next steps of, like, things that will take us from where we are today as just basically an editor's draft spec into the hands of consumers in the browser. And so the footage will be coming out in the next two weeks. So stay tuned for that. We got a lot of our lightning talks that were recorded. And I think people enjoy it.
[00:09:55.854] Kent Bye: To me, there's a bit of a chicken and egg issue here with people having to have access to some implementation of the APIs to be able to start implementing to see how it needs to be changed. And so it seems like at this point, everything's been put into these developmental experimental nightly builds that people have to go out of their way to download and then to be able to both see and create content. And so it sounds like we're going to be in that realm for a little bit longer before kind of the official versions of the browsers start shipping. Or are we going to start to see some of these APIs in an experimental form shipped, but with the kind of caveat of like, Hey, this is not solidified, but we just want to make it easier for people to consume it without having to go out of their way to download a browser.
[00:10:37.555] Chris Van Wiemeersch: So actually, on that, I should have noted, Brandon Jones from the Google Chrome team and the other folks at Chrome have publicly committed to Q1 is when you're going to see WebVR in a browser. So that could mean mobile Daydream, unlikely desktop, but I can't speak on those details. So you will have bits of Brandon's code in Google Daydream in Q1 WebVR browser. And in terms of us at Mozilla, Firefox, we are shipping Q1 in Firefox 51, but with a test pilot program. So it's to a certain targeted audience. So it's going to be released. And also we have another initiative, Servo. And I'll let Diego talk about that.
[00:11:22.807] Diego Marcos: So for those that don't know, like Servo is an experimental browser engine built from the scratch. So we develop also the programming language that you use for Servo is called Rust. So our research branch decided to do some investigation about new programming languages with the goal of replacing C++. We've been stuck with C++ for a very long time and people had ideas on how this could be improved. But having a language without a big project to validate your ideas is not a good strategy. And they decided, OK, what do we do at Mozilla? We do browsers. So let's build a browser with this new programming language that we are designing. And then it will validate the ideas that we have, if they are useful or not. And Servo started as an experiment, but it grew a lot of excitement around that and expectations and the results were very, very, very, very appealing in terms of performance, in terms of like energy efficiency, and people decided to put more resources on it. And there was a point that said, oh, we have something that kind of works, and we might be able to build a browser on top of it. But as you know, building a browser from scratch is a lot of many years of effort to flesh out all the edge cases and all the quirks that people do in websites. So it is going to take a long time. And people were trying to figure out, what could we apply this new browser so it's immediately useful for people? And there's a lot of conversations still going on at Mozilla, but in the team we thought, okay, so WebVR could be a good niche because at this stage of the game, we don't care about the 2D web. So we are building like more or less like a parallel web that is just 3D using WebGL and all the DOM and 2D stuff is on another dimension. So we can take a lot of shortcuts because it's a lot of APIs and a lot of stuff, features that we don't have to implement in order to ship a VR browser. So we decided to put the resources to fill the gaps in terms of WebGL, Windows support, WebVR APIs, Web Audio, Gamepad API, and to cover only the APIs we need to do WebVR, and eventually ship a browser based on Servo. We posted a blog post last week with the first pieces that we've been landing on Servo proper, and we hope that next year we will be in shape to ship our first MVP browser to users.
[00:13:39.589] Kent Bye: Yeah, and in talking to Josh Carpenter back in the spring, one of the things that he was saying that one of the things that people could do to most support WebVR was to actually start creating WebVR content just to show what's possible. And so given that there was just a WebVR workshop, and it sounds like there's a process of before it gets, you know, completely standardized and put into all the browsers in a kind of uniform fashion. So in this kind of interim time, what are some things that people could do to get more involved into either creating or participating in the process?
[00:14:10.635] Chris Van Wiemeersch: So we have a couple initiatives in the pipeline. You know, we put A-Frame out there as a feeler, but also as a catalyst. So we wanted to see what people reacted to and what their pain points were. And a lot of those have been on, how do I build a scene? I mean, I designed 2D websites. How do I do this in three dimensions? So, learning 3D space, learning about VR and all the caveats and all the varied hardware and the fragmentation, and dealing with the experimental APIs. And then another one has been, how do I make something that looks good and feels good and feels fun? And this is something that is not necessarily achieved to a good degree on native yet. If we think about what the web has to offer, we have hyperlinks. So my colleague Casey and I demoed at the W3C workshop, link traversal. Blog posts coming very soon, video demo. That's huge. What that enables is for people to go from one scene to the next while staying in VR. Now this is something that is not achievable today from SteamVR or for Oculus Home. Not to say it's not technically possible, but it's not there. And the frictionless accessibility is huge for users and for machines and search and so forth. So what we're going to start seeing is a catalog, a directory, if you will, of sites that are WebVR enabled that work well for your device that you plug in. And so you'll be able to filter and see some of the best showcase pieces. The other thing is asset creation. People don't know where to find good assets. There's a format being led by the Kronos Group called GLTF. It is meant to kind of be the JPEG of 3D and it's meant to replace formats like Collada and FBX. So you're gonna see a lot of push from us and from the other web VR participants and trying to evangelize GLTF as the new future looking format that's optimized for content delivery and that we can get this in the hands of people and have artists contributing to this virtuous cycle.
[00:16:06.213] Kent Bye: Yeah. And one of the things that I've seen come up in some of these discussions is the concept of a distributed web, because right now all the web content is served from centralized servers. And, you know, I'm just curious to hear how you see the future of this potential distributed web and how that plays in and why that's important for web VR and VR in general.
[00:16:26.989] Chris Van Wiemeersch: So I can speak on some of the thoughts and experiments we've been doing. I think Diego has some ideas here too. So I'd encourage folks to look at IPFS. IPFS is built upon the blockchain, like how Bitcoin is. And the idea is, instead of having centralized authorities where they host, they own the content, if I have a 400 megabyte video file, I got to pay for that and put that on S3, or put it on Heroku, or put it on GitHub pages. And that's a liability, not just in terms of SLA, but in terms of potential censorship by political governments from the state. Or someone might just forget about it and it might go down. So IPFS is an attempt to make a distributed, not necessarily decentralized, but even distributed way of storing and accessing data. So, it's very similar to Torrent. I also encourage people to check out WebTorrent. So, you can imagine some experimentation being done with IPFS as a distributed data store where I can just take my video file or take my assets, artists can take their assets, post them to IPFS. They have a unique hash. And so, whenever anyone else uses these assets, they can be uniquely attributed to that content creator. And I think there is kind of the first seed of where we're going to start seeing possibilities for monetization, for proper attribution, and it's really going to catalyze content creation and I think give people that metaverse they've been thinking about.
[00:17:58.888] Kent Bye: And also give the internet a little bit more resilience because right now it's really up into the individual website owners to host and pay for all that content and if something goes viral they have to pay for it, but yet This is kind of a way to distribute that, but also give a longevity to something that really actually want to last for a long time.
[00:18:16.918] Diego Marcos: Yeah, that's the struggle. And it's the role that Mozilla plays. So you have each big company creating their own verticals. They want to control identity. They want to control like content distribution. They want to create their own economies around their own ecosystems. And at Mozilla, we want to do the opposite. We want to decentralize and democratize all these different pieces. Some of them we are experimenting and others we don't even have an idea how to handle identity in a decentralized way, right? Or what Chris was saying is like, if you're going to share your assets, how can you make sure that you protect your assets and you can make money? And there's proper attribution and precedence. And if you do derivative work, how can you refer to the original work? Those things are questions that we are trying to find the answers for, but the starting point for Mozilla, we want to solve those problems, but in a decentralized manner. So yeah, we'll see how it goes.
[00:19:07.989] Kent Bye: No, that's really cool. I think it solves a lot of the issues, especially with things like Linden Lab, which if you look at Second Life, which was just sort of like you were paying rent to be able to actually host. content, and I think that you still have to pay. So either someone else is controlling you in some way, or you have to pay out of your pocket. And so it just seems like taking the lessons of the blockchain and applying to the web seems like it makes sense. It's just making it happen and facilitating it such that the next version of the metaverse, the web, which feels like it's being built upon the foundations of the internet, but trying to iterate and do it a different way in terms of solving a lot of the issues that were there before.
[00:19:46.080] Chris Van Wiemeersch: Yeah, yeah, no, totally. We're coming to a different landscape too, right? With the introduction of Electron, it's kind of shown that you don't need the web browser to do web things. So I can be a web developer and develop an application like Spotify, and people can download it. It can be distributed through the traditional App Store models. But I can also still have people load my website through a browser and not alienate anyone. I can reach a wide audience base. And so things like React Native, Ejecta, a project for iOS, are attempts to take the web technologies and not necessarily be married to them, but use them as a way to get out there. So we already have these existing, we already have Steam, right? And so today there are games in Steam that are built atop WebGL, but you wouldn't know. They want to write their game once and have it work everywhere, so they wrap their game in Electron, they ship it in Steam as a standalone literal browser, and people download it. And they can't tell the difference. So you can imagine a very, very, very near future where people will be both submitting their content to Oculus Home, to Steam, as traditional apps, because that's what people are familiar with in the gaming world, but also you can deliver it through a website. And I think that experience of going from one to the next Not having to download 50 megs or 100 megs for a browser is immensely, immensely satisfying. It's things like service workers, things like HTTP2 server push. There's a lot of folks out there really pushing on the kind of progressive web apps of WebVR. I'd encourage folks to check out Brian Charles of DataVis. He has a lot of good projects evangelizing the best practices here. So, if you want to find out more information about kind of what we've been up to and what's in the pipeline, I encourage people to check out modsvr.com. We'll be linking to a lot of these kind of best practices.
[00:21:40.396] Kent Bye: And finally, what do you think is kind of the ultimate potential of virtual reality and what I might be able to enable?
[00:21:47.392] Diego Marcos: That's for me an easy question because it's the reason that made me start doing VR. I just want to transcend the physical world and see how society transforms where we free ourselves from the constraints of the physical world. It's like manufacturing is not relevant, traveling is not relevant, your physical appearance is not relevant. So how the economy, society, and interactions between people are going to change in this completely new world that we are building from scratch. For me, that's the ultimate potential, like transcending the physical reality of our society.
[00:22:27.417] Chris Van Wiemeersch: That's deep. Oh yeah, for me VR is important because it's where we're furthest along with 3D. And you have AR, you have MR. These things are not in competition with VR. They're all kind of pushing towards the same thing of having a third dimension to our existing reality. Imagine holograms too. So that's what we want for the browser. I'm a 2D web developer for 20 years. I want to learn 3D web skills, and because I know that's the future of the web, is the third dimension. So that's what we're reinventing. This is literally a renaissance of going from the 2D to the 3D web, and VR is kind of that first step. So I'm really looking forward to it, and I invite everyone to participate. Like, let's remember, the web is not owned by anyone, and web VR is not any different. It might just be an API today, but it's an ecosystem, and it's a community, And we're all part of this.
[00:23:21.144] Kent Bye: Awesome. Well, thank you so much.
[00:23:22.908] Diego Marcos: Thank you. Thank you for having us.