Mozilla Hubs is an open source social VR platform built using open web technologies. You can create your own 3D scenes using their Spoke platform, or launch your own private Hubs server using the open sourced infrastructure code for Hubs. Mozilla has long been an advocate of the open web and has been helping to build future of virtual reality on the web with WebVR and WebXR since the beginning of the modern resurgence of VR.
I had a chance to drop by the Mozilla headquarters in Mountain View ahead of the Augmented World Expo in order to talk with Hubs engineering lead Greg Fodor, product manager Liv Erickson, and Hubs engineer John Shaughnessy. We talked about how to architect for privacy on the web, some of the open web technologies that they’re using, the advantages of developing on the open web, and how they’re building trust with users by making all of their WebXR code and infrastructure code openly available on GitHub.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
There have been monthly gatherings of a Sensory Design Slack group led by Adobe’s Silka Miesnieks, and a big reason why they chose to host using Hubs was the ease of use of being able to share a link on social media and to be able to join a virtual meeting space on either a laptop, mobile phone, mobile VR headset, or PC VR headset.
Disclosure: Mozilla is financially sponsoring the Voices of VR podcast to collaborate on four different events in 2019, and this is our second event. The first event was the Women in VR discussion that happened at Sundance.
I’ll be at the View Source 2019 conference in Amsterdam recording a live session of the Voices of VR podcast. It’s an open web conference sponsored by Mozilla, Samsung, Microsoft, and other members of the W3C, and here’s a discount code for 25% off: VR_VS2019
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. So the day before the Augmented World Expo began this year in 2019, I dropped by the Mozilla offices in Mountain View, and I had a chance to talk to the team that is working on Mozilla Hubs. So Mozilla Hubs is a social virtual reality space, and it's made completely within WebVR. You can share a link on Twitter and you can just hop right in into an experience. It's also very cross-platform, so you can look at a mobile phone, on a tablet, on a VR headset, mobile VR, full-blown PC VR. So it's really quite flexible in terms of being able to hold these different gatherings and to be able to have a spatialized experience. And using a lot of different open web standards from WebRTC and WebXR, So it's really pushing the edge of what's even possible with creating a social VR experience using these open web technologies. And all of the social hubs is open source and all the infrastructure to be able to get it up and running is also open source. And so Mozilla is really trying to take a firm stance of trying to say, okay, if we were going to really architect a system for privacy, this is what it would look like. And you can actually go and audit the code and everything else like that as well. So at the end of last year, someone from Mozilla came to me and asked if I might be interested in exploring a number of different collaborations. And so they actually gave me a sponsorship to do a series of different events. And this is kind of like the second event of our series. The first event was back at Sundance where they sponsored a discussion about women in XR and some of the issues that come up. So the things that Mozilla is doing, I really respect just because they're really embodying their values. And actually there's a lot of alignment between a lot of the issues that I talk about here on the podcast around privacy and along, you know, the future of the open web, decentralization, you know, there's so much alignment between what I'm doing, what Mozilla is doing that just made sense for us to explore a variety of different collaborations. And so this is the second collaboration that I'm doing with Mozilla. And so this was just an opportunity to drop by their offices and to kind of get up to speed as to what's happening with their open web platform and where they're taking this as a platform. So that's what we're covering on today's episode of the Wastes of VR podcast. So this interview with Greg, John, and Liv happened on Tuesday, June 28th, 2019 at the Mozilla headquarters in Mountain View, California. So with that, let's go ahead and dive right in.
[00:02:35.847] Greg Fodor: I'm Greg Fodor. I head the engineering team here on working on Hubs. And I've been working in social VR for a number of years now. Previously, I was at AltspaceVR and been at Mozilla about a year and a half working on Hubs.
[00:02:48.830] John Shaughnessy: My name is John Shaughnessy. I'm an engineer on the Hubs team. And I guess I got interested in VR just thinking about how we're going to use it to change how we communicate, how we're going to augment our conversations with media, and how VR is going to play a role in that.
[00:03:04.060] Liv Erickson: My name is Liv Erickson. I've been working in social VR for about two and a half years now. I joined Mozilla about two months ago to work on Hubs as a product manager.
[00:03:13.163] Kent Bye: Great. So maybe give me a bit more of context as to how did the Hubs project come about?
[00:03:21.768] Greg Fodor: Sure. So Mozilla's been kind of pushing the web forward for VR for a number of years now. And so I guess it was about two years ago, you know, Mozilla was starting to experiment with social VR before I joined, but there were a few prototypes in place for doing avatar tracking and things like that to kind of record and play back avatars and get a sense of like, what could you do? And so it was around then that a bunch of folks from AltspaceVR were kind of going through a transition, and so a number of us were able to get connected with Mozilla on some of these social VR efforts they were already kind of working on, and Mozilla was looking for a way to accelerate those. And so we kind of joined the team, I guess it was now about a little less than two years ago, and our first task was to kind of prove out what could you really do. You know, I think at the time the prototypes were getting the basic pieces in place for replaying avatars, getting tracking and all that kind of stuff from controllers and stereoscopic rendering in the browser. And then what we were doing from there was trying to see, could you get a multi-user experience that actually worked with networking, with WebRTC, getting good performance on mobile devices, all that kind of stuff. And so it was about, I'd say, two to three months we did some prototyping and we had something that worked fairly decently well for kind of a small group of co-present avatars in a virtual space running on the lower end mobile VR headset devices that were available at the time. I think that generation was like a S8 Gear VR. And so we actually did a really lightweight demo of that at an SBBR meetup out here in the Bay Area and got some really positive feedback. And so we kind of came back from there. I think this was around January of last year. And so we kind of huddled up and tried to decide what we were going to do next. We really did feel that the bar had just kind of been reached where you could really sit down and build a pretty compelling social VR application that ran entirely in the browser. And it would be a lot of work, but it felt like it was tractable to be able to do that and deliver a reasonably decent experience on mobile devices as well, which was really important to us. And so that's kind of what kicked off our decision to productize the prototype we had made. And at that time, very, very simple, just avatars co-present. But you could see each other. You could hear each other. You had head tracking. You had hands. And so over the last year and a half, we've kind of iterated on top of that prototype and kind of made it into a product that we think is pretty good for getting a lot of people co-present into a space with no downloads, very low friction, cross-platform, privacy preserving. And so it's been a really, really fun journey building up to that.
[00:05:45.557] Kent Bye: Yeah, there's been this sensory design group that's been meeting on Slack and then started to have these weekly meetings. And looking at all the different options, it turned out that the Hubs was actually the easiest to not only get into an experience by following a link and then just jumping right into that experience. But also, if people don't have a VR headset, they could use the mobile phone or on their PC. Or if they did have VR, they could also have a full embodied experience within that. Thought that the audio was like great. It wasn't lagging too much. It was actually a really good experience Although I will say that I feel like it's still kind of to the point where that's maybe not to the full 1.0 release it feels like it's still like maybe beta or alpha like still bugs and stuff getting figured out but in terms of like Being able to go from social media from one context, jumping right into an immersive world with other people speaking in real time, it's super impressive that you're able to get it to the point of actually running it in the browser. And so I guess one question that I have is this tension between the native code and what you can do on the open web. And I feel like, actually, this is a really good example of really pushing the open web to its limits. And I'm just curious what kind of either trade-offs you had to have in terms of having some sort of native app on the side, or how were you able to get to this point of pushing the open web to the point of being on parity with native code?
[00:07:10.948] John Shaughnessy: I think one trade-off we identified early on is the difference in what kind of tools are available for game creation on the web versus on native. So we chose a tech stack that was what those early prototypes were using as well. It was A-Frame and Three.js all built on top of WebGL. And that's worked out pretty well for us. And what we learned by doing it on the web is all these extra benefits of being in the browser firsthand. So you alluded to it already. You're in a social media context, and that's where you're communicating with your community. And then you can immediately add on this layer of avatar-embodied communication. You're co-present in this space. Because we're on the web, we can integrate seamlessly with a lot of your existing communication channels. So I guess that's like a trade-off in favor of the web of you don't have to own the entire stack of conversation. And this will, I think, help us bootstrap and get people started. And you don't need a VR headset to make use of spatial communication. The headset is one part of what makes even a meeting like this. I mean, we're huddled together really close with each other. But if you heard your name somewhere over there, or if people were talking in the corner of this room, we could have these simultaneous conversations going, which you need space. You need audio to be spaced out in order to make this happen. But you don't necessarily need a VR headset to do that. have the benefits of this. If those meetups were happening in a video conference, well, it'd be a lot harder to have multiple conversations going or anyone being able to submit new visual information to everyone at any time. You'd have to share the screen and say, OK, it's my turn. I want to take over everyone's box, and I want to put something on the box.
[00:09:05.283] Kent Bye: Yeah, I'm wondering from your perspective, coming in for the last couple of months, what you see as your role of taking what's already been done with Hubs and what you've been able to do the last couple of months and where it's going next.
[00:09:16.766] Liv Erickson: I think what gets me so excited about Hubs is that because of the benefits of being on the web, we're able to reach a lot of people with the communication and collaboration aspect. So for me, seeing how people are using Hubs today and thinking about, what we collectively as humans will do as these technologies continue to evolve in order to be more effective at how we collaborate and communicate with each other. That's really powerful and figuring out how to build features that solve those challenges of being spatially co-present while we're geographically distributed and what that means for the different types and formats of communication that we have. and how that fits into how, at one point, maybe we're talking via text message, or we're sharing GIFs of something, and then that seamlessly being able to move into a shared spatial environment. All of those questions about what that fundamentally can do for communication and then figuring out how to make that a product that runs in the browser is what I'm going to be looking at in the coming months. And it's something that I'm particularly excited about because of the number of people that it can reach.
[00:10:20.127] Kent Bye: Yeah, and I know that at Altspace, there's actually quite a bit of a JavaScript SDK to be able to start to pull in different elements of the web into what was a native code foundation for the Altspace experience. And so you had this blending of pulling in elements of JavaScript in the open web and trying to empower the web developers to be able to create spatial experiences. And so it seems like it's a nice fit that a lot of the people that were working on that are now working on just a pure open web implementation of that. But at the same time, there's a little bit of lack of official launching of WebXR spec. You have A-Frame, which is also there, that is kind of a wrapper around the 3.js. And I'm just curious to hear a little bit more in terms of, the underlying technologies that are piecing all of this together and what you're relying upon in terms of, you know, I know that just in talking to Diego, who used to be at Mozilla, talking about how Mozilla was really wanting to get this out there as soon as possible and to use A-Frame to be able to have implementations of WebVR, WebXR. where you would allow the framework of A-Frame to be able to handle the upgrade process. But that was years ago now that he told me that, and that we're still kind of waiting for the official launch of WebXR for the entire web community to be able to do that. So I'm just curious to hear a little bit more in terms of the underlying tech stack that is enabling all of this.
[00:11:46.507] Greg Fodor: Sure, I'm happy to talk about that. So one of the things that's been really interesting working on now this project for a few years, and having worked on native applications for VR as well, is it's been really surprising to just see what things are actually a lot easier on the web. And then there's also some things I think people acknowledge are more difficult, right? And so some areas that we've leveraged that were amazing to see how easy they were comparatively are first, every mainstream browser now has first class built-in support for voice and video streaming, right? And so this is something that is a pretty big challenge to implement in a native application. You can sometimes pull things off the shelf, but oftentimes you have to kind of work on it further from there. And so we've been just super thankful for all of the work people have put into web browsers, including the WebRTC stack we're relying that, as you've experienced, provides this rock-solid voice communication that just works. And so that's like an aspect of a part of the tech stack that we've had to do very little work our own. And it's just worked very, very well. In addition to that, another aspect that I found to be extremely nice working in the web is the speed of iteration. And so browsers over the last five years or so, or maybe longer, have introduced incredibly powerful debugging tools and inspection tools for understanding the state of what's going on inside the computer, right? The speed at which I can make a change and actually see it on a headset, especially a mobile headset, is absolutely crazy compared to what you might have to typically go through to get a fully production simulated build onto a mobile device. Like typically in a native application, you have to recompile, you have to package things up. And sometimes those compilations take a long time because if you want to run the actual thing you're going to ship to users, you have to run a whole bunch of optimizations and so on. And so whereas in a browser context, you literally refresh the page. And you can actually attach a debugger to it over a socket and start inspecting things without having to do any upfront work on your own, other than just connecting your phone to your computer or your headset to your computer. So those are like two elements where it's just a huge leap in terms of velocity. Now, of course, the web ecosystem is one where it's a very open source ecosystem. And open source has this inherent thing where there's a lot of stuff going on. And it's about consensus and moving things forward in a way that everyone feels makes sense collectively. And so there's a lot of work going on that is arguably behind in terms of raw capabilities, but it's kind of moving forward in a very methodical way. And so from a first principles perspective, the way I look at it from an engineering point of view is what's the kind of low-level infrastructure that either exists or is, like, tracked to become existing over the next several years. And that kind of gives you your table stakes, like, of what is possible to do in the browser. And things are looking really, really good right now in terms of just being able to deliver a parity kind of experience in a browser compared to a native. You have all kinds of things, obviously, going on, like you mentioned, in terms of just having the VR support be built in. There's consensus on that spec, and things are going to be moving out for that. But you're going to be able to do Vulkan-style draw calls with WebGPU. You're going to be able to do off-screen canvas rendering to move your draw calls off of your main thread. WASM is something we're already leveraging extensively in hubs for doing our physics simulations. So we're leveraging more amazing work by people out in the open source community who have ported C-based physics engines to the browser. And so now, with very little effort comparatively, we're able to integrate world-class, production-grade physics into our experience, and it's running at relatively close to native speeds. And so all of the kind of pieces are kind of either in place or coming into place, where it's now kind of the, quote, simple matter of programming, right? You just need to kind of iterate and keep working on things at a higher level in the stack. But things are, I think, looking pretty good for having that basic infrastructure in place in the browser over the next several years, where you will have at least the medium you need to deliver something comparable to a native experience. But there's all these other things too. Like I mentioned that it's not just a fixed one is better than the other There's areas where I found a huge improvement and a huge increase in velocity and then other areas where you know You can kind of see it coming down the line and people are all pushing in that direction So it's been really fun to work on though. It's been a really kind of new area of seeing what you can do. So I
[00:16:04.849] Kent Bye: Yeah, and one other big point is just to have a graphical user interface for people to go in and create your own worlds and to pull in different assets from either Sketchfab or Google Poly. And I'm just curious if you could expand a little bit more on the Spoke and this almost like game engine in the browser, or at least a scene creator that you're able to build out scenes.
[00:16:25.538] Liv Erickson: Yeah, so being able to create content for virtual spaces is something that's a pretty critical part of being in a shared space. It really emphasizes and amplifies the experience that we have, and it affects our communication. If we're in themed rooms for meetings, we tend to generate content that's related to that room, and sometimes it makes sense to have a business meeting in a very professional environment, and sometimes we have environments where we're laid back and hanging out with our friends. So being able to create environments super easily that people can use in Hubs is a motivating factor behind Spoke and making that available in the browser, too, so that being able to composite 3D assets that you're either sourcing from, as you mentioned, places like Sketchfab or Google Poly or uploading yourself, mixing that in with other types of content that already exists out on the web, so images, GIFs, videos, all of that's a really important strategy for recognizing the different ways that we currently communicate and being able to start taking the steps that we think is really critical for having these types of shared collaborative spaces by making the environment reflect the type of communication or experience that you want to have. Spoke was originally launched last year as an executable that you would download and run on your local computer, and it's now, as of a few weeks ago, entirely available in the browser. So it's seamlessly, you know, you can go to hubs, you can look at the scenes, you can create a new scene and then deploy that and be in the room. And speaking for myself, my 3D modeling experience is a little bit rough if we're being generous. So being able to composite from this amazing ecosystem of Creative Commons licensed content that artists have made available to users and bring that together to create scenes and environments for the types of meetings that I'm having has been a really, really powerful tool. And we want users to be able to have that power to create spaces that genuinely reflect them and the types of communities that they're meeting with in Hubs.
[00:18:20.518] Kent Bye: Yeah, when I was talking to Neil Chevet of the Khronos Group, one of the things he said to me that really stuck with me back in, like, 2015, he said, for every successful open standard, there's a proprietary competitor. And looking through all the history of graphics and also on the web, that there seems to be the closed-wall garden approach and then the open ecosystem approach. And so now with the launch of the Oculus Quest, it's pretty locked down relative to previous versions of Oculus. But yet there's still, thankfully, a web browser that is almost like a backdoor for people to be able to have all sorts of web VR experiences that people could still have access to. And so one of the things that I tested out almost immediately when I got my early version of the Quest was to jump into a social VR hub and just to see if it worked. And it did. It looked great, and it felt great. And so I imagine that there's going to be a lot more of web VR content that's there. But I'm just wondering if you could speak a little bit about the strategy of having a hosted open web version of both the hubs. and the spoke, and that with the future of the completely decentralized open web, where you see it going in the future. Because it feels like there's economies of scale, and you don't want to have to have people need to install their own web server in order to even have opportunity to have access to some of these experiences. But I can imagine a future where you'd want to empower people to either host or create their own version, and kind of fork it off and, have people do what they want with it. But I'm just curious to hear a little bit of that strategy of this kind of hybrid, centralized, but sort of decentralized empowerment of the open web, but things that you're hosting locally there via Mozilla.
[00:19:57.671] Greg Fodor: So I think there's a lot of dimensions to this. There's a lot of these problems that are kind of often coupled together, but I think they can be separated somewhat. So one of the things that's, I think, been at least really important to me with this project is if it does prove itself to be useful for people, that it has a variety of ways to succeed. And that no one particular variable, if we guess it wrong, could jeopardize the future of the project. And so I think, in general, decentralization is one element of how you can pay down some of the potential existential risks to a project like this. But I think some of the other things we're doing around how we're open sourcing the project and the kinds of things we're building towards for people to do with it also help mitigate some of that risk. So one example of this that I'm actually pretty proud of is that we've open sourced not just the software, which is useful, but we've also open sourced all of the operational scripts and Terraform scripts and all the infrastructure scripts, and we basically have shown everyone how we operate it. And having come from a background working on web services and e-commerce and things like this, operations and systems design and reliability and all these things are just as important as software engineering and architecture and client features and UX. And so to me, that's an important thing, that if you have an open source project that you expect others to be able to run, having openness about your operations and your systems design and that kind of thing is a critical part of that if you expect it to be carried forward in situations where maybe you couldn't carry it forward yourself. Another element of this that I think ties into things like privacy and decentralization and user control is this notion that you want the software to be auditable. You want to be able to prove that the interaction you're having with the software is actually the bits that are published. And so this is something that we haven't done a whole lot of work yet on, but it's also extremely important to me that if we do find ourselves in a world where you have a lot of VR headsets out there and people being tracked through VR headsets to embody avatars, there needs to be a way for people to prove what's happening to that data. And it doesn't just mean that you read a privacy policy. I think it also means that you have some kind of like cryptographically secure most way to know that the bits that you're interacting with across the entire system are the ones that are published publicly by the organization who's serving you. Right. And so I think that The mental model that everything is going to either be decentralized or centralized is not quite the right one. I think there's going to be a spectrum of these things. The way we're approaching this now is trying to be as open with what we're doing, making it so that it minimizes the existential risk that it could be carried forward by others if it needed to be. Beyond that, we're carving out tactical paths for one-off. customizations, or self-hosting, or things like this. So the paths we have right now are you can take hubs, you can clone it out of Git, and you can run a client fork, which basically, it's a streamly learning curve, right? It's kind of as you would imagine. You just download it, you change the code, you put it up somewhere on the internet. and it should just work. And the way that that works is it uses still centralized servers that we run, but we made that an important workflow because I think that's, if nothing else, the easiest one to get started with. So we've had people take hubs, fork it like that, change the graphics a little bit, or change the UI a little bit, and now they have their own private little version of it, but they're still going through our servers. Then beyond that, there's other paths for people that we're kind of starting to look at for ways that you could actually run the entire infrastructure yourself. And there's a whole spectrum of options that you could probably imagine could exist for that, that are kind of supported by us to varying degrees. And so that's stuff you'll probably start to hear about over the coming months. But overall, we're kind of looking at this from many dimensions, as I'm kind of getting at, so that we can have the ability for people to take and run with it now in different directions, but also making sure that in the long run, that if this is a useful thing for people, that it ends up having a very long life and is something that can be used for a variety of different reasons.
[00:23:44.187] John Shaughnessy: Yeah, I think, you know, when you talk to a lot of people about what kind of communication apps they're using now and their privacy concerns around it, a lot of people are resigned to say, well, maybe I'm being spied on by this company or by this agency or something, but I don't want to convince all my friends to move to a different platform when that platform, maybe it's owned by the same company or a company doing just as much stuff with my data that is unsightly. Like I don't want them to. There's this book, Surveillance Capitalism, that I read recently, and it talks about this problem of, well, I want this service, and the way it's going to use my data, like the way I interact with service, is going to inform it of all these things I didn't mean to tell it to. One version of what success looks like for our project is to give an example of a robust, competitive, social VR app that actually does respect your privacy. That says, hey, it's really important that you can feel safe expressing yourself to this community, your private community, in a way that's not going to Suddenly out you for something that you didn't mean to go public with or well, we don't do this but a lot of companies like to take Really specific information about what you look at and for how long and who do you talk to and what words you use when you? Talk to them in order to serve you ads or to change your future behavior in a way that benefits them in the marketplace like it's not for the benefit of you is for the benefit of the person who's sending your letters back and forth, right? So I think a lot of people aren't aware that they have options that protect them from these things and that choosing those options matter. Like supporting the company behind your communication platform means that company can continue doing the work that they're doing on your behalf. So I hope that As we see more of these types of applications and platforms come up, native or on the web, there are examples, ourselves included, of, hey, this one is actually pretty good to its users and for the community of people who use it, not just for the company who captures the market.
[00:25:49.410] Kent Bye: Yeah, I really appreciate that emphasis on both the ethics and the privacy, because I think there's a lot of discussions that are happening. And it's nice to see that there's alternative architectures and code that's out there that are implementing a different perspective. And I'm curious how that gets embedded into either the code or the architecture, how to embed your values of that. and what that looks like with the product, even if they fork it off, if it sort of carries forth, and what the type of things that you were specifically looking for, either towards the privacy concerns or permissions and asking for consent and not trying to ask people to use their camera and then secretly spy on them later, being fully disclosed, things like that. But also, yeah, there's a certain sociological and cultural things that can only happen for once you're in there. But I think also things that you can do from the architecture perspective to be able to instill these types of values. So just curious to hear a little bit more about that.
[00:26:44.780] Liv Erickson: I think Greg talked a little bit about some of the decisions around, you know, how we've chosen to open source all of the platform in order to, like he said, be auditable about how we are using our position at Mozilla to be building this platform that's, you know, a highly trusted format of communication. VR in particular can be very, very personal with the biometric data that can be surfaced, you know, reasonably so to get those services to you. We've made a couple of decisions about product design and product features that are based in this putting the user first. And that's kind of through and through a big part of Mozilla's mission is building a strong, healthy web ecosystem as it's evolving that is focused on putting the user's needs first and foremost. So when we think about that with relation to hubs, part of it is the design of some of the core use cases that we're looking at supporting, like making private spaces for you and trusted users that you already know and not having a hosted central repo of all of the different hubs that people are creating so that you can go and explore places. The internet has, a lot of social media sites that are out there today have not been built with this idea of privacy and consent being the first and foremost factors of why they were created and how they're being run. And we think that with hubs, you know, we're in a position to look at communication and collaboration a little bit differently with those principles in mind by making it consent driven. You as the room owner are choosing who you are bringing into your space, and that may change from meeting to meeting. So from looking at the product perspective, that we have that kind of core of Mozilla's mission there, and we want to make sure that we're making decisions that amplify that and support that mission. And I know that everybody on the team feels really strongly about that, too, as kind of a personal value that we hold.
[00:28:33.162] Greg Fodor: One thing I could add to that. So one thing that's been really, I would say, rewarding and fun working on this project is that this is almost kind of like a fun puzzle as an engineer. Previously, basically every product I've worked on has utilized data collection for the purpose of doing analysis of user behavior for the purpose of improving the product. And that is kind of this inherent trade-off, right? I mean, ultimately, your observability into a system and a product informs how you improve it and so on. So one of the kind of exercises we go through with something like Hubs is every feature we think we want to build or every element to a product that may require data, we try to game it out and think, how could we do this without this data? How could we do this with data that isn't stored on our servers but is instead stored in the browser? For example, lots and lots of the information about your setup is not stored on our server. It's stored in your local storage in your browser. And those were things that normally the first thought would be, oh, let's just put in our database, right? And so there's a lot of these kinds of design decisions that whenever we even come up to a place where we're like, OK, we need send some new information over the internet, or we need to stick some more information somewhere that's stored on a disk somewhere, we always game it out and think, OK, can we kind of fake this feature halfway without doing any of that? Or can we do it in a way that it's completely not on our server? And all that kind of stuff is actually a really fun engineering problem. And it turns out to really be nice, because now what you end up with is a system that has way less information about what people are doing, and who they are, and all that kind of stuff. And so it just kind of is a self-sustaining thing. Like, for example, When you create an account in Hubs, we don't actually store your email address, right? And that's counterintuitive. You would think we would have to, but we actually came up with a way. It's fairly simple, but it's not something that you would just immediately think of unless you were coming at it from this design goal of, like, how can we absolutely minimize this as a first-order concern and then build the feature that we want still, right? So that's been really fun.
[00:30:34.073] Kent Bye: And so as you look forward, I'm wondering what type of open questions you're trying to answer or what type of open problems that you're trying to solve.
[00:30:43.021] Liv Erickson: I think I spoke a little bit to this. Fundamentally, we now have these new devices that are changing the way that we can collaborate together. What we've seen with flat social media is that we have this kind of unprecedented connection to almost half of the planet. And when we think about, as the other half of the planet comes online, and now all 7.5 billion people are able to be connected, and they're able to be connected in a way that feels very shared and are sharing these virtual spaces together and able to pull in all of this content from all over the web on a huge array of devices and operating systems. That's a kind of unprecedented ability to knowledge share up until this point in history. We're kind of sitting at this problem, looking out at the future of communication and the Internet and thinking, like, OK, well, what do we do in the next six months? What do we do in the next year? And we're fundamentally thinking about what it means to bring in all of these different pieces of this puzzle that has started to be connected with the growth of the Internet, with the advent of all of these different devices, augmented reality devices, better VR devices. we don't really have a standard other than what we've established for face-to-face communication. And that hasn't been as cross-culture as the internet will allow it to be as it continues to expand. And to me, that's a very large potential space of just humanity's next step to think of. Like, that's a big question. But that's kind of what we're thinking about with what we're thinking about in terms of building platforms that support private human communication across devices with all different kinds of media.
[00:32:21.625] John Shaughnessy: Yeah, I can tag along with this. This is a VR podcast, and so we highlight the VR aspect of our work. But when you think about it, we're in this space that has a much longer history even than VR. And I'm not talking about second generation. OK, 10 years is not a history, right? But even before VR, we're using computers to mediate communication and to mediate between groups trying to cooperate. So in that space, trying to figure out, OK, we have new devices coming, and we have a bunch of new users coming online all the time who don't have Old metaphors of how they interact with like a keyboard and a screen already built into their head like the open problems There are what should computers do in between my brain and yours like what how can we? augment that I mean even in something like avatars. You know, you talk to a lot of people about avatars, and they want a lot of different things. Some people want to express themselves. Some people want to express as little as possible, like, where am I in our shared space, and what am I looking at? Like, I don't want you to think that you're learning personal information about me by looking at my avatar, just real-time useful information. Or, you know, the opposite end is, I want to just make my super unique avatar that I wear to every VR event I go to and people can recognize me. When you think about an avatar in the abstract, it's like this constant visual message you're sending to the people around you. And if you can animate it and you have some utility around it, your avatar is that plus all the ways you can change it. So, Begs the question where does your avatar end if I can show you an image at any time? Like is that image part of my avatar not really because it's not persistent because I might take the image away But this question of like well, what should avatars even be it's which part of this computer mediated interaction Do you want to be persistent and identified as you?
[00:34:23.209] Greg Fodor: Like to throw my thoughts on some of this too, I mean, so I've seen people interact with each other in a VR context in a way I know they cannot interact with other media through the internet, right? So you have fathers getting physically on the floor with their kids and playing drawing together, right? I mean, they're doing this through the internet, right? And they're having this deep, real emotional connection with people they love, people they care about, people they just met maybe, right? And so, It's profoundly important, I think, that we build stuff for this that gets things right. If you look at the way people are communicating right now online, I don't think what we have today really helps people bring out their best selves, to put it lightly. I think the potential is there for this to do that in a very big way in a global scale. I think that the ability for people to be able to have a low-stake interaction with others, where they feel like they're with them, and feel safe and feel like they can be themselves is a profoundly powerful thing. And I think the scary thing about it, though, is that these things will be figured out, I think, in the next several years at most. And then it's going to be very, very hard to change from there. And so I'm hopeful that there's going to be a variety of people working on this in different ways. that in the end, we will see this as a really profoundly critical part of being a human being, being able to connect with people regardless of where they are, regardless of who they are, and feel comfortable and feel like you can be yourself and be a more empathetic person than having to be throwing stuff at each other through the internet verbally, as we, I think, see people doing a lot now. And so just in the small, the evidence I've seen is that this medium, especially if the creator of the experience did a good job, really does help people bring out their best selves. and really makes them feel like a person. And I think that's really, really exciting. But also, you know, there's a lot of potential for harm if it's not done well, so.
[00:36:23.092] Liv Erickson: This is all fundamentally why it's really important that we build this in an open and transparent way. Because I don't think any team can make all of the right calls if they're operating in isolation. And that's one of the things that I feel very strongly about in terms of being open source, being on the web, and being here at Mozilla, working on that project, is that it's something that everybody could use. And it needs to reflect a lot of different people's opinions.
[00:36:52.330] Kent Bye: Great. And finally, what do you each think is the ultimate potential of immersive technologies and what they might be able to enable?
[00:37:03.066] Liv Erickson: I might have covered that with my, you know, this is the whole future of everything about how everyone communicates. For me, it's the power that comes with enabling anybody, regardless of where they are in the world, to be able to be connected to another person and to share very meaningful stories and to share their experience in a way that gives you perspective that you wouldn't already have. I mean, Just speaking for myself, since I joined the VR industry, my worldview has been so dramatically widened by the number of people that I can meet on different platforms and see how they express themselves in different ways. And thinking about how this technology will ultimately empower everybody to be able to do that, it's a little bit hard to kind of wrap my head around sometimes, but I think that that's what it is. It's being able to share and experience stories of being human that we just don't even know about right now because our perspective has been up until this point very limited in terms of what we can understand.
[00:38:03.917] Greg Fodor: I mean, I would be completely satisfied if the most this technology enables is for people to live out at least what I consider my best memories when I was with my friends in my buddy's basement watching TV or at my family reunion, you know, watching my sisters play a board game or whatever. And so, having the ability for just really basic human needs to be fulfilled in a way that isn't coupled with physical co-locality, I think on its own would be a huge achievement for humanity to have access to globally. Now, of course, there's much farther reaching goals you can go from there, right? Like, can this really extend our ability as human beings in a fundamental way? But I think the internet today still hasn't even hit the bar of just the basic human needs we have to be with each other and have social presence and feel connected to one another. If anything, it's arguable that it's almost taken us a step back as much as it's gone forward. I think that that is the goal I have in mind for what would be a really great achievement. Maybe we'll see if it goes further from there. But to me, if that's something that can happen, then that would be just great. I would be so happy if that happened.
[00:39:18.312] John Shaughnessy: It's a difficult question because you use the word ultimate, like, you know, how long into the future are we talking? If you can just beam light into my retinas and maybe you could do that from an early age. Like, I don't know what you could teach me with random spurts of light. I think what these two have said, I also aspire to like, that sounds pretty good. And. I think right now the complexities of the world somewhat exceed our ability to match them. And that's been the case for a long time. Doug Engelbart is this computer scientist who I'm a bit fascinated with who said this for a long, long time. So even with new technology, we often have old ideas in our head of how things work. A related idea here is, like, Brett Victor gave this talk called the Humane Representation of Information. And what kind of representations of information forces humans to do a lot of work in order to understand a concept? So for me, I'm hoping that with virtual reality technology, we're going to have better ways to communicate representations of ideas. that are visual because we have this really great medium for visual thinking and that are spatial or knowledge that's embedded in your body. If you think about like craftsmen over a table working with wood, you sort of learn by doing but also watching very closely and I think virtual spaces with a version of you and a version of me hanging out together, teaching each other and learning and showing each other things will just enable us to learn faster, hopefully learn better than we can right now.
[00:40:56.412] Kent Bye: Great. Is there anything else that's left and said that you'd like to say to the immersive community?
[00:41:01.977] Liv Erickson: Try hubs.com.
[00:41:06.150] John Shaughnessy: People will go to different VR apps for different reasons. Some VR apps are really good for a particular community because they really like to do this one activity that maybe is a little extreme or exaggerated in some fashion, or the app is a game and everybody likes that game. But for a lot of things, what you really want is just to be there with other people. And when that's what they're doing it for, just to get together, think about, is the platform that I choose to do this going to respect me five years from now? Is it going to be the one that I want to see improve and want to see all the communities who are inviting me to events use in the future? If it's going to require a retina scan in order to get in and maybe a blood sample, maybe you don't want that. But wherever the line is for you, maybe I just encourage people to think of it and then try hubs.com.
[00:41:54.545] Greg Fodor: I mean, I guess one thought that might be worth sharing for people who are working in this space is I've come to really believe that the stuff that we're working on and the stuff that is going to end up being what we see is going to be about this fundamental idea of avatar embodiment and shared spatial awareness, right? And so this is already happening in a variety of different media, right? people that have been playing MMORPGs for many years, and they have this kind of sense of social presence they get just from interacting with each other as avatars on a screen. You have these apps that are on your phone now where you're able to use AR overlays over your face, and a lot of those are a bit novel right now, so they're kind of being used as the end in itself. You use the thing to take the picture of yourself with the AR overlay, but over time, I think those will become less novel, and you'll be using them as a fundamental part of your identity, as a way you express yourself, and that's just always there. to some degree, right? You'll be a knob or something like that. And so I think that this detachment of yourself from your physical identity and your geographical position in space is going to be something that is going to incrementally increase over time. Like you're going to be able to represent yourself these virtual contexts in different ways and so ultimately I think we should be designing for the future we want for what that is going to look like for ourselves and our kids like what is going to be the humane place for them to take on these kinds of alternative representations of themselves or take on presence in whatever it ends up being a virtual space or a virtual group of people it's totally undefined right now and And so I think, you know, we have a limited window where we can kind of lay down some of these metaphors for people. And then, as we've seen, you know, like, now that the Internet is everywhere and that there are these global platforms, it becomes very difficult to make big changes to them without breaking society. So that's kind of at top of mind for me. VR is a very important piece of this puzzle. But I think the trend that we're seeing and that I expect to play out is this notion of the detachment of self, the detachment of your physical presence from your actual presence, which is going to be in part physical, but in part virtual. And it's very, very out there sounding, but it's actually already happening today in the concrete terms. So it's just kind of knowing where that's going to go and how it's going to evolve from there. And as they say, the best way to predict the future is to invent it.
[00:44:14.158] Liv Erickson: I think to that point, it's something that everybody can contribute to right now. I've talked to a lot of people who are just getting started in the VR industry in various capacities, and a lot of the times if they don't have a programming background, they're not entirely sure how to contribute or if there's a place for them. And I just want to emphatically say, yes, there are a lot of ways that people can contribute, whether that's getting involved with advocating for privacy and user security on platforms, being vocal about how you want to use these types of platforms and what features you think that we need to be building. creating art, creating worlds and sharing them, creating spaces where you're opening up dialogues about conversations that you want to be having, filing bugs. There's so many things that you can do to contribute to open source projects. That was actually new to me when I first started getting involved with open source is not knowing the entire realm of potential out there for how somebody can get involved. Of course, writing code is a very strong element of that, but there are a lot of other things, too. So I encourage anybody who wants to get involved with a project like this, reach out to us. Let us know that you're interested and what you want to do, because there are so many ways that people can advocate for and help create technology across the entire spectrum of users. And I want to encourage people to get involved if that's something that they care about.
[00:45:31.510] John Shaughnessy: I'd even go on and say there's lots of types of technology people can submit to, right? There's this idea of cultural technology, how to be polite or how to allow a group to feel comfortable changes with where you are in the world and what the cultures and the norms of that space is. You know, you may not realize it, but by attending an event and being yourself and like trying to feel the vibe of the room and act accordingly, it's similar to, you know, before you post on a message board. Well, at least it used to be the case that the polite thing to do is read for a while, to lurk, to kind of understand the norms of that space before just like, you know, spazzing out. And a lot of communities actually enforce this, saying, you know, you're not allowed to write a message in this message board until you have been around for a while. So in these social spaces that are pretty new to us, OK, you could do a voice call a while ago through the computer. But what we're seeing in VR is like this new fascination with this activity. And going in there and figuring out how to set the rules of the space so that everyone can be engaged safely and know what they're getting into and be able to opt out of things that they don't want. I think that's an important type of technology anyone can support and add on to.
[00:46:49.928] Kent Bye: Yeah, I love it. Awesome. Well, I just wanted to thank you all for sitting down and joining me today on the podcast and giving me an update as to all things that are happening on the hubs and all the other initiatives there at Mozilla when it comes to working on this social communication platform. So again, I just want to thank you for joining me today. So thank you.
[00:47:07.401] Greg Fodor: Thanks so much. Thanks. Thanks.
[00:47:10.100] Kent Bye: So that was the Mozilla Hubs team, including Greg Fodor. He's the head of engineering there at Mozilla Hubs. John Shaughnessy is an engineer in the Hubs team, as well as Liv Erickson. She's a product manager on Mozilla Hubs. So I've remembered different takeaways from this interview is that first of all, Well, the thing that really stuck with me was the fact that Mozilla is open sourcing all of this code. Because when you think about, like, can you trust what is happening on these social VR sites, then, you know, you really don't know what's happening on the back end, where the data is going. But in the case of the social hubs, all the code is there, you can look at it, you can audit it. In terms of a standard for transparency to ensure that there's nothing nefarious that's going on, I think that's a pretty good standard. But there's also not only just the code for the actual WebXR implementation of social hubs, but also all the infrastructure code so that if you want to be able to get up and running all this stuff on your own, then you can start to start up your own private instances of these social VR hubs. And in my experience of having different social VR gatherings within social hubs, it's super convenient just because it's got different Slack integrations. You'd be able to type in and it keeps the log of what was ever was typed within these meetings into like the chat windows. But also just the audio is just really cool to see how well WebRTC is working. Really gives you a different sense of actually having a spatial gathering with people and I think that was a big point that they were focusing on which is like this is changing the way that people are communicating with each other as well as collaborating with each other and finding new ways to be able to use these different integrations with either Sketchfab or Google Polly and and to just make it easy for people who don't necessarily have a lot of coding skills to be able to start to search through these different repositories of different models and start to construct these different scenes and you know eventually get to the point where you're starting to create these symbolic reflections of whatever topics being covered you can actually architect an entire environment context to be able to really dive into some of these topics and so Something that Liv said is that as they're starting to meet and gather in these different spaces, they're starting to play with architecting and designing entire spaces that may be able to focus on specific topics or themes to kind of bring out different interaction dynamics within these different group meetings. So I remember way back at the very first Silicon Valley virtuality conference, when Vlad Fasovich, who was at that time working at Mozilla, and he was talking about trying to bring the web to virtual reality. And what's interesting to me is to see how fast things have progressed. It's been a long time that virtual reality experiences on the web were far away from being on the same type of immersive experience. And you would get from a native application, but. I think over the last couple of years, actually, it's gotten to the point where it's really quite comparable, especially when you look at, you know, some of the different social gatherings that they're starting to have here in the hubs interface. And I think being able to actually pull in these different physics libraries and, you know, there's something about the exponential increase of how fast things can go when you develop on the web. you know, there's going to be all these different other aspects are going to be able to start to pull in the resources that from the broader web development community, all these other open source projects and things like these physics engines. And, and I'm, I'm excited personally to see where all the data visualization and being able to, you know, have some sort of spatial representation of knowledge and what I would consider to be somewhat of a, like a memory palace, how to architect that. I think there's actually a lot of work to be done in terms of what the immersive internet is actually going to look like. I had a chance to go to London earlier this year to talk at the Architectural Association about some of the ideas that I had about the future of the immersive internet and what that might start to look like and some of the big questions there. And I did a number of different interviews with architects who are normally designing buildings but are very interested in seeing what type of new opportunities you have within these virtual spaces to start to architect these social dynamics that would be either impossible or too difficult to start to do within reality. And so to me, I think there's a lot of very interesting insights into virtual reality and virtual worlds when it's looked through the architectural lens. So this is still in the early, early days of where this is all going to go. I expect to see a little bit of an exponential growth in this specific realm, especially when the WebXR 2.0 spec finally gets finalized and launched out into the world where you have the entire industry that's really getting behind these open standards. Right now, it's still kind of like in this interim phase where anything that's being built is going to have to eventually be updated. unless you use something like A-Frame that's going to be taking care of that on the back end. And so you have these kind of declarative language approaches where you can still design and construct these different scenes. But once the browser support is on to every single browser, then I think we're going to see a lot more growth and initiative when it comes to where this is all going to go. And I'm just super excited to where the potential is going. I go back to this quote that Philip Rosedale from High Fidelity said, which is Metcalfe's law, which is essentially like the value of a network is equal to, you know, how many nodes are in the network and you take the square of that. So how many other different either websites or people that are connected into these networks, it becomes just more and more valuable. And so. the open web is so valuable because you can just link off into anywhere and start to create these interconnected relationships between pieces of information and help guide people and to point them into other places that are worth looking at. And so once you start to add that into these spatialized worlds, then I'm excited to see where this interconnected metaverse of information is going to end up going. But if you haven't had a chance to check out Mozilla hubs and definitely check it out, uh, the sensory design meetups that happen there are really great one to check it out. Cause I know they've been having some pretty consistent gatherings there, I think at least like once a month or so. And so really talking about some of the variety of different issues that come up when it comes to designing for immersive spaces and spatial computing. The final thing that I just wanted to mention is that there's actually a collaboration between Mozilla and Microsoft, Google, Samsung, and the other members of the W3C. They're actually going to be having this really cool conference in Amsterdam. It's going to be from September 30th to October 1st. I'm actually going to be there recording a live recording of the voices of VR talking about the WebXR and what's happening in the immersive spatial computing on the open web. So I'm excited to be there and to kind of help cover what else is happening there with the future of the open web. If you're interested in attending this view source conference, actually have a discount code. If you're interested in registering, it's all capital letters. It's VR underscore VS 2019. And I'm also actually going to be at the Decentralized Web Camp. Last year, I was at the Decentralized Web Summit, where I did a number of different interviews. I've published about 11 of those different conversations. I actually published a Medium article last August. It's called A Primer on VR and the Blockchain, 20 Big Ideas About the Decentralized Metaverse. And so I'm really excited to see what's happening with this gathering of the Decentralized Camp this year. They're actually going to have this camp out. Previous years, it was just the summit. I think it was like 2016. And then they had the second one last year, 2018. Really fascinating gathering of people bringing together like luminaries from all over the open web from like Tim Berger's lead to vent surf. And I'm not sure who all is going to exactly show up to this decentralized web camp, but it's going to be like three days of camp out and talking about the future architecture of the decentralized web. And so I'm just super excited to be there and help capture and document and really learn. what the future of the decentralized web is going to look like, what are some of the big problems, what are the architectural solutions that they're coming up with, because I do think that it's going to be a huge part of having this counterbalance to a lot of the centralized powers that we have with a lot of these other communication networks. I think it's actually going to be providing a lot of antidotes to the way that the existing internet is set up today and maybe some of the downfalls. And so I think there's still a lot of challenges and in a conversation with Vint Cerf last year, he was really kind of skeptical to see like, you know, how far can you really take the future of these decentralized architectures? So I'm just excited to be there and to listen to the conversation and record a number of different conversations there. And, uh, what's been really happening in that realm. So that's all that I have for today. And I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends and consider becoming a member of the Patreon. This is a listener supported podcast. And so I do rely upon your donations in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
