#1311: Bringing React Native to Enterprise XR App Development with Matt Hargett

I interviewed founder of Rebecker Specialties Matt Hargett at Meta Connect 2023. See more context in the rough transcript below.

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

Rough Transcript

[00:00:05.452] Kent Bye: The Voices of VR podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. It's a podcast that looks at the future of spatial computing. You can support the podcast at patreon.com slash Voices of VR. So this is episode number six of 12 of looking at my series of different interviews from MetaConnect. Today's episode is with Matt Hargett. He's the founder of Rebecker Specialties, and he's been shipping products in Silicon Valley for over 25 years. So after I had a chance to do an interview with Liam Brozo, he was going over to the WebXR meetup that was happening in the midst of MetaConnect. And I wanted to go to that meeting and overhear some of the different discussion, because it was an opportunity to bring together, like, Brandon Jones, who's the spec editor of WebXR and WebGPU, and also on the Chromium team, and also the entire MetaQuest browser team was there, and then also other representatives who were developers within the context of the WebXR community. Diego from Supermedium, you have Tom Emmerich from ETHWALL, and actually a lot of the conversation was being driven by both Liam Broza of Ethereal Engine and also Matt Hargett of Rebecker Specialties. So Matt, as he explains in the course of this interview, had started to write some of these WebXR applications but then found that he was actually running into different blocks where there would be certain regressions that were happening in the context of the web browsers. And so he looked into these alternative pipelines where he's been involved with react native and just very integrated into the community. And so he's collaborating with call stack and this dot labs, and they're using the web XR and open XR specification to create react native code. That's going to be able to compile down until like the apple vision pro. So I'm really super excited to see where this goes in the future. So that's what we're covering on today's episode of the Wastes of VR podcast. So this interview with Matt happened on Thursday, September 28th, 2023 at MetaConnect at Meta's headquarters in Menlo Park, California. So with that, let's go ahead and dive right in.

[00:02:10.510] Matt Hargett: I'm Matt Hargett, and I'm the founder of Rebecca Specialties. And I've been shipping products in Silicon Valley for about 25 years now. And I've been kind of poking around XR since actually the Oculus Kickstarter, which I backed. But also before that, in the late 90s, I had a device called a VFX-1 headset, which was guaranteed to make you sick and give you a headache. But even that early VFX-1 headset planted a seed where I'm like, these trajectories for displays for all this stuff is going to get better and better. And so I've been tracking it at a distance with great interest, but founded a new company just this last year.

[00:02:45.102] Kent Bye: Great. Maybe give a bit more context as to your background and your journey into working with VR.

[00:02:50.586] Matt Hargett: Yeah, my mom was a programmer and a gamer. So starting in 1984, I was programming on a computer called a TI-99-4A in BASIC. And then from there, we got a PC in around 1987. And that's when I got into x86 assembly. And I never stayed in one language, both as a child programmer, but also throughout my career. So some people know me from my talks at C++ conferences, some people know me from the kind of mono and like the .NET community. But pretty consistently just kind of like looking for the new technology opportunities, what kind of efficiencies it unlocks from a product development perspective, but also like a user wow perspective. And starting about two years ago, I started to see these trajectories lining up for XR.

[00:03:36.057] Kent Bye: Yeah, and so we just got done with a meetup that's here at MetaConnect, where it was a lot of folks from the broader WebXR community, representatives from the spec editors of WebGPU with Brandon Jones from Chromium team. We had a whole team from Meta, the browser team, and a little bit of an opportunity to give some feedback. And so I'd love to have you ground us into this context of what was your take on this conversation and how you see this broader aspect of WebXR fitting into the larger ecosystem of the XR industry in general.

[00:04:06.193] Matt Hargett: Yeah, I think to kind of summarize the conversation, there are folks like myself, like Liam from Ethereal Engine and other people that are kind of beyond the novelty part of XR and trying to build kind of high value applications that people return to but that aren't necessarily social, that are just kind of high value for knowledge workers or executives or DoD military or those kinds of folks. And as a result of us trying to build a more scalable business and more scalable teams within the WebXR technology space, we're bumping into some ceilings where it's hard to get efficiencies of scale for developers, because how do you automate tests? Well, that's not commoditized right now. And it was hard in browsers 20 years ago also, where it was the same problem with this browser technology works, you can make awesome apps in it, but how do you have continuous automated testing so that you can scale the rigor of your engineering smartly and not just throw money at the problem with like QA after the fact and sort of stuff like that. And so there's those kinds of things. And then in general, I would say that monetization on WebXR is not impossible, but high friction or very challenging. Yes, you can find people that will type in their credit card number on a VR keyboard, somehow lifting up their headset, looking at their credit card, putting it back down. There's a non-zero number of those people. But even if you're not thinking in kind of venture capital terms of what a successful business means, if you're just trying to cover reasonable salaries for like a team of four people, I don't know that that monetization motion is there yet. That flywheel isn't spinning. And some people can view it as, no, no, it's totally viable for business. And it's like, there is a W3C standard called the Web Payments API that can hook into platform payments like the Edge browser does, or like on Safari, hooks into Apple's platform payments. This kind of can be done, it has been done on mobile, because mobile had the exact same problem, and it was the exact same problem back in the covered wagon days of the 1990s, where commerce on the web was people typing in their secret credit card numbers into strange, brand new websites. us old-timers have kind of seen this cycle before, where it's a little bit of a chicken and egg problem, where it's like, how do you smooth that out as a platform holder but still provide security? How do you do that kind of a smart, intelligent way that protects the users, monetizes your platform in the way that you want it to, but also allows for that level of, even, I say minimal, even as a solo developer that doesn't want to eat SpaghettiOs out of a can, Again, like if you just want to pay yourself like minimum wage, like $38,000 a year or something, how can you even make that size of a third party successful? And I think that WebXR is hitting similar speed bumps that we've seen over the decades, basically, when it comes to just like, yeah, how do you deliver value to the user, but then like not live in like a refrigerator box down by the river at the same time? and I have every confidence that the ecosystem as a whole will get there. One of the things at the end of the meeting was everybody has a lot of principles and values alignment from a technical perspective and from the high-level perspective of third parties need to be successful, period. Not just first-party platform holders or preferred vendors, but the random 14-year-old that makes something that strikes gold, some random idea that nobody ever would have thought of, and then they go on to success. And this is what it was like with 8-bit micros in the 80s with the bedroom coders on the ZX Spectrum and the Commodore 64 and that kind of stuff. And I don't think that can quite happen with WebXR yet, where you can get maybe a lot of attention for a bit. And if you can monetize that via socials or other ways, but that's not directly, wow, I really liked your thing, let me give you $5. So I think that that was a kind of a general topic of how do we have even a minimally repeatable business, nevermind scalable business, because scalable means different things to different people. But I think that across Google and Meta and independent third parties, like Rebecca Specialties, my company, and at Ethereal and Wonderland, and the Babylon team, the folks at Microsoft, We have, I think, all the right brain trust and all the people kind of blowing into the same sail and trying to figure out how can we divide this work up amongst ourselves to move the open standards forward and the implementation of those open standards and the general open source ecosystem forward so that these barriers to entry for building WebXR apps are lowered so that people can just try stuff. more easily, but then also like the startup cost of how much venture funding do you need to do your WebXR thing? Less and less and less over time. So I think that that's already happening. It is getting cheaper and cheaper. But when it comes to things like, yeah, we kind of need a headless WebXR browser so that we can run our automated tests in a GitHub action. That's a big gap, for instance, right now.

[00:09:25.030] Kent Bye: Yeah, and speaking of that headless browser, if there was an open source release of a lot of the code that Meta has been working on with the Atlas browser, that would perhaps enable all those things, right?

[00:09:34.118] Matt Hargett: In general, yeah, and so, you know, I think that the kind of Immersive Web plugin that I think Mozilla started and now Felix here at Meta now sort of maintains, gets us closer, right? We can run kind of a headless browser that uses the Immersive Web plugin and get some feedback loop there, like stupid obvious bugs without ever putting on a headset. That's awesome. But we need to get to like the next level of safety net in terms of early bug identification or sort of catching. And there might be other ways to do it. It's possible that the Immersive Web plugin itself could be enhanced or leveled up to be able to catch that next level of bugs. But, yeah, there is a, I think, a little bit of a challenge right now where, because of the lack of source availability, the community can't self-organize around this, like the way that Puppeteer does for Headless Chrome, or the way that, which is a third-party thing, somebody in open source just decided to do that seven or eight years ago. So because the source isn't available, even partially, it's harder for people that have been capitalized and staffed, where they need to solve the problem so they can scale their product teams, so they're interested and committed to solve that kind of problem. Right now there just isn't as much opportunity as maybe there could be before you have to kind of go on-device, which is kind of the ultimate way to test your app, right? And you kind of need to get there, but if you can find easy bugs fast, much, much cheaper, and run thousands of tests in a cheap Linux container somewhere that only has like 128 megabytes of RAM, That's where, from my perspective, from a team building, from a company building perspective, where it's like, could I hire more than 10 engineers and have them be effective? How quickly does that productivity thing trail off when the number of collaborators or committers gets above a certain number? And right now, my personal feeling, my personal Matt Hargat opinion on this is, This is the thing where I'm like, yeah, I don't think I would try to do that with more than 10 people without major capitalization, where it's like, we'll just spend, you know, $8 million in QA, who cares? But if you're trying to run kind of a sort of a lean business and self-finance, and therefore you're extremely incentivized to be even smarter with your money and more fiscally responsible than you might be, I think it's a challenge. And it's possible that a reference implementation of a WebXR Chrome-based browser would help. It's also possible that there might be some more relatively low-hanging fruit in the Immersive Web Simulator as well. In the Immersive Web Simulator, even for manual testing for the prototyping phase that I was doing with my partner, this .labs, from December until about March, Those developers use the Immersive Web Simulator a lot, manually, and it saved them so much time from going in and out of headset. Even with the HoloLens, where you don't need to take it off necessarily, it was just the iteration time got that much faster, and even at a prototyping phase, probably gave us like a 30 or 40% uplift in terms of total features we got done, just because that inner developer loop was that much cheaper and that much tighter. So it's possible that the Immersive Web Simulator could be leveled up in some way, I don't have specific ideas, but I think that looking at every time there's a bug, how could we have caught it earlier? How could we have caught it cheaper? It might be that we don't need an open source reference browser for Chromium from Meta or Qualcomm or some other stakeholder. It might be that, no, for the kinds of bugs that people are running into, we can get that done just with some extra little features in the Immersive Web Simulator. I think that that's entirely possible. So I don't know that the lack of an open source reference browser for XR is a showstopper for scaling at this point. they might eventually sort of get there. But I think that there's options. And I think that as long as everyone in the community who are trying to build a very valuable, scalable product and scalable company and teams, as long as we all keep talking to each other and sharing high candor information about the challenges that we're having, communicating what we need to the vendors and stakeholders with one voice.

[00:13:52.448] Kent Bye: Yeah, I wanted to bring up a couple other points that were brought up. One is that the idea of a progressive web app, I guess the ideal of a progressive web app is you go to a website and you click a button, and now all of a sudden it's basically treated as a native application. And that the way that the implementation at Meta that they've chosen is that it has to go through the store. So you can't just make that. direct translation from the browser and create something as a native app. But if it were possible to do a progressive web app and put it in the store and have all the other aspects of the permissions, because that's another thing when you open up some web browsers, then there's additional friction in terms of having to authorize and authenticate. And so there's a lot of workflows in terms of like, Sending it to your phone and then verifying it's the right domain and then opening it directly in the browser So there's lots of like onboarding optimizations that were talked about but what I was also fascinated about is this is it possible to do a Rendering out into a progressive web app that could then be sold in a store which could be another monetization option and then the other thing that you had mentioned that I was really excited to hear is folks talking about is react native and being able to actually take this web stack and to render out a native application that has all the Native hooks and implementations so that you could be highly optimized and could also be a potential way to have an application put in the store That's not in the traditional Unreal Engine or unity stack and especially in the context of all the runtime fees that John Rick Attella dumped on people and walked back But it created this motivation to create alternative pipelines to be able to produce WebEx our experiences and I feel like react native given that meta has such a a a robust experience with not only helping to create it, but to really shepherd it as an open source project, but have other folks like Microsoft have the capability to take that and then make it both native apps for both Mac and PC. So there's a lot of potential there on the React Native side that's also exciting. So I'd love to hear any of your thoughts on both the Progressive Web App and also what you've been looking at in terms of React Native as an alternative pipeline for producing WebXR experiences.

[00:15:50.157] Matt Hargett: Yeah, like I said, we prototyped with my partner, this.labs, we built a WebXR prototype. And part of it was, what can this do? And really, for me, the existence proof for WebXR, like the standard works and whatever, was Mozilla Hubs. I joined a Mozilla Hubs meetup. And, you know, it worked. And that's all open source, all open standards. And I was like, this is the inflection point. This is the existence proof I was waiting for. So, when I saw that, and we're like, okay, this is awesome, because why would you target WebXR? It's like, well, any new device that comes out, if they have a WebXR-compliant browser, your app's there, assuming you can monetize it. Maybe you don't care about monetization, right? But it's kind of that forward, future-compatible sort of thing, just like there's HTML.1.0 websites from 1990, whatever, that might still render in Firefox if they're still well-formed and stuff like that. So that's really attractive both from a preservation perspective, like a software or game or whatever preservation perspective, but also from a business perspective where it's like, I'm going to put this thing out there and wait for the hockey stick to happen. If it doesn't happen, whatever. But if it does happen, that's great. But if it doesn't happen, I probably don't need to keep going re-updating that experience. It'll probably stay forward compatible. So we found like testing on the Oculus Go, there were WebXR experiences that worked in Oculus Go that worked fine on Quest 2, that more or less work okay on HoloLens, that also work on the Pico. That's really attractive in terms of ongoing engineering costs, like thinking fiscally, right? Then the next question is, how do we monetize, and can we get store placement? I'm trying to figure out how to be fair to everybody. So there was this, I forget what it's called, there's a PWA builder, which is a project from Microsoft, and when they announced PWA builder, they announced that you could do PWA builder for Mac, Windows, iOS, and the quests, and it's like, hot damn, let's go. You know, like, that's great, because Electron builds, it's a barrier to entry, it's not insurmountable, people do it, but, like, this is great, but then come to find out, oh yeah, we announced that, but we aren't actually doing that on the PWA builder side. And it just turned out that that's not something that, for some of these platform holders, they're kind of app ingestion pipeline, how do people submit apps, how do you, as automated as possible before it gets to manual review, funnel it through, and it just turned out that there was a lot of work to do on that app ingestion pipeline. And I, as having worked for platform holders, I sympathize, empathize. I'm like, okay, that was harder than you thought it was gonna be, and so no PWAs. Or if there are PWAs, they are manually one-off things that you do. And PWAs don't have to be immersive. Slack. an Electron app, right? Where it's like, I want to have my Slack up and be looking at a 3D video or whatever, right? But like, not even that is a paved road quite yet for first parties or third parties. So that was also kind of made me go, okay, WebXR standard works. There's a lot of things to like about this, and we had a great prototype. But then, around the end of March, we had a pretty, on HoloLens, Edge browser, hands tracking not only stopped working, but it turned out that hands tracking is no longer compiled into the Edge binary. It disappeared. And we could not get any response or motion on that whatsoever.

[00:19:24.364] Kent Bye: Yeah, Brandon Jones said that there's a lot of layoffs that that team was impacted.

[00:19:28.448] Matt Hargett: There's a general economic contraction. And in 2007-ish, when there was an economic contraction, a lot of evergreen products, like browsers, had a rough couple of years due to staffing volatility. It happens, right? So it's not one company. It's not one whatever. I would anticipate that there will be a little hiccup in a lot of these. ecosystems that are kind of running pretty smoothly because all the smart, talented people are gainfully employed to do good public works effectively. So I would expect other kind of hiccups to happen, but yeah, the staffing volatility slash layoffs across multiple companies makes it a little bit challenging. And then there were some other WebXR browsers that also just had regressions. And when I kind of went back through the WebXR Discord history, Like, how often is this happening? And it was like, about once a month. And what's cool is that those vendors are present on the WebXR Discord, they're responsive, they go, yep, I can reproduce it, and then often, not always, this is not a commitment of SLA, but like, two weeks, three weeks, there's a new build, it fixes the problem, great. That's really cool, but they find out about the problem because we tell them. And to me, trying to build a business that's trying to sell into enterprises, DOD, military, XR is not a novelty thing. It's not like, as Frank Zappa said, the world's finest optional entertainment. It's solving a very specific problem. It's sold for more than five bucks or 20 bucks or whatever. There's higher expectations in terms of level of service there. And I just personally, with my own sort of risk appetite, was like, I don't think I can ship this to enterprise and military customers on WebXR browsers today. But up to that point, we had a great prototype and a great demo. The tech was all, that was all working. Then it just came to like thinking about support costs and how high touch was it going to be. Can I afford to hire three support people just to deal with, hey, the thing stopped working all of a sudden. They're like, did your browser update or what? Yes. Damn it. And then because it becomes interrupt driven and you can't plan around that. Some people can absorb that. I felt like that wasn't something I could absorb. So, before this WebXR thing, across multiple companies, I had brought React Native to other platforms with partners. So, at BlueJeans, we brought React Native to Windows 7 and 8, which is where all of BlueJeans Enterprise customers were at that time. on those operating systems, and I did that in collaboration directly with Microsoft, committing directly into github.com slash react-native-windows, and the outsource engineering vendor on that was Infant Red, and they were great because they had the inroads into the React Native community at that time. That was highly effective for everybody, and that started a flywheel inside of Microsoft around React Native, because we had an enterprise shipping app, great public-private collaboration and sort of it was all open source, but everybody still made out really well. And in terms of engineering efficiency and cross-platform code, it was great for BlueJeans. And then at my next job, we brought React Native to game consoles as well, up to 50-plus apps, and that was like a 500-person team across geographies. And then at my last job at Roblox, and I just did a talk about this at React Native EU conference in Poland, we ported React Native into Lua and Roblox's game engine to get really fast lists and a bunch of other kind of stuff. So that was a through line, but I wasn't planning on using React Native again necessarily, but when I reached this kind of point with WebXR, and I was like, it's going to get there, it's going to be stable, I have every confidence about that, it's just not right now, but I'm starting my company right now, and what React Native kind of allows me to do is to write a lot of code that we can write to a WebXR API and a WebGPU API. and basically rebundle it for web later when that is in a spot where I feel like my target customers and the stability of those platforms will be at a point where they converge with expectations and stability and sort of capabilities. Awesome.

[00:23:43.689] Kent Bye: Well, I have to run off to my next event, but super excited to hear that there is at least somebody out in the world doing a React Native pipeline to do WebXR, WebGPU, because I feel like as people are looking for alternative pipelines for production, there's a lot of excitement for something that's maybe an open standards-based web stack that can actually develop native quality code. So excited to see where that goes. So thanks again for taking the time to help explain your journey and a bit about what you're working on. So thank you.

[00:24:06.816] Matt Hargett: Yeah, thanks, Kent. And I'm super excited to be working with the Babylon team and my outsource partner at Callstack and this .Labs. I think we're going to be closing a lot of gaps, not just from delivering my product, but closing the gaps in open source for a lot of people that are trying to develop XR applications.

[00:24:21.990] Kent Bye: Awesome. Well, thank you so much.

[00:24:23.271] Matt Hargett: You're welcome. Thanks.

[00:24:24.521] Kent Bye: So thanks again for tuning in to one of my dozen episodes about MetaConnect. There's lots that I've been unpacking throughout the course of the series, and I'm going to invite folks over to patreon.com to be able to join in to support my work that I've been doing here as an independent journalist trying to sustain this work. Realistically, I need to be at around $4,000 a month to be at a level of financial stability. I'm at around 30% of that goal. So I'd love for folks to be able to join in, and I'm hoping to expand out different offerings and events over the next year, starting with more unpacking of my coverage from Venice Immersive, where I've just posted 34 different interviews from over 30 hours of coverage. And I've already given a talk this week unpacking a little bit more my ideas about experiential design and immersive storytelling. And yeah, I feel like there's a need for independent journalism and independent research and just the type of coverage that i'm able to do and if you're able to join in on the patreon five dollars a month it's a great level to be able to help support and sustain it but if you can afford more than 10 20 50 or even 100 a month are all great levels as well and will help me to continue to bring not only you this coverage, but also the broader XR industry. I now have transcripts on all the different interviews on the podcast on Voices of VR and in the process of adding categories as well into 1,317 interviews now that have been published after this series has concluded. So yeah, join me over on Patreon and we can start to explore the many different potentialities of virtual and augmented and mixed reality at patreon.com slash Voices of VR. Thanks for listening.

More from this show