#1113: Matthew Ball’s “The Metaverse” Book is the Most Comprehensive Description of the Metaverse Stack

Matthew Ball’s “The Metaverse: And How It Will Revolutionize Everything” book (launching today) is the most comprehensive articulation of the Metaverse technology stack, as well as it’s foundational principles catalyzing a paradigm shift from 2D into 3D into spatial and immersive computing. What started as an essay on Fortnite (Feb 5, 2019), then evolved into an essay on the Metaverse (Jan 13, 2020), then evolved into a series of essays with his Metaverse Primer (Jun 28-29, 2021), and now has tripled in size with the publication of Ball’s The Metaverse book (Jul 19, 2022) that dives into specifying each layer of the Metaverse tech stack, where it’s at now, and where he expects it might go in the future.

[See the full transcript of my podcast interview with Ball down below].

I find Ball’s definition of the Metaverse on page 29 of his book to be the most complete and satisfying I’ve seen so far.

[The Metaverse is] “a massively scaled and interoperable network of real-time rendered 3D virtual worlds that can be experienced synchronously and persistently by an effectively unlimited number of users with an individual sense of presence, and with continuity of data, such as identity, history, entitlements, objects, communications, and payments.”

He spends the middle section of the book elaborating in great detail the low-level details of each layer of the Metaverse stack including networking, computing, virtual world engines, interoperability, hardware, payment rails, and the potential opportunities and perils of the blockchain. This middle section was a lot of wonky technical details to sort through, but it’s necessary to understand the bigger picture and larger argument that Ball is making in his book.

Ball sees that the aspirations of real-time, spatial computing and immersive 3D virtual worlds are pushing the technological, cutting edge of each layer of the Metaverse stack. He lays out in the middle part of the book how there are many yet-to-be solved challenges that are preventing the Metaverse from living into it’s fullest potential.

The Metaverse will be a confluence of all of these foundational elements coming together, which will then have the potential to usher in a complete paradigm shift of computing. But we won’t know what that will eventually look like as it’s an iterative and emergent process. He points to paradigmatic examples such as Fortnite, Minecraft, Roblox, EVE Online, and Microsoft’s Flight Simulator to give us some ideas of how these underlying principles have been playing out in a gaming context, but there’s many more unknowns than knowns for how these underlying principles will get translated into other contextual domains.

Ball elaborates in the first section how Metaverse faces many of the similar challenges that tech futurists faced at the advent of the World Wide Web in trying to predict the full scope of their influence on all aspects of how the new technological, cultural, legal, and economic dynamics all fit together. Ball documents many of the wrong predictions of the Internet and WWW that we can look back on and laugh at with 20/20 hindsight, but this makes it clear for how we’re still well within a similar occluded zone with the Metaverse for how it’s really difficult to know how all of these new capabilities will combine together and impact society.

He includes a brief survey at the end of his book of some of the ethical and moral challenges of the Metaverse, and Ball says that the open questions on ethics is a common open question that often comes up. We’re still in the process of trying to understand and deal with the many ethical fallouts of the 2D and mobile web focusing on text, pictures, and video, let alone how the movement from 2D to 3D and asynchronous to synchronous creates some new and even more challenging problems.

There’s still obviously lots of work to be done in the realm of ethics for the Metaverse, but there’s also existing prior work worth mentioning like Madary and Metzinger’s “Real Virtuality: A Code of Ethical Conduct,” XR Access & XR Association’s XR Accessibility GitHub, XRSI’s Ethical Research and Standards, the IEEE Global Initiative on the Ethics of Extended Reality published a series of eight white papers [published in 2022 after Ball’s 2021 writing deadline], and I’ve given a number talks on XR Ethics, XR Privacy, the need for robust responsible innovation frameworks, and the need for Ethics within the Metaverse. [Update: Philosopher Evan Selinger outlines more XR ethical issues in his book review.]

My personal process of peering into the future has been to focus on the direct, embodied, phenomenological and subject experiences of VR and AR applications across many contextual domains for the past 8 years, and to listen to key developers through oral history interviews. My episode #1000 of the Voices of VR podcast elaborates why I’m personally all-in on spatial computing as a foundational paradigm shift as I gathered 120 answers to the ultimate potential of virtual reality since May 2014.

Ball takes a much more objective, distant, and quantitative approach in telling the story of the Metaverse through the economic, cultural, and gaming trends that he’s been tracking. He takes the view that the Metaverse is hardware agnostic, and so it’s not predicated on specific VR or AR technologies. But he also expects that XR devices and the emerging, embodied neural interfaces that are coming along with it will likely be the most compelling ways to access these real-time, interconnected virtual worlds to fully appreciate the new levels of immersion and presence.

I’ve seen a lot of wrong and overly optimistic market predictions about the project size of the VR and AR market over the past eight years, and so I suppose that’s one of the reasons why I’ve paid more attention to the direct experience of XR technologies and listening to what the artists and pioneering XR developers across multiple domains are doing with the tech. There’s not a lot of Ball’s own direct experiences of the Metaverse within his book, or original on-the-record reporting for the companies or experiences he’s featuring. Ball told me that’s because he wanted to show how these broader trends of the Metaverse are happening independent of any one singular company or technology stack. It’s more of a survey and synthesis of publicly-available reports and tech insights of the gaming community in how Fortnite, Minecraft, Roblox, EVE Online, and Microsoft’s Flight Simulator hold some key insights for the future of computing.

The end result is that Ball’s Metaverse book helps to set a broader context for many of the same technological shifts and trends above and beyond what’s happening in XR. Despite my more narrow focus on XR, Ball and I come to many similar conclusions for how the shift towards real-time, interactive virtual worlds is a provocation for the next phase of computing as seen in the major trends of social gaming and technological roadmap for some of the biggest Big Tech players in Silicon Valley. There’s also a clear convergence for what’s happening in the most popular virtual world platforms that are currently primarily accessed through 2D portals, and how XR tech will be some of the most compelling ways to interface with these 3D virtual worlds.

I’d highly recommend checking out Ball’s book to hear more about the specific examples and underlying principles that he lays out for each layer of the Metaverse tech stack. Ball’s writing is helping to tell the broader story of the Metaverse to leaders of the tech industry, but helping bring a more grounded and clear definition of the Metaverse reflecting what’s already happening in gaming right now and where it’s likely to go in the future. It also helps set a broader context for how the future of networking, computing, virtual world engines, interoperability, hardware, payment rails, and potentially the blockchain will play into this Metaverse ecosystem that will very likely include XR devices and new neural HCI input devices that help make the transition from 2D to 3D.

I had a chance to do a half-hour interview with Ball on Tuesday, July 5th, 2022, and you can read the full transcript below or listen to it here.


This is a listener-supported podcast through the Voices of VR Patreon.


MATTHEW BALL: My name is Matthew Ball. I’m the author of “The Metaverse: And How It Will Revolutionize Everything”, which comes out on July 19th. But I wear a different set of hats. That is, an investor, an entrepreneur, a producer in TV, film and video games, usually in and around this theme, but sometimes in more traditional media, and so forth.

KENT BYE: I read an interview you did with SIGGRAPH that traces the evolution of this as an idea where you had some essays about Fortnite and then you moved into the Metaverse, and then did a 35,000 word series of essays on the Metaverse. And then now you’ve tripled that into over 100,000 words of the Metaverse. And so maybe you could give a bit more context as to how you started to write some of these series of essays on the Metaverse.

BALL: Sure. So you’re quite right. I talk about in my book the fact that the Metaverse is a nearly 30-year old term. Of course, anyone who’s listening to this podcast know it originated in 1992. Snow Crash. But the ideas it describes can be traced back nearly a century. That starts in the 1930s, with the first known discussion around VR goggles into immersive VR environments, A.I., Holography through the thirties and fifties.

And so it’s not new that we’ve considered the Metaverse. I’ve been familiar with the term since the late nineties, and I’ve known about games that aspired to build it since around that same time. But my personal experience relates to my last job, which was head of strategy at Amazon Studios, which at the time ran nearly everything that we think of Prime Video to be.

And for much of the last decade, that was the frontier. The new frontier in media was direct to consumer streaming services. And I started to get the sense that gaming really was ready for prime time. Of course, it had been growing slowly for 50 years, but it felt like it was on the cusp of cultural domination that really didn’t even seem possible at the start of the last decade.

And then in 2018, I started playing a lot of Fortnite, a lot of Roblox, and I could start to get the sense that it wasn’t just that gaming was moving to the front lines. It’s this fantastical idea of the Metaverse was starting to become a practical opportunity and that some had actually started to build it, that the foundation was in place.

And so I wrote a piece at the end of 2018 called Fortnite is the Future, but Not for the Reasons You Think. And the goal of the piece was to break down many of the narratives about Fortnite at the time that were running, the day that was. “Wow, I can’t believe this game is free. And yet it generates more revenue than any other.” “Wow. Look at this game’s cross-platform nature. Wow. Look at the idea that this is being patched on a weekly basis.”

And I was trying to articulate that none of these things were new, actually. What was new was their creative implementation, the enormity of their popularity. But more important was how it was leading up to the Metaverse and what was starting to become a visible Metaverse strategy at Epic, though, they hadn’t said as much.

A year later, I wrote a dedicated Metaverse piece and this was pre-pandemic but became quite popular as the pandemic began to take off. Then a year later, I wrote The Metaverse Primer, which was an effort to really encapsulate my learnings over the past year and a half about the technical requirements. And then, of course, in 2021, we saw the Metaverse popularized as a theme.

Mark starts talking about it in July and October. He renames the company by the end of the year. Unity and Roblox are the two largest gaming IPO’s ever. Roblox is the largest gaming platform in the world. And so I decided to write this book, which really culminates five years of writing and thinking on the topic.

BYE: Yeah, I just had a chance to finish it and I think it’s a pretty authoritative history assessing all the different confluence and and concrescence of all these technologies coming together. And what I really appreciated about the book was pointing towards different examples from each of these different sections that are paradigmatic examples that can extract the philosophical principles from those examples, extrapolate them out as they continue to diffuse out into larger and larger scales and out throughout the culture.

But before we start to dive more into the book, I wanted to get a bit more context as to your background and your journey into this, because your style is very interesting in your writing because I get the sense that through the process of writing your essays — I know you’ve had conversations with people like Mark Zuckerberg, and I’m not sure if you’ve been in conversations directly with Tim Sweeney, but you’re certainly well-informed, and at the center of a lot of it.

But your writing style is very distant in the sense where I’m not seeing any direct quotes. You’re using quotes from other news articles, so I’d love to hear a little bit more about your process for how you put this stuff together. And if you’re doing consulting with some of these companies behind the scenes? Or if it’s just more of your role as an analyst to be able to immerse yourself to the extent that you do and then try to gather all the publicly-available information to draw out the larger economic story that makes this argument that you’re seeing all these different confluence of technologies come together?

So I’d love to hear a little bit more about your process for how you do this sensemaking process of what’s happening in the realm of technology and where it’s all going.

BALL: So you’re right. In the book, I don’t use any direct quotations, and in fact that was partly a reflection of the constrained writing environment. I wrote the book over three and a half months. At the end of last year. There really wasn’t the opportunity to go deep, to do investigatory pieces, to understand the multi-year history of many of its leading individuals.

And so that was partly a constraint. But the goal here really wasn’t to get deep into any one company. In fact, one of the things that I take personally a lot of happiness from is you’ll note that there’s oppositional or competing endorsements for the book. So I have Epic CEO and Unity CEO. I have the Sony CEO. And then I have the Microsoft Gaming CEO.

And that was because my goal with the book was not to talk about specific instantiation of the Metaverse, not a specific philosophy or ideology, wasn’t to go into the history of Tim Sweeney’s efforts to build the Metaverse nor those of Second Life. It was to provide a survey as to the technologies in the multiple different theses. I got one piece of feedback from the CEO of a large tech company saying, I loved your blockchain section, but I came away not sure whether or not you were pro or con.

And I said, “That’s the best compliment I can imagine.” Because I wasn’t trying to advocate for this. So when it comes to your question as to the writing process, most of the education came from entrepreneurs and founders. I have a venture fund. I’m a partner at MAKERS Fund, I’m an industry advisor at KKR. And so I do early stage investing, kind of mid-stage and then late stage growth equity investing.

And those entrepreneurs are outstanding because they have 20 years of experience. Sometimes they’re disgruntled, sometimes they feel like they wasted ten years of their life trying to solve a problem at a big company that was incapable of doing it. And so there’s this bounty of information, but I think I’ve always been good at doing at distance summaries of the marketplace.

And I see this very much as the manifestation of five years of work on theme, but 15 years of writing as a blogger.

BYE: Yeah, yeah. I think my assessment of the overall book is that you get the overall story completely nailed and correct — in terms of my own assessment. And I really actually did appreciate your critiques of the blockchain because, you know, there’s things that I come to a lot of similar conclusions in terms of like there’s potential for some of it, but there’s also a lot of challenges and risks and other problems that you elaborate in your book.

But I guess one of the other differences that I’d say from how I approach covering the industry is much more direct, phenomenological, immersed into the embodied experience of a lot of these technologies. And I had a harder sense of seeing where you’re oriented when it comes to the direct experience of virtual reality or the direct experience of AR.

Because you talk a lot about Fortnite, and Roblox, and Minecraft and EVE Online as these kind of leading indicators and maybe just one mention of VRChat or Rec Room, which, you know, for me, when I think about the future of the Metaverse and immersive technologies, I think about my own experiences I’ve had in Rec Room and VRChat and the qualities of presence.

And in your definition of the Metaverse, you mentioned presence, but I wouldn’t say there’s a deep elaboration of the concepts of presence within this book, so I’d love to get a little bit more context for you of like your own journey into VR and how you assess the roles of VR and AR technologies into the continued evolution of the Metaverse.

BALL: So I love this question. I really like VR. I get super excited about AR. There are many people in my network who just believe that this technology is so far off that it’s actually absurd that we spend much time focusing on it. I certainly think the technologies are hard, the problems are hard, they’re worth solving. But I am also of two perspectives.

One is that they’re not neigh, which is to say, I don’t think that we’re going to be replacing our smartphones within this decade. I don’t think we’re going to be doing it probably within the first few years of the next decade. The second thing is, I don’t believe that the requirement for the Metaverse there was this great Tweet Thread I’m sure you saw from Neil Stephenson last week or the week before.

And he talks about the idea that when he wrote Snow Crash centering around AR and VR were what he called a good hypothesis, especially if you were a science fiction author at the time. And he said what he couldn’t have imagined was that decades later, you would have billions of people inside 3D render real time environments interacting purely via touch or even WASD, right? Keys on a keyboard for forward, back, right and left.

That would have been an unintuitive estimate. And so I think about VR and AR devices as more intuitive as being essential to further immersion in the Metaverse. They’re doubtlessly destined to become the best, most popular and preferred way to interact with these environments. But I don’t think there are requirements. And in fact, and this is probably one of the reasons why you see them as relatively non-focal is I think that separating the relevance of 3D simulation in the Metaverse from that hardware helps to explain to many people that the Metaverse is not 2050.

The Metaverse doesn’t need you to believe we’re going to replace our smartphone tomorrow. It’s actually about the more underlying technologies, a persistent virtual network that is relatively endpoint agnostic doesn’t mean you’re not going to have different end points with different experiences. My personal experience with VR, I mean at Makers Fund, we’re early investors in VRChat. I use it a lot.

I find it really fun. I’m really lucky that I don’t experience nausea. And I’m also, I think, fortunate in the sense that I just have a more intuitive sense and feel for virtual immersion. And to the extent I might be predisposed to nausea, I think my mental expectations help. But I think it’s an incredible environment for young people.

When you like some old, but like I’m no ranting because I get excited about this topic. But you know, Chris Dixon talks about the fact that when a person realizes that their active brain can’t overcome what they know to be a fantasy in VR, it’s a pivotal moment, right? You put someone at the edge of the cliff in VR, and even though it’s low risk, even though it’s low frame rate, they struggle to jump.

That’s the first time you can tell. This technology may be far from prime time, but it’s not far from substantial immersion that deeply affects us. The second is when you give it to children and they have this natural, instinctive feel, you see this a lot in Rec Room and VRChat where to some extent I still know I’m in VR or when I’m in VR.

But you look at a six year old and they’re just playing around and like that idea that like they’re turning their head to the right and it’s not a physical thing they’re looking at. That boundary just seems totally diffused at this point.

BYE: Yeah. Yeah, I think that makes total sense and also really agree with that because I know Tony Parisi came up with The Seven Principles of the Metaverse, and one of it was that it’s hardware agnostic. And I think another big point that’s made in your book is how Fortnite was catalyzing this cross-platform play that both Fortnite, Minecraft and Roblox are available and all these different platforms. And so Rec Room is another one that I think is taking that same path of being on all these different platforms.

So I think that’s actually a key part to not just tie it to the VR. I’m wondering if you can maybe — I’ve watched the video where you just rattle off your authoritative definition. I’m wondering if you could just share that definition and then how you’ve structured the book, because you’re breaking down each the different chapters going into great detail of your definition. And so I’d love to have you share that and I have some thoughts.

BALL: Sure. So I actually cheat a little bit. I describe the Metaverse primarily around technical keywords and concepts. I’m not actually defining it. The reason why I do this is if you take a look at the definition of the Internet, you’re talking about it either as a network of networks. Again, a little bit of a description, but you talk about it as the Internet protocol suite, not a very helpful definition.

It’s more technical. It’s talking about the protocols. A description of the Internet tends to be more intuitive. So I describe the Metaverse as a massively-scaled and interoperable network of real-time, rendered, 3D virtual worlds, which can be experienced synchronously and persistently by an effectively-unlimited number of users, each afforded an individual sense presence while supporting continuity of data such as communications, identity, history, entitlements, objects, and so forth.

What we’re really doing is describing the technical requirements and experiential observations to have a proper parallel plane of existence. Right? That’s essentially describing the things that exist in the real world today. And so the middle third of my book is a deep dive into what’s required for that to work. We’re talking about networking capabilities, computing requirements, the actual tools to create and render virtual environments, interoperable standards to exchange information coherently, comprehensively and securely.

Talking about the payment systems that are required to actually build a thriving Metaverse, not just a functional one. An examination into blockchain, and why many believe it’s essential. Others believe it’s useless in all applications. And what potential middle grounds might look like? That’s the middle third of the book. It’s building the Metaverse. The first third is focused on “Why now”? What is it?

Explaining that definition in great detail, but also getting into some of the fundamental questions as to why do people believe that there’s a war here? Why is it important? Who wins? And then why — and this was one of the most fun things for me to examine — does it seem that the forerunners of a multitrillion-dollar transformation are gaming companies, which have otherwise been a relatively-trivial part of the economy focused just on consumer leisure?

And then the last third of the book is the more speculative elements. Start to talk about what might the value of the Metaverse be, who might win, which technologies are likely to prevail, what sorts of businesses will be built? And then for consumers concerned about what the future might be, how do they get involved, what should they do?

And then lastly, I should note I finish by trying to remind people what you can and can’t know about the Metaverse today.

BYE: My experience of reading each of those chapters was that there was a lot of really low-level technical details, but as you get through it all, the big picture all comes together. I think as you see the confluence, just like you talk about near the end in terms of the iPhone moment of the iPhone and concrescence of all these technologies coming together. And so I think similarly, you’re telling the story of all of these things that are coming together right now, and why it’s important.

Now, a couple of things I wanted to ask. One is that I know that the challenge of writing a book is that there’s always stuff that happens afterwards. And just a couple of weeks ago, there was the Khronos Group in the Metaverse Standards Forum that had this big coalition.

In the book, you’re talking about how it’s very difficult to have people collaborate. And I guess when I was reading those, I was like, “Yeah, but they have already collaborated on OpenXR. They’re already are a lot of these interoperability efforts. There’s already open standards in terms of object formats, in terms of glTF, and there’s USD.”

So I’m just curious what your reaction is in terms of some of this latest news from the Khronos Group and the Metaverse Standards Forum. And if that supports a lot of this larger thesis that you have and the difficulties of coming up with those standards of interoperability? And if you have any additional thoughts that you didn’t have time to include in your book because it wasn’t even created yet?

BALL: So I think I’m very optimistic in the book about the establishment of interoperable standards. It’s the single biggest area of pushback I receive, which is do you actually believe this can happen? And so I do. And I believe that the primitives or pressures or gravity of expanded networks will deliver. In that regard. The Metaverse Standards Forum is an important first step.

At the same time, I’d say the following the easiest part about establishing standards is always getting a bunch of people in a room together. We’ve seen that numerous times. You’re right, we have OpenXR and WebXR. But very few people support them and more importantly, their antecedents OpenGL and WebGL are very rarely supported. None of the major consoles support them, for example.

Actually, Oculus is the most used console to use these. You’ll also find that there are multiple other standards forums of some way, shape or form in some regard. Khronos exists to do that, but then you have the WesXR and Metaverse Standards Group. You’ve also got the XR Association. There are multiple different groups. Most people sign up for them because they would rather be heard than not heard and they would rather shape the standards than have their competitors shape them.

I use this XKCD comic in the book that basically jokes that a bunch of people get together to say There are 14 standards. We should have just one. That’s how you end up with 15 standards. And so that’s true. But I’m still hopeful, right? You have to have communication to actually end up aligning on something. At the same point, we can see some important omissions from the Metaverse Standards Group.

Most obviously Google and Apple, neither are participating in the forum. But I think more notable is the fact that there aren’t really other application or content-layer members. Epic is in there and Meta’s in there, but Activision isn’t in there. EA’s not in there. Roblox isn’t in there, Ubisoft isn’t in there. And so what ends up happening is lots of technologists can agree on what the best protocol or tech standard will be, but if it’s not then deployed into application-layer services content, it’s just a technical standard, right?

In some regard, it’s like Esperanto. We come up with better artificial languages all the time, but if they’re not actually adopted and deployed, it doesn’t matter. And so, again, I’m optimistic this is right. I actually think that the Metaverse Standards Forum has far more participants than I would have originally guessed, and they’re bona fide a top to bottom.

But no one has been asked yet to make a concession. No one has been asked to deploy a standard that doesn’t optimize for their system or that might advantage one of their competitors. And we don’t yet have an operating network. There’s also another point that many have made at this point, which is there wasn’t too much on ethics, if anything, as to how are those standards going to manage for the softer issues around different platforms rather than just technical interoperations?

But again, it was just announced. The hard work is yet to come.

BYE: Yeah, I was involved with the IEEE Global Initiative on the Ethics of Extend Reality, where we’re been digging into some of those issues. But yeah, it’s huge, huge issue. One other big point I wanted to get in — because I know we have limited time here — is the anti-trust aspects of both Google and Apple seem to be really, really key in terms of, you know, there’s the lawsuit from Epic Games and Tim Sweeney going against Apple.

There’s a lot of information from discovery that was made available that’s in your book that helps paint the picture of some of these different potentially, anti-competitive dynamics of these duopolies in both Google with Android and Apple’s iPhone iOS. In terms of the 30% tax. It seems to me that there likely needs to be some sort of government intervention to break up this.

Otherwise we’re going to have the same type of thing with Meta seems to be wholeheartedly adopting the same thing that they’re critiquing — like Zuckerberg’s critiquing Apple around the 30% tax, but then they’re turning around and doing the exact same thing. And then adding even more in terms of Horizon Worlds in terms of their taxes. So it seems to me that in order to really have this open, interoperable Metaverse that is going to even have the potential to have like an open web manifestation, we need to break apart this 30% tax that’s at the hardware layer.

Otherwise, we’re going to be living into this world where just a handful of companies control our digital future. So I’d love to hear some reflections on that, because that seems to be, for me, at least one of the big takeaways. I’m glad that you are articulating that because I have similar frustrations with how Apple drags it’s feet with implementing WebXR, and not really great implementations with WebGL and it all serves their own ecosystem.

But in order to break that apart, it feels like you need to have at least some level of government intervention.

BALL: So I wholeheartedly agree. I think it’s nice that you and I are speaking today on, what is it, July 5th, where the EU started to announce more of their digital markets reforms. And they’re coming pretty firmly for a lot of the concerns that we have or you and I have, which generally seem to be shared by many in the developer community.

Epic Games as one example. The challenge about the Metaverse is we’re talking about a virtual platform, a persistent network of experiences which exist irrespective of any execution, any hardware, any platform. And yet we have to access them through a platform, a hardware device. There are essentially two of them. And of the others, they’re all contending to be the payment gateway.

Why wouldn’t you? Right? Visa’s one of the best businesses on Earth, and the Apple App Store is an even better one. And it’s not just that it increases payment fees, it’s that, as you’ve also astutely observed, they cripple competitive technologies, they stymie competitive business models. And we know that it’s ultimately transferring money from the pockets of independent creators to the largest and wealthiest companies on Earth.

At the end of the day, I think the challenge here is we’re actually talking about, in some regard, penalizing companies for having been so extraordinarily successful, for having built extraordinarily great products. Right? The Apple iPhone’s integration verticalization is why the mobile era is so accelerated. We wouldn’t be where we are today without that device. And yet we can now start to feel that as we shift to this next platform, it’s impeding us.

Sometimes I think you can very justifiably say in a maliciously, deliberately self-preferential and externally punitive way. But again, talking about the EU, we see evidence of regulatory action. I think you can see — like I’m Canadian, Tim Cook did not used to do interviews with the Toronto Star on privacy. I think that they’re doing that because they understand that they now need to win hearts and minds very differently to maintain their stewardship of the world’s most important computing platform.

But yeah, we need a lot of regulatory action. I’m hopeful there as well. I spent a lot of time at the end of the book saying that we’re disappointed with the last 15 years of digital regulation, but many of us, especially in the millennial generation, assume that that’s the pattern for regulators. And of course, we have political dysfunction that’s new today.

But through all of the 20th century, in the 19th century, new technologies were very vigorously defended by regulators telecommunications, energy, rail, steel. The government was usually pretty early. The Internet Engineering Task Force comes from DoD. The Internet comes from DoD. And the U.S. government had the foresight to relinquish control of those working groups and birds of feathers and standards bodies, understanding the criticality of doing so.

And so I’m hopeful, but the challenges are tough.

BYE: Yeah. Yeah. I guess the last question I have is what do you think the ultimate potential of the Metaverse might be? And what it might be able to enable?

BALL: Well, so this is always a fun question because you have Jensen Huang, the founder and CEO of NVIDIA talking about the fact that he believes that the GDP of the Metaverse will essentially, eventually exceed that of the physical world. The physical world economy today is roughly $70 trillion. We have another $20 trillion that’s coming from digital. And so we’re looking at $50 – $60 trillion over time that might go to the Metaverse.

You can describe it differently as say, billions of individual people reaching almost every consumer, every country, every sector, globally. But the humanist perspective is, look, I believe the digital era has had a lot of bad things, dis and misinformation, data rights, data security, the role of algorithms, toxicity, abuse, harassment, radicalization. But I think that technology is fundamentally agnostic. And I think that overall, I believe that the Internet has been a profoundly good thing for the world, especially when it comes to the democratization of information.

And so I’m hopeful that the Metaverse will allow us to correct many of the problems of the last 15 years that we as developers, users, and consumers can positively affect the trajectory of the Metaverse, who leads it, with which philosophies, and why. But that fundamentally, it will continue that transformation. I talk about in the book that education is an area of extraordinary importance, but which has been barely impacted by technology, especially when it comes to access to educational resources.

And so I’m hopeful that the Metaverse can really improve that, while also bringing more job opportunities to those who, unlike myself, are not born upper middle class in Canada, in a large city, with access to many of the best jobs in the world.

BYE: Awesome. Is there anything else that’s left said that you like to say to the broader immersive community?

BALL: Well, well. Kent, let me ask you a question. What are you least certain about the Metaverse?

BYE: Well, I mean, I really appreciated your elaboration on the blockchain, because I think that the way that I see the blockchain, you know, the Peer-to-Peer Foundation came up with an Accounting for Planetary Survival White Paper, and what they described was the difference between what they see as blockchain based upon libertarian values versus blockchain based upon common-based, shared resources and values.

So every single compelling aspect of the blockchain use [in your book] was around like the RNDR token or shared use of resources to do distributed computing. So I think there’s a real compelling use case for the future of distributed computing. But in terms of the libertarian value exchange that is doing rent seeking behaviors and basically replicating the scarcity model of the existing economy, I’m not convinced that buying and selling virtual land plots is going to be the future.

I think it’s going to be much more experiential-based and more along the lines of what Tim Sweeney and Fortnite have been doing to create really vibrant ecosystems and economies. So I really appreciated that you collaborated some of those different use cases of the blockchain, but I think for me at least, I think there’s a lot of engineering flaws that I don’t know at this point can be overcome. Because there’s like Sybil attacks and with proof of stake moving over, then you could have basically one whale that overtakes and controls everything.

So it’s not actually decentralized. So every functional utility that’s coming out of the decentralized systems have some sort of centralized point that I feel like can be vulnerable to being taken over and replicating the existing power dynamics of the existing economy. So that’s my hot take in terms of where the cryptocurrency is, but it’s actually not a matter of the technology [being] agnostic, it’s more the values that are underneath the technology.

So whether it’s being driven by a libertarian scarcity model that’s trying to do rent-seeking behavior or it’s generating more communal, shared resources to create something that would not be possible as an individual. And I’m more excited about the Decentralized Web efforts from the Internet Archive and more of those technologies, the RNDR token and other things that are going to actually bring utility that are experiential rather than for speculation and creating backdoors for fraud and abuse and money laundering and all the stuff that are the challenges with that.

So when I’m reading your book, that’s the things that I appreciated that you’re articulating those perspectives because I felt like my perspectives are being reflected in your book. But yeah, to me that’s the biggest open question in terms of that.

For me, I’m all in in terms of spatial computing. And I also appreciated the call out to CTRL-Labs and the change from 2D to 3D, meaning moving from a keyboard in a mouse and having the user input that’s using more electromyography and wrist-based inputs, and neural inputs, noninvasive neural implants, and brain computer interfaces.

That for me, the shift from 2D to 3D has to do with the types of embodied and spatial computing that comes from completely new human computer interaction interfaces that seem so much like science fiction of being able to detect the firing of individual motor neuron, meaning that it can detect your intentions without you actually moving. And how that’s going to be translated into how we interface with computing, I think is going to blow people’s minds.

So I appreciated that you included the CTRL-Labs in there as well, because I do think that the ways that we interface with computing is going to be so radically different in the next 15 to 20 years. And just from what we’ve seen already in terms of the neuroscience and these principles, we’re both kind of identifying these new methods of interfacing with the spatial computing, I think are going to be so revolutionary that once they really look into the neuroscience and the technology trends, it’s going to get really weird.

But yeah, the biggest questions are the centralization of power. Having just a small handful of companies that are controlling everything, and privacy and the ethics are the big things — and NeuroRights are the other things. For me, that’s the thing that I’m not knowing how that’s all going to play out. If we’re actually going to come to the other side of this and figure out how to as a collective, overcome some of the power law dynamics of having a small handful of companies that are controlling [a] disproportionate and asymmetrical [amount of] power.

And actually what the business models are actually going to be that go beyond surveillance capitalism, [which] seems to be fueling the existing methods. And if there’s going to be ways for people to take more ownership of their data. But also have business models to go completely away from that more extractive model of surveillance capitalism. So anyway, that’s a little bit more of my thoughts.

BALL: No, I’m — I’m totally aligned with you. I mean, look, the virtual scarcity model, virtual real estate I don’t believe in. I want to reserve the right to change my mind, but I don’t see something there yet. It seems like, you know, we always start every new computing wave by trying to recreate the thing closest to reality. Right? The skeuomorphism, the game center on the iPhone, the yellow line notepad on your iPhone Notes app.

This seems like the worst possible instantiation of trying to reproduce the real world with more tools, virtually. But I think everything the rest that you said is really I’m aligned with, so I think we can leave it there.

The only other thing that I’d say is the thing that excites me most about the advent of new hardware is the accessibility improvements are extraordinary. The Xbox accessibility controller, what we’re starting to see with CTRL-Labs and hopefully BCI can bring so many people into the digital era, not to mention virtual experiences that simply cannot today. And I think that’s a really important good. We talk about the Internet and games bringing people together, but often forget how many people just can’t through physical ability.

BYE: Well, Matthew, thank you so much for writing this book and taking all the time to detail all these things. I think you’ve provided a really nice mapping of where things are at now and where they could be going with a lot of examples to get some insights for the deeper patterns and trends of this Metaverse that you’re helping to specify the structures and forms.

So thanks again for writing it all down in the book — I know it’s not easy to do that — and to join me here on the podcast to help unpack it all.

BALL: Thank you. It was my pleasure.


This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality