Asad J. Malik is an AR artist who spent a lot time making work for the HoloLens on the film festival circuit, and then started a AR Hologram company named Jadu creating volumetric captures of celebrities and musicians using the high-end Microsoft Mixed Reality Capture Studio at Metastage. He was losing some of his enthusiasm after doing 30-40 hologram projects and navigating egos the bureaucratic dynamics of working with musicians, and then he started experimenting with cryptocurrencies and NFTs. He was also searching for alternative business models beyond advertising or high-end, holograms as a service, and had aspirations for creating a world-scale, AR game and product that allowed him to experiment more with tapping into the affordances of AR beyond passively watching volumetric performances from famous people.

On September 16th, Jadu released 1,111 jetpack NFT wearables that earned them over $400,000 in initial sales and over $450,000 in secondary market commissions over the first couple of months with their 5% commission rate. On December 12th, Jadu released 6,666 NFT Hoverboards earning over $4.3 million dollars. These NFT wearables will be used within the context of their AR application that will allow users to connect one of the supported 3D NFT avatar collections, and then puppeteer these avatars from a third-person POV in an AR app as they fly around on their jetpacks and hoverboards. They’re aiming to create an entire “Mirrorverse” game and platform with a scavenger hunt of geocached NFTs that are easier to find and discover if you already own a jetpack or hoverboard.

Jadu's 1/1 Gold Jetpack just sold for 111 ETH ($400K)

The Mirroverse is upon us!https://t.co/35ONi2dp8t pic.twitter.com/u0dEPY8QeC

— Jadu (@JaduHologram) October 7, 2021

Jadu was able to leverage the success of their jetpacks to raise a seed round of venture capital being announced today [Update: here’s Jadu’s announcement of a $7M seed round] to more fully make the pivot into exploring the “web3” potentials of shared ownership and decentralized finance that go beyond the existing web 2.0 business models. I share some of Malik’s optimism in the long-term potential of web3 to create new models of economic exchange while also simultaneously agreeing with some critics of the web3 branding as well as some of the larger ecological concerns around Proof of Work. Ada Rose Cannon is the co-chair of the W3C Immersive Web Groups, and I agree with aspects of her incisive criticism about the web3 branding when she says, “I get so pissed off every time I see Web 3.0, NFT assholes have co-opted the branding of the Web to imply so many untrue things about their cryptoshit tech and set themselves up as the successor to the Web platform all in a single move.”

Malik concurs that there are plenty of fair criticisms in this sentiment as the web3 term is often completely disconnected from the infrastructure aspirations of the W3C and the current open web. There’s still lots of gaps for how web3 would replace the existing web2 world wide web, which results in many hyperbolic statements that are still largely aspirational. Jadu’s application won’t be using any WebXR technologies, and will be focused on creating their Mirrorverse within the context of a 2D native phone application starting on iOS. So I am unclear as to what exactly web3 means just as the larger crypto and XR community has been overusing the buzz word of the “metaverse” to describe a wide range of 3D virtual world projects to 2D *.jpg NFT collections. Malik feels that web3 still describes the aspirations and intentions for what Jadu is aiming to create with their Mirrorverse in terms of creating an ecosystem of shared ownership where there’s 95% of the value being exchanged and owned by the community maintains while Jadu takes a 5% cut. This is a vast shift from the existing value exchange on the existing web that’s fueled by a lot of surveillance capitalism and advertising.

I’m sympathetic to the need and desire to experiment with new models that move beyond surveillance capitalism, and as Joseph Poon told me at the Decentralized Web Summit 2018 that it is hard to learn about new models of value exchange when looking at playing poker games with fake money. Because it’s difficult to simulate how the incentive structures will pragmatically play out in reality, then crypto projects have to “do it live” in order to really find out how people exchange value with each other. Is this is what is happening with NFTs? Or are they all just scams or multi-level marketing, pyramid schemes?

I suspect that the truth is probably somewhere in the middle where there are plenty of grifters and scams that you need to look out for, but there’s also plenty of earnest artists and companies experimenting with new models of decentralized economic exchange in the spirit of moving towards more equitable models platform cooperativism and decentralized autonomous organizations. Will Proof of Stake approaches address some of the the ecological concerns? Or is this the type of centralization scope creep of decentralized finance that Malik says that Bitcoin maximalists have been warning about?

If Jadu moves towards creating a DAO, then it won’t be making decisions about the lore, mythology, and underlying game mechanics of their Mirrorverse. Some of their challenges will be to find out to balance the new user experience of folks who don’t own one of their NFT wearables that start at a floor of 3.7 Ethereum (~$13645.74) for one of the 1,111 Jetpacks or a floor of 0.45 Etherum ($1659.62) for one of the 6,666 Hoverboards. They want to create a dynamic where one of the owners of their NFTs will gain some advantages within their game world and platform, but not to the degree that will alienate new users who don’t own any jetpacks or hoverboards.

Malik acknowledges the dynamics of compounding gains for crypto early adopters, and strives to find ways to create a compelling experience for users beyond their core early adopters and whales who are buying and selling their NFTs to create a recurring revenue stream that allows them to build out their independently driven AR platform. In order to compete with the AR platforms of Snapchat, Instagram, or TikTok, then it may turn out that jetpack and hoverboard NFTs will end up bootstrapping more grassroots innovation, shared ownership, and the gamification of AR by Jadu. Or we may be in a giant NFT bubble that bursts at any moment. Either way, it’ll be interesting to see where Jadu takes the future of their Mirrorverse and what type of interoperability they implement for how other avatars could use their object-oriented wearables across other cryptocurrency-based metaverse platforms like Decentraland, Somnium Space, Cryptovoxels, or The Sandbox.

LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

On March 23, 2021, Rec Room announced that they raised $100M with a valuation of $1.25 billion, which makes them the first VR software company to achieve “unicorn” status. Rec Room was launched for the Vive on June 1, 2016 after 99 days of development, and they’ve continued to rapidly iterate and develop it over the years with a release cadence ranging from a weekly to every other week for five and half years now. They’re the most cross-platform VR application being on Oculus Quest, PCVR on Steam and Oculus Home, PSVR for PS4 as well as a 2D version for PS4 (with a PS5 release coming soon), XBox, iOS, as well as Android.

I had a chance to catch up with one of the co-founders of Rec Room & Chief Creative Officer Cameron Brown on October 7th after the third annual Rec Con community conference. I wanted to get more context on the origin story of Rec Room, which was started by six people who all worked on creating augmented reality experiences for the Microsoft HoloLens — including Nick Fajt, Cameron Brown, Dan Kroymann, Bilal Orhan, Josh Wehrly, and John Bevis. I also wanted to get an oral history of the early days and evolution of their social VR platform, and so we cover some of the big turning points in the evolution of the user-generated platform, their in-game economy, and focus on creating opportunities for social VR interactions that’s embedded into everything that they create. There’s a lot of social VR and user interface innovations they’ve pioneered we well as helped to define many of the design paradigms that will no doubt persist into the evolution of the metaverse.

LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

The Oregon Shakespeare Festival in collaboration with Artizen producers Ana Brzezińska and René J. Pinnell commissioned & produced four VR pieces that premiered at the inaugural Quillsfest, which was a two-day event November 19-20 held within the Museum of Other Realities. Each of these commissions paired established theatre makers with veteran XR artists to produce four different types of VR experiences that blend the design practices from a broad range of design disciplines. There were also five behind-the-scenes exhibition installations giving insight into the creative process of a variety of different XR productions, some of which had more live performance aspects. All of these pieces are still available to be seen within the Museum of Other Realities until December 18th, which requires a PCVR system to see them.

I had a chance to talk about all of these XR pieces with Scarlett Kim, who is an Associate Artistic Director & Director of Innovation & Strategy at Oregon Shakespeare Festival. We talked about OSF’s journey into immersive technologies, and their collaboration with Artizen in order to help pair established XR artists with established theatre makers. There was a lot of exploratory process to merge the different design disciplines in these distributed and at times asynchronous collaborations that spanned five different time zones amongst all of the creators. We also talk about how XR technologies are part of a larger accessibility roadmap for OSF in order to make immersive art and live performance more accessible to people who are not able to attend their destination theatre offerings within Ashland, Oregon.

LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST

Below is more information provided by Oregon Shakespeare Festival about the four commissioned VR pieces and their creators as well as the five behind-the-scenes exhibitions.

Here’s a teaser trailer of some of the pieces shown at Quillsfest:

Guardian of the Night

Dede Ayite – Lead Artist
Michael Joseph McQuilken – Lead Artist
Joel ‘Kachi Benson – Lead Artist
Michael Thurber – Composer and Sound Designer
Jennifer Harrison Newman – Movement
Tyler Alexander Arnold – Set PA

Costumes Built by The Public Theater Costume Shop in New York

Within a number of West African traditions, there is a belief in spiritual guardians known as Zangbeto, Coumpo or Kwagh-hir. These beings act as spiritual guardians and emerge in a whirlwind of energy during festivals to speak to the people.

This project creates a virtual reality experience that immerses the viewer in a dark, unfamiliar forest. A guardian will appear and through dance and movement, illuminate and guide the viewer.

Raffia and a variety of recycled materials (solid worldly materials) are used to create the body of this otherworldly guardian, who appears only for a brief moment to reflect our existence back to us, and in so doing help us find our way.

This experience is an exploration of movement, and an embodiment of our oneness with the earth and nature.

Anakwad

Ty Defoe – Creative Director
Dov Heichemer – Creative Co-Director
alpha_rats – Developer
Clara Luzian – 3D Artist
Suzanne Kite and Devin Ronnenberg – Music

An Anishinaabe tale brought to life in an Indigiqueer dreamscape summoning the shapeshifter queer ghost as it nullifies linear time and learns to unlock syllabic truths to regain balance in the destructive destiny of now.

Ordinary Gesture

Raja Feather Kelly – Lead Artist, Writer, Co-Director
Illya Szilak – Co-Director, Creative Producer
Cyril Tsiboulski – Art Director, Technical Director, Lead Developer
Christoph Mateka – Sound Design & Score Composition

Ordinary Gesture is a Virtual Reality Theatrical experience that intersects theatre, meditation, and movement. The experience seeks to surrealize the experience of empathy by situating the player in 5 scenes that expand from their body to space-time (the universe) and back again. Inspired by the movies Magnolia, Melancholia, Waking Life, the poem “You Are Never Ready” by Nicole Blackman, and the writing of cultural anthropologist Ernest Becker, Ordinary Gesture asks the player to contemplate existence, suffering, compassion, and gesture as both ingredients to create theatre and a means to perhaps better understand empathy.

O-DOGG: An Angeleno Take on Othello

Performed by Tariq “Black Thought” Trotter
Executive Produced by Nataki Garrett for the Oregon Shakespeare Festival, Kirkaldy Myers, Shariffa Ali in partnership with Artizen & the Museum of Other Realities
Creative Producers: Joe Brewster & Michèle Stephenson, Rada Studio
Production Company: AliAlea Productions
Producer: Adrian Alea
Line Producer: Emma McSharry
Co-Directors: Shariffa Ali and Brisa Areli Muñoz
Writer & Dramaturg: Alex Alpharaoh
VR Engineer: Sagar Patel
Costume Designer: Tanaka Dunbar Ngwara
Sound Designer/Composer: Josh Horvath
Performer: Tariq “Black Thought” Trotter
Director of Photography: Kris Pilcher
Assistant Director of Photography: Kevin Laibson
Sound Mixer: Christian Guiñanzaca
Video Editor: Micah Stieglitz
Teleprompter Operator: Rudy Dedominicis
Production Assistant: Tanéyah Jolly

In this immersive experience, users are launched into a cacophony of chaos during the 1st night of the 1992 Los Angeles Uprising. Unable to look away, the participant is forced to contend with a city divided along race, class, and immigration lines as revolt fills the city, provoked by the acquittals of four White LAPD police officers who beat and nearly killed Rodney King. From Shakespeare to Shakur, Black Thought to Alpharaoh, these poets and lyricists have a visceral way of speaking honestly about the history of our times with critical precision. This piece, effusive as much as it is restrained, offers a heated conversation about race, colorism, bias, and culture in America through liberatory practices of hip hop, spoken word, lyricism, rhythm, and flow, inspired by Shakespeare’s well known Othello.

Here are the five behind-the-scenes exhibitions:

• Laila is an interactive work created by Esa-Pekka Salonen, Paula Vesala, Tuomas Norvio and the Ekho Collective for the Finnish National Opera, music and visuality evolve and change in interaction with the audience.
• Satore Studio’s Cosmos Within Us delivers a sense of hope and understanding to anyone affected by Alzheimer’s.
• Dazzle by Gibson/Martelli + Peut Porter, recreates the optimistic, rebellious spirit of the 1919 Chelsea Arts Club ‘Dazzle Ball’.
• POV by GRX Immersive Labs is a hyper-digital sci-fi virtual reality series immersed in a near future Los Angeles where personal data is the new currency and weaponized A.I. Police drones enforce the law.
• Finding Pandora X by Double Eye Studios is a modern take on an ancient myth that shifts the perspective on a narrative that has long been misinterpreted.

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

Exploring Home is a full-body tracked, dance performance in a VRChat world that premiered at Venice VR Expanded 2021. It explores issues around identity, avatar representations, being outcast and discriminated against, and it’s a unique blend of different genres and affordances of performance, world design, theater, sculpture, and audio production. It was written, produced, and performed by Sara Lisa Vogl and was catalyzed after she had an experience of discrimination for not wearing an anime avatar within a VRChat dance club. This brought up traumatic experiences of racism and discrimination for being a half-Iranian immigrant in Germany, and so she turned to the medium of VR to create an immersive experience that allows you to step into her avatar skins, go through the nested stages of her life, and listen to audio reflections of her own journey and experience with identity, community, and the feelings of shame from not being accepted. It’s a piece where you can fearlessly break through barriers, leave things behind, and be able to see tings from new perspectives.

I had a chance to talk Vogl after seeing the World Premiere performance at the Venice VR Expanded Festival where she shared her journey in creating this unique piece that’s part interpretive dance, part immersive theater, part guided tour, part social VR worldhopping, and part deep reflection on identity, embodiment, and avatar representations. It’s also worth noting the evocative sound track by Iranian musician Ash Koosha as well as another Iranian artist who cannot be named.

LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

Hope for Haiti is a Haitian-based, non-profit with the goal to improve the quality of life for Haitian people through education, medical, clean water initiatives, and economic development programs. After a visceral experience with a number of VR projects at VR World in New York City, Sarah Porter, who the Director of Business Development & Strategic Partnerships at Hope for Haiti, set out on a journey to explore how to use immersive technologies for outreach, education, and fundraising. She collaborated with Max Noir from China-based FXG Studios to create a Unity-based Hope for Haiti VR world (available on SideQuest) featuring a classroom (with interactive chalk), clean water punk, some informational videos, waterfall, beach, and campfire that serves as a social VR platform to hold events and guided tours through some of the projects, people, and stories of people working for their non-profit. They also have a NFT gallery featuring a number of artists who donated art to be sold to raise money for their non-profit.

Porter was a speaker with Noir at AWE in a session titled Virtual Reality for Social Impact in Haiti, and also had a spot on the AWE Expo floor playground showing off their experience to get feedback and support from the broader XR community. They held an initial event on October 20th, and have more plans to expand their virtual Haiti work and have more outreach, education, and fundraising events. I had a chance to catch up with Porter at the end of the Augmented World Expo to recap her experiences on the expo floor as well as the feedback she was receiving about their project from the broader XR community.

The Hope for Haiti experience is now available on SideQuest.

LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

The MetaMovie Presents: Alien Rescue is one of the most immersive experiences I’ve had in VR. It’s part Cinematic Adventure, part Immersive Theater, and part game that’s fusing the affordances of each of these different mediums in a unique way. There’s one interactor who is cast as the protagonist who meets up with three other immersive theater actors who are taking you on a fairly linear adventure to rescue some aliens. The main character gets to do some live-action role playing, customize their character’s identity and expertise, and make a number of different choices along the way to potentially go down some forks in the story. There’s also a number of other audience members there who are cast as iBots, which don’t have as much narrative agency as they can’t really speak, but they have more embodied & locomotion agency in terms of being able to explore around the environment as well as chose which characters and storylines to follow as the part splits up at different points.

I’ve had a chance to see MetaMovie twice now. I saw the first half at their initial premiere at the Venice Film Festival 2020, and then the full experience again as a judge for Raindance, where it won the best multi-player experience and got runner up for best game. It’s part story, part game, and part immersive theater adventure, and so I’m glad that it was able to win the multi-player award as the overall sense of social presence has been some of the deepest that I’ve had in any immersive VR storytelling or immersive theater experience that I’ve had so far. It’s quite a unique blend of agency and story, and it’s a mix that has taken a lot of time to develop through years of different experimentation on previous MetaMovie Project experiments.

I had a chance to talk with most of the cast and crew after I saw the experience at Venice in September 2020, where I got a lot of context and history about how the series of MetaMovie Project experiments have evolved. This interview includes director and creator Jason Moore, producer Avinash Changa, and actors Nicole Rigo, Kenneth Rougeau, and Marinda Botha. The fourth actor Craig Woodward was unfortunately not able to join us for this interview, but we’re able to explore both the past, present, and future of immersive storytelling in this wide-ranging conversation from the POV of the director, screenwriter, producer, and actors perspectives.

Moore tells me that they’re currently planning on doing weekly runs of The MetaMovie Project, and I HIGHLY recommend checking it out. I’ve been able to be the hero twice now, and it’s super compelling. But I also hear how satisfying it is to be an iBot as well to be able to help solve puzzles, track different characters, and have a bit more agency to explore the environmental and storylines that are the most compelling.

LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

Tosca Terán (aka Nanotopia) is an interdisciplinary artist at intersections of ecology, bioart, mycology, and sound, and she collaborated in creating the “Mycelia” performance with the Metaverse Crew and Sara Lisa Vogl (aka _ROOT_) to bring her fungi biosonification music into an amazingly immersive, audioreactive VRChat world.

Premiering at the AMAZE festival in July, Terán held a number of musical performances where she translates the conductance from Oyster Mushrooms into MIDI and a form of ambient electronic music in addition to her own musical performance. My mind was pretty blown after hearing her describe her process of how she generates her music, and I wanted to sit down with her to do a super deep dive unpacking each step of the process (See the show notes down below for more details as you listen in). I had a chance to catch up with her the day before her Venice performance in September, and I had a chance to see it a second time at the Raindance Immersive festival, where it ended up winning one of the Spirit of Raindance awards for it’s innovative, independent, and pioneering spirit.

This ended being quite a wonky deep dive into the audio production pipeline of biosonification of fungi, but also has some deeper thoughts about the implications of interspecies communication, the potential of using haptics, and spatialized ambisonics and sonification to further explore biometric or physiological data from either humans or non-human species. The Mycelia performance has been one of the more magical experiences I’ve had in VR, especially considering that it does provide a portal into the biorhythms and proxies of consciousness of non-human intelligence. So hopefully this conversation will not only help explain Terán’s creative process, but also help to inspire other bio-artists to continue experimenting and exploring the potentials of biosonification with the context these immersive worlds.

SHOW NOTES

• InterAccess Slime Mold as Capacitive Storytelling
• Elecricity for Progress – Sam Cusumano
• MIDIsprout MIDI Sprout Biodata Sonification system
• Midnight Mushroom Music – SoundCloud
• April 2020 – Goethe-Institut New Nature Exchange
• An Immersive Media and Climate Science Exchange between Canada – Germany – Mexico – US
• Symbiosis and Dysbiosis – working with Haptic – bHaptics
• Ganglion Board from OpenBCI
• EmotiBit: Wearable biometric sensing for any project!
• HerpDerpinstine VRCBhapticsIntegration
• 555 Timer – acting as a galvinizer
• Pleurotus ostreatus – Oyster Mushroom
• 16MHz Crystal selection for ATMega328P
• Animoog app
• Euroracks
• CV Tools
• Nanotopian bio-sonification modules (outputs MIDI)
• Biosonofication images
• Mycelia at Raindance Oct 29, 2021
• Mycelia at Venice VR Expanded
• Chaos fungorum: Nuit Blanche live performance & exhibit Sept 29 2018
• Coalesce: Center for Biological Arts at Buffalo University
• Tangerine Dream
• Isao Tomita
• TouchDesigner
• Open Sound Control (OSC)
• Max (software)
• Holobiont is an assemblage of a host and the many other species living in or around it
• Brendan Lehman
• Mycelia in VRChat
• Poiyomi is creating Shaders
• Grone Drone Synth
• Moog Mother 32
• JOVE Filter
• Mutable Instruments Yarns
• Bastl 1983 Eurorack Module – Polyphonic MIDI to CV Interface
• Forest UnderSound Installation at The Museum in Kitchener Ontario for SONICA21
• Mellotron
• Moog Grandmother Semi-Modular Analog Synthesizer and Step Sequencer
• Moog Mother-32 Semi-modular Eurorack Analog Synthesizer and Step Sequencer
• Metaverse Crew

LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

One of the most magical AR demos I’ve ever had a chance to see was Tilt Five at Augmented World Expo 2021. Jeri Ellsworth stumbled upon a complete paradigm shift towards AR while she was at Valve and accidentally had a beam splitter turned around and it was shooting beams of light that happened to hit some retroreflective material another Valve engineer was experimenting with. She saw a beautiful, high-contrast image that inspired her to continue to develop this idea into a fully-fledged, tabletop gaming focused AR headset that she was able to acquire from Valve after getting fired, go through another startup cycle of beginning and ending of castAR with too broad of a focus, and then eventually into Tilt Five, which has been much more laser-focused on tabletop gaming.

I had a chance to catch up with Ellsworth during a busy AWE showing, where she shared quite a lot of details about her journey into XR really starting with helping to bootstrap Valve’s hardware division, some of the internal dynamics there, her journey with castAR, and then finally with her latest efforts with Tilt Five. They should be shipping out their Kickstarter units here soon. It’s a completely different and unique approach to AR, and it’s also one that creates some pretty magical experiences when focused on a table-top gaming retroreflective material. It really feels like this is a device that is going to bootstrap quite a lot of innovation with AR gaming and the affordances of tabletop holograms, and I look forward to see how it continues to develop. But definitely keep an eye on Tilt Five, and try it out for yourself to see how they’ve been able to bring the magic of holograms to live with their Tilt Five glasses.

LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST

Here’s a previous interview with Ellsworth conducted by Valve News Network’s Tyler McVicker that I mentioned in this interview.

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

I had a chance to do a demo of the Lynx R-1 standalone mixed reality headset at the Augmented World Expo, which was a really compelling experience that blended the virtual and real better than any other headset I’ve seen do before. Part of the magic was having a headset specifically engineering to minimize the distance from the the cameras from my eyes, therefore minimizing the normal proprioceptive disconnect between what I’m feeling in my body vs the offset that I’m seeing in my arms and hands. It’s the first XR HMD that I’ve seen that can legitimately call itself a Mixed Reality device — as opposed to Windows Mixed Reality headsets, which are really just VR HMDs with the aspiration to maybe eventually live into their names of doing mixed reality.

I had a chance to catch up with Lynx CEO Stan Larroque on the last day of AWE to recap his first public demos of the Lynx R-1, their collaboration with Qualcomm on using the full capacity of the mixed reality features build into the XR2 chip, their unique four-fold catadioptric freeform prism optics design, what their OpenXR runtime integration will be able to unlock with Qualcomm’s Snapdragon Spaces, their successful Kickstarter raising 2.4x their target amount, how they’re leaning into and leveraging the broader open source communities to compete with the biggest players in XR, and his own first real experience of mixed reality with the Lynx that only happened about a month ago.

The Lynx R-1 is expected to ship some of their first units in April 2022, and I’m excited to see what innovations in mixed reality and augmented reality prototypes are able to be created with this headset. They’re one of the few remaining, independent headsets out there competing against the biggest tech companies in the world, but yet taking a leap of faith in how compelling headsets that are truly optimized to be standalone and all-in-one devices capable of virtual reality, augmented reality, or mixed reality. They’re optimizations to focus solely on mixed reality yield some really interesting tradeoffs, but the end experience is totally worth it is a device that starts to show the real power of seamlessly blending the virtual and the real.

LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

On November 9th at the beginning of the Augmented World Expo (AWE),Qualcomm announced Snapdragon Spaces, which is a series of AR tools striving to cultivate an “open, cross-device horizontal platform and ecosystem.” The Snapdragon Spaces tools include “environmental and user understanding capabilities that give developers the tools to create headworn AR experiences that can sense and intelligently interact with the user and adapt to their physical indoor spaces.” This specifically includes spatial mapping & meshing, local anchors & persistence, positional tracking, plane detection, image recognition & tracking, object recognition & tracking, occlusion, & scene understanding.

What’s especially interesting to me is that all of this is “based on the Khronos® OpenXR™ specification to enable application portability and is the first headworn AR platform optimized for AR Glasses tethered to smartphones with an OpenXR conformant runtime.” The Snapdragon Spaces platform will enable Qualcomm to start leveraging the additional computational resources of mobile phones in order to enable a distributed compute and rendering capabilities that allows AR to become like an extended spatial display to existing mobile phone apps.

It’s also starting to build out the more software-driven potentials for innovation that comes from specific application developers, who will be empowered to write OpenXR extensions and modules that not only benefits their specific application, but potentially the broader XR ecosystem. This is a really exciting development to see Qualcomm go down this path of cultivating an open ecosystem like this, and it makes a lot of sense why they wanted to become a big sponsor of AWE 2021 with an opening keynote slot with Hugo Swart announcing Snapdragon Spaces (registration required) as well as a couple of sessions by Steve Lukas diving into more specific details of Snapdragon Spaces in the Ramp to the Future of AR session (registration required) as well the more generalized tips for what type of AR applications they’re looking for in order to grow the AR ecosystem here in this session on Designing Your Mobile App for Qualcomm’s Tools (registration required).

They also announced an early access program for XR developers called The Pathfinder Program that’s a new program for Snapdragon Spaces “designed to give qualifying developers early access to platform technology, project funding, co-marketing and promotion, and hardware development kits they need to succeed.” Generally availability for Snapdragon Spaces won’t be until the Spring of 2022.

Going to AWE 2021, it was made really clear to me the impact that Qualcomm has had on the cultivation of the standalone VR and AR HMD market So many of the latest standalone devices use either the XR1, including Snap Spectacles 4, Ray-Ban Stories, Lenovo ThinkReality A3, Vuzix M4000 & M400, or use the the XR2 including Quest 2, Vive Focus 3, Pico Neo 3, & iQIYI QIYU 3, Magic Leap 2, or HoloLens 2.

In fact, since 2016, there’s been over 50 devices that have launched on either the Snapdragon 820 (announced September 1, 2016 at IFA), Snapdragon 835 (announced January 3, 2017 at CES), Snapdragon 845 & VR Dev Kit Reference Design (announced February 21, 2018 at MWC and shown at GDC March 2021 + my previous Voices of VR interview with Qualcomm at GDC 2018 after seeing that reference design), and then their XR-specific XR1 chip announced at AWE May 29, 2018 and then their XR2 chip announced December 5, 2019.

Here’s a graphic that Steve Lukas presented at AWE that shows 33 out of the 50+ XR devices that have launched with Qualcomm chips since 2016.

I had a chance to catch up with Qualcomm’s VP & GM of XR Hugo Swart during AWE on November 11th, where I was able to get more context for their new Snapdragon Spaces platform and open ecosystem they’re cultivating, but also to recap the evolution of standalone VR and AR devices since 2016 when the Snapdragon 820 was announced as being the first chip capable of handling the needs of standalone XR devices.

LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality