#544: Google Tango’s Engineering Director on AR Capabilities Enabled by Depth Sensors

Johnny-LeeAugmented Reality has played a huge role at the developer conferences for Microsoft, Apple, Facebook, and Google, which is a great sign that the industry is moving towards spatially-aware computing. Microsoft is the only company to start with head-mounted AR with the HoloLens while the other three companies are starting with phone-based AR. They are using machine learning with the phone camera to do six degree-of-freedom tracking, but Google’s Project Tango is the only phone solution that’s starting with a depth-sensor camera. This allows the Tango to do more sophisticated depth-sensor compositing and area learning where virtual objects can be placed within a spatial memory context that is persistent across sessions. They also have a sophisticated virtual positional system (VPS) that will help customers locate products within a store, which is going through early testing with Lowes.

I had a chance to talk with Tango Engineering Director Johnny Lee at Google I/O about the unique capabilities of the Tango phones including tracking, depth-sensing, and area learning. We cover the underlying technology in the phone, world locking & latency comparisons to HoloLens, virtual positioning system, privacy, future features of occlusions, object segmentation, & mapping drift tolerance, and the future of spatially-aware computing. I also compare and contrast the recent AR announcements from Apple, Google, Microsoft, and Facebook in my wrap-up.

LISTEN TO THE VOICES OF VR PODCAST

The Asus ZenPhone AR coming out in July will also be one of the first Tango & Daydream-enabled phones.

A video of one of the Tango Demos at Google I/O

Demo video of Tango’s Virtual Positioning System

Video of “Into the Wild” 10,000 square foot Tango AR installation at the Marina Bay Sands Art & Science Museum

Here’s a Twitter thread discussing the different AR SDKs from the major tech companies

Here’s the What’s New on Tango presentation from Google I/O 2017

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

#543: Visualizing An Invisible Cyber War with Zero Days VR

yasmin-elayat There’s an invisible cyber war that’s happening between major nation states, and Zero Days VR takes you inside of it in a completely new way using virtual reality. You go on a journey into a hyper-stylized cyberspace world where you embody the Stuxnet computer worm as it navigates programmable logic controllers, changes code, and destroys Iranian nuclear centrifuges.

Zero Days VR is one of the most powerful VR documentaries that I’ve seen so far since it uses the unique affordances of VR to visualize what’s at stake for weaponizing security vulnerabilities, and it uses these volumetric affordances to innovate what’s possible in immersive storytelling. The end result is a visceral and embodied experience of an otherwise complex and abstract topic of cyber warfare that is probably one of the most important stories in our world today.

Zero Days VR is based upon the journalistic work of Alex Gibney’s Zero Days documentary, but it’s not a promotional experience for the movie but rather a self-contained experience that uses VR to tell aspects of the story that didn’t work as well in the 2D version. The VR experience tells the story as if the main character is code, and they created different immersive environments that reflected testimony from a range of computer experts as well as a number of official government denials.

At Sundance, I had a chance to talk with Scatter’s Creative Director Yasmin Elayat about directing Zero Days VR, and how this project came about through the use of their Depthkit technology in Gibney’s documentary. We also talked about their failed experiments to make this into a non-linear and interactive experience. It turned out that too much journalistic integrity and overall context was lost when they surrendered control over the linear release of evidence, and so they had to abandon the more interactive components of the experience that they were building off from their previous experience on interactive VR doc CLOUDS created by James George and Jonathan Minard.

LISTEN TO THE VOICES OF VR PODCAST

Zero Days VR was released on June 8th on Oculus Home for both the Oculus Rift and Gear VR, and it also won an award for Narrative Achievement at Unity’s Vision VR/AR Awards.

Here’s the trailer for Alex Gibney’s original documentary of Zero Days

Here’s a behind-the-scenes look at Zero Days VR from Unity’s 2017 VR/AR Vision Summit

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

#542: Bringing Otherworldliness, Fun, & Whimsey into Your Home with AR game “Woorld”

robin-hunicke-2017Funomena’s Woorld won the best AR experience at the recent Google Play awards. You scan your room with a Google Tango-enabled phone, and then you’re encouraged to decorate your space with extremely cute art and characters designed by Katamari’s Keita Takahashi. Part of the gameplay in Woord is to figure out how to combine different object together in order to unlock new objects and portions of the story in your space.

Funomena had to innovate on a lot of augmented reality user interaction paradigms and spatial gameplay in designing this game. I had a chance to catch up with Funomena co-founder and CEO Robin Hunicke at Google I/O to talk about her game design process, as well as her deeper intention of bringing sacredness, mindfulness, calmness, worship, spirituality, love, empathy, and kindness into your environment through augmented reality technology. She takes a lot of inspiration from Jodorowsky’s Technopriests as well as the sci-fi novel Lady of Mazes by Karl Schroeder. Hunicke also sees that there’s a split that’s emerging between the commercial VR and the indie VR scene with the character of content that’s being funded, and she talks the importance of supporting indie game creators.

LISTEN TO THE VOICES OF VR PODCAST

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

#541: Bearing Witness to Sexual Assault Stories & Healing from Trauma with “Testimony”

zohar-kfirTestimony is one of the most profound and powerful applications of virtual reality that I’ve seen so far. It’s an experimental documentary that captures the stories of sexual assault from five women broken up into five segments. You’re completely immersed within a virtual sphere with these five stories that are represented as sequences of circles on different lines. As you look at a specific circle, it comes into the full frame and plays a 2D video segments from that victim either sharing their story of sexual assault, the aftermath, their process of healing, or their ideas for how to reform the criminal justice system. The depth of immersion and intimacy that the virtual reality medium enables allows you have much more capacity to provide your full attention and to bear witness to these stories of deep emotional intensity. It’s a radical application of VR that represents a revolutionary approach to healing from trauma.

Testimony premiered at Tribeca in April, and I had a chance to catch up with Zohar Kfir to talk about the challenges and shame that sexual assault survivors experience. We also talk about how the virtual reality medium is unique suited to provide a platform and medium for sexual assault survivors to share their stories of survival. It’s been a profoundly hearing experience for these women to authentically share the emotional intensity of their sexual assault experience, as well of the challenges in dealing with the criminal justice system, and process of healing from trauma.

LISTEN TO THE VOICES OF VR PODCAST

Testimony shows that virtual reality is able to carry a depth of emotional intensity of trauma that previous mediums where maybe not as well suited for. The interactive nature of Testimony provides the affordance of being able to look away from a testimony story if becomes too intense, and it’ll stop playing and you’ll retreat back into the sphere of women metaphorically standing in solidarity with each other.

I think that it’s an experience that would be difficult to pull of in previous 2D mediums, and I think that it demonstrates how VR has the unique capacity to discuss the types of trauma that was only previously discussed behind closed doors in the context of a therapy session. The level of emotional intimacy and presence that you can achieve in VR allows for a reciprocal transmission and reception of topics that have been either too taboo or intense for previous communications mediums.

Kfir also has plans to make keep this project going as a living and interactive document. So other sexual assault victims will be able to record their stories of sexual assault and contribute them to the project where they can be witnessed and heard. Providing a platform for having your sexual assault trauma being heard, witnessed, and believed is going to have profound healing implications for the women who participate. It’s a form of distributed and asynchronous Truth and Reconciliation process that will allow victims to release their shame, humiliation, and trauma around being sexually assaulted.

There’s still a long ways to go to reform the criminal justice system around cases of sexual assault, but Project Callisto that was recently announced. It allows victims to report the details of their sexual assault and their perpetrator online. If there are multiple reports against the same person, then it will trigger the criminal justice process and optionally connects the women. This is a huge improvement in the current process, and seems to be a model that has been gaining some traction in other countries.

Testimony is now available as of June 1st, 2017, and it’s one of the most profound and moving experiences that I’ve had in VR so far. Definitely check it out, and share it with your friends and family. You can learn more information from their Testimony website, or follow online with the #ShatterTheSilence hashtag.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

Support Voices of VR

Music: Fatality & Summer Trip

#540: Investing in Exponential Technologies in VR, AI & Drones with A16z’s Kyle Russell

kyle-russellKyle Russell is on the deal & research team for Andreessen Horowitz (aka A16z) where he’s focusing on investing in technologies ranging from virtual & augmented reality, artificial intelligence, drones, and other exponential technologies like quantum computing. A16z has invested in a number of prominent VR companies including Oculus, Magic Leap, Within, BigScreen as well as Lytro, & Improbable.

I had a chance to catch up with Russell about how these exponential technologies are combining together in order to solve real problems. Russell used to write for Techcrunch, and so in order to keep up with the latest tech trends he’s become a power user of Twitter lists, subreddits like /r/MachineLearning, and an AI-research aggregator called the Arxiv Sanity Preserver. He also is tracking many different possible futures when it comes to emerging business models based upon the decentralized blockchain, self-sovereign identity initiatives such as the Decentralized Identity Foundation, as well as thoughts on the growing wealth disparity and blending of cooperative and competitive economic approaches.

LISTEN TO THE VOICES OF VR PODCAST

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

#539: Google Expeditions is Leading Innovation in the Future of Immersive Education

Jennifer-HollandGoogle’s mission to organize all of the world’s information, and so it’s a natural fit for Google to be one of the leading innovators for using VR for immersive education. Google Expeditions was born out of a hackathon soon after the Google Carboard launched at Google I/O 2014, and it’s since been shared with over 2 million students who have gone on virtual field trips. At Google I/O, they were showing Tango demos that showed me just how compelling augmented reality is going to be in the future of collaborative & embodied educational experiences.

I had a chance to catch up with Daydream’s Education Program Manager Jennifer Holland at Google I/O where we talked about the history of Expeditions, and how successful it’s been in creating new levels of immersion and engagement with students. She talks about how the Expedition experiences are designed to be agnostic to any specific age or subject matter, but also independent of specific teaching strategy or philosophy. Google has been rapidly iterating on creating useful tools that are immediately useful teachers to introduce immersive experiences into their lesson plan, and there’s a lot that is left up to the teacher to be able to guide and direct the interactions and group learning exercises. Holland talks about some of the tools that have been built into expeditions, as well as the feedback that is driving the future of immersive education towards shared augmented reality experiences with Tango-enabled devices.

LISTEN TO THE VOICES OF VR PODCAST

One of Google’s biggest strengths in the VR community is cultivating mental presence by using open web technologies to fuse together information about our world so that we can experience it in a new way. Google Earth VR is a perfect example of fusing many different sources of data about our world, and providing an entirely new immersive experience of it in VR.

Right now, Google’s Expeditions team and their collaborators are the only ones who are creating educational experiences, but they’d like to eventually make it easier at some point for people to create their own Expeditions. The Google Expeditions team announced during their Google I/O session that they’ve been using Mozilla’s WebVR framework A-Frame in order to rapidly prototype Expeditions experiences in VR, and Unity to prototype experiences in AR.

I expect that WebVR and WebAR technologies will be a critical part of Google’s VR and AR strategies, as they’re helping to drive the standardization process with the work of WebVR primary spec author Brandon Jones. AR has the advantage over VR that the students faces aren’t occluded, and so there is a bit more collaborative learning and interaction between students, which you can see from this video of Expeditions AR:

My direct experience of seeing the Tango AR experiences at Google I/O is that the 6DoF inside out tracking is so good that it’s possible to feel a sense of virtual embodiment as you walk around virtual objects locked in space. I haven’t been able to experience this level of quality tracking in phone-based AR before, and so it was really surprising to feel how immersive it way. You’re able to completely walk around virtual objects, which triggers a deeper level of embodied cognition in being able to interact and make sense of the world by moving your body.

Embodied Cognition is the idea that we don’t just think with our minds, but that we use our entire bodies and environments to process information. I feel that the world-locking capabilities of the Tango-enable phones start to unlock the unique affordances of emobodied cognition that usually comes with 6DoF positional tracking, and it was a lot more compelling that I was expecting it to be. But after seeing the Tango demos, I feel confident in saying that AR is going to be a huge part of the future of education.

The Google Cardboard or Daydream hasn’t generated a lot of excitement from the larger VR community as they’re seen as the gateway immersive experiences to have higher-end, PC-driven experiences. But Google’s ethic of rapidly iterating and creating a minimum viable products that are highly scalable has given them over two years of direct experiences of innovating with immersive education. They’ve been able to reach over 2 million students, and they’ve also been doing a number of research pilot studies with these VR expeditions. Google researchers Matthew Kam and Jinghua Zhang presented some of their preliminary research at the IEEE VR Embodied Learning Workshop in March, and you see some of the highlights in this Twitter thread, including work that’s happening to create an immersive education primer for Circle Center.

I’m really excited to see how Google continues to innovate with immersive education, and you can look forward to seeing a solo version of Expeditions on Daydream that will be released soon that features guided tours, history lessons, and science explainers. What Google is finding is that Expeditions is not just for students, but also adults for casual and continuing education, enterprises for training applications, and even Major League Baseball have started to explore how to use immersive education experiences to engage audiences in a new way. At the end of the day, Google is showing that if you want to expand your mind and learn about the world, then Daydream & Expeditions are going to have some killer apps for you.

For more information on embodied cognition, then be sure to check these previous interviews:

You can watch the full Google Expeditions session from Google I/O here:

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

Support Voices of VR

Music: Fatality & Summer Trip

#538: How Mozilla is Driving WebVR Content & Tooling with the A-Frame Framework

dmarcosMozilla’s mission statement is to ensure that the Internet remains a global public resource, open and accessible to all, and they’ve been helping bring VR to the web for the past three years. A-Frame is an open source framework that has gained a lot of momentum over the last year with more participants on the A-Frame Slack channel than the official WebVR Slack.

Kevin-NgoI had a chance to catch up with A-Frame core developers Diego Marcos & Kevin Ngo at the IEEE VR conference in March to get an overview of A-Frame, and how it’s driving WebVR content and innovations in developer tools. Mozilla is also planning on shipping WebVR 1.1 capabilities in the desktop version of Firefox 55, which should be launching later this year in August.

LISTEN TO THE VOICES OF VR PODCAST

Mozilla believes in open source and the open web, and they have a vibrant and very supportive community on the A-Frame Slack that is very helpful in answering questions. Ngo has been curating the weekly highlights from the A-Frame community for over a year now posting the latest experiences, components, tools, and events in his Week in A-Frame series on the A-Frame blog, which has helped to grow the A-Frame community

A-Frame uses a entity-competent model that’s very similar to Unity’s model where you spatially position 3D components within a scene, and then you add behaviors and scripts that drive the interactive behavior. There’s a visual editor to move objects around in a scene, and a VR-editor is on the roadmap to be able to put together WebVR scenes in A-Frame while being in VR. There’s an open source collection of components that is being officially curated and tested in the A-Frame registry, but there’s also various collections of interesting components on GitHub repositories such as these Awesome A-Frame components or this KFrame collection of components and scenes.

Google even announced at Google I/O that they’re using A-Frame in order to rapidly prototype Google Expeditions experiences. WebVR and A-Frame is a perfect combination for Google as they’re trying to organize all of the information in the world. The strength of the open web is that you’re able to mash-up data from many different sources, and so there are going to be a lot of educational and immersive experiences focusing on mental presence are going to built on top of WebVR technologies.

In my interview with WebVR spec author Brandon Jones, he expressed caution of launching the Google Chrome browser with the existing WebVR 1.1 spec because there were a lot of breaking changes that will need to be made in the latest “2.0″ version order to make the immersive web more compatible for both virtual reality and augmented reality. Because Chrome is on over 2 billion devices, Jones said that they didn’t want to have to manage this interim technical debt and would prefer launching a version that’s going to provide a solid future for the immersive web.

Some WebVR developers like Mozilla’s Marcos and Ngo argue that not shipping WebVR capabilities in a default mainstream browser has hindered adoption and innovation for both content and tooling for WebVR. That’s why Mozilla is pushing forward with shipping WebVR capabilities in Firefox 55, which should be launching on the PC desktop in August.

Part of why Mozilla can afford to push harder for earlier adoption of the WebVR spec is because the A-Frame framework will take care of the nuanced differences between the established 1.1 version of the WebVR spec and the emerging “2.0″ version. Because A-Frame is not an open standard, they can also move faster in rapidly prototyping tools around the existing APIs to enable capabilities, and they can handle the changes in the lower-level implementations of the WebVR spec while keeping the higher-level A-Frame declarative language the same. In other words, if you use the declarative language defined by A-Frame, then when the final WebVR spec launches then you’ll just have to update your A-Frame JavaScript file, which handles the spec implementation and allows you to focus on content creation.

Mozilla wants developers to continue to develop and prototype experiences in WebVR without worrying that they’ll break once the final stable public version of WebVR is finally released. Mozilla is willing to manage the interim technical debt from the WebVR 1.1 spec in order to bootstrap the WebVR content and tooling ecosystem.

Mozilla is also investing heavily in a completely new technology stack with their Servo browser, which could eventually replace their mobile Firefox technology stack. Marcos previously told me that Servo is aiming to be built to support immersive technologies like WebVR as a first-class priority over the existing 2D web. Servo has recently added Daydream support with GearVR support coming soon. They’ve shown a proof of concept of a roller coaster Daydream app built in three.js that runs natively as a native application within Daydream.

Overall, Mozilla believes in the power of the open web, and wants to be a part of building the tools that enable metaverse that’s a public resource the democratizes access to knowledge and immersive experiences. There’s a lot of questions around concepts like self-sovereign identity, how an economy is going to powered by some combination of crytpocurrencies and the Web Payments API, as well as the concepts of private property ownership and how that might be managed by the blockchain. A lot of the concepts of a gift economy that Cory Doctorow explores in “Walkaway” are being actively implemented by Mozilla through the open source creation of the Metaverse, and everyone in the WebVR community is looking forward to a stable release later this year. For Mozilla, that begins in August with Firefox 55, but this is just the beginning of a long journey of realizing the potential of the open web.

Support Voices of VR

Music: Fatality & Summer Trip

#537: The Future of VR for Google is on the Open Web with WebVR & WebAR

brandon-jones-2017Google’s mission is to organize the world’s information, and so they’ve been long-time advocates for the open web. At Google I/O last week, they announced that they’ll soon be shipping Google Chrome for Android with WebVR, and that they’re going to start experimental builds for WebAR. During the WebVR talk at I/O Google showed how to write a progressive web application with three.js that could be viewed on a desktop computer, mobile phone or tablet, or Google Cardboard or Daydream virtual reality headset. Google is pushing the hardest for platform-agnostic WebVR applications on the web as mental presence is their core strength.

I had a chance to catch up with primary WebVR spec author Brandon Jones at Google I/O to talk about why they’re holding off on shipping WebVR 1.1 and waiting until the latest WebVR “2.0″ version is ready. It’ll ship as WebVR 1.0, but there has been such major refactoring to account for augmented reality that internally it’s referred to as the 2.0 spec. Mozilla will be shipping the 1.1 WebVR spec in their browser in August, but Jones says that the Chrome team doesn’t want to have to maintain and support the 1.1 version, which is sure to quickly be deprecated.

Jones and I talk about the differences between WebVR & WebAR, and the long process of developing the WebVR API over the last three years, VR’s relation to other exponential technologies, and the philosophy of being a immersive technology platform developer for billions of devices.

LISTEN TO THE VOICES OF VR PODCAST

Check out my previous interviews with Brandon in 2016 and in 2015.

Here’s the Building Virtual Reality on the Web with WebVR from Google I/O ’17

The Google Expeditions team is also using A-Frame to rapidly prototype Google Expeditions experiences

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

#536: “Walkaway” Author Cory Doctorow on Gift Economies & Privacy

cory-doctorowCory Doctorow’s new sci-fi book “Walkaway” is a optimistic disaster novel that imagines what society might look like if people walked away from competitive market-driven laws, norms, and technological infrastructure and into an open-source inspired, collaborative gift economy. He’s a co-founder of Boing Boing, and he uses the daily transom of tech culture and innovation as inspiration for world building fodder for his potential utopian futures. Privacy is also a recurring theme as the cyberpunks of the future have a number of different tactics for going dark and protecting their private data from being exploited.

I had a chance to catch up with Doctorow on his book tour stop in Portland, and we talked about gift economy economic paradigms, theoretical self-provisioning Spime objects that instantiate when you need them and disintegrate into elemental form when complete, the extent to which market forces continue to infiltrate all dimensions of our lives, and how we’re reaching a “peak of indifference about surveillance.”

LISTEN TO THE VOICES OF VR PODCAST

At his core, Doctorow identifies as a pulp writer who puts zeppelin rail guns battling mech warriors at the forefront of his incredibly well-specified sci-fi worlds. He’s crafted an engaging drama that’s creatively exploring new cooperative economic paradigms. He isn’t trying to present a viable hybrid synthesis of the old and the new worlds, but rather creatively explore building a greenfield world from scratch that’s based upon a post-scarcity mindset and fueled by open-source principles. It’s a world where it’s inhabitants exhibit an enlightened Buddhist level of deattachment to physical objects to the point where concepts of personal property have vanished.

Virtual Reality may prove to an experimental sandbox environment to prototype greenfield environments that could foster this type of post-scarcity gift economies. I’m already starting to see this type of gift economy exchange in virtual worlds that would be much more difficult given the constraints of real reality. You can find examples of cooperative asset sharing in Anyland, Tilt Brush worlds can be remixed, and you can mash-up A-Frame WebVR code with one-liners.

If the technical infrastructure moves to a model of distributed file storage, then this will invert the inverse relationship between supply and demand in that “the more popular something is, then the more available it becomes.” In other words, it’ll be kind of like a Bitorrent peer-to-peer network in that popular files have more seeds and are generally more available and faster to download. In other words, the more valuable an asset is, the easier it is to get ahold of it within these digital peer-to-peer sharing networks.

This is the opposite for how physical reality usually works, and so online virtual worlds will likely be the proving grounds to explore new cooperative models of yin currencies. It will also enable self-provisioning “spime” resources to arrive at the moment of need and then gracefully bow out back into the material stream of “feedstock.” Doctorow calls this the “Zip Car version of fully-automated, luxury communism.”

In this day and age, it doesn’t take a lot of imagination to come up with a variety of potential dystopian futures that could be episodes of Black Mirror. What’s harder is to come up with a number of potential utopian White Mirror futures that extrapolate current technological metaphors into the future. Doctorow is one of the most sophisticated and nuanced sci-fi world builders out there today, and he’s constructed an inspiring vision of a potential gift culture paradigm. It may be a while before the full realization is possible to achieve in real reality, but this type of world and cultural norms can start to be prototyped and built in VR today.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

#535: An In-Depth Look at the Microsoft HoloLens & Their Mixed Reality Ecosystem

Brandon-BrayThe HoloLens is the most impressive augmented reality HMD on the market today, and their developer kit is already being deployed into production in industries ranging from architecture, engineering, design, sales, medicine, and education. Microsoft is taking a holistic approach with Windows Mixed Reality being baked into Windows 10, meaning that developers can create a single application that can run either on the HoloLens, on one of their partner VR headsets, on a Surface tablet, or as a desktop app. At Microsoft Build today, they’re announcing a new OEM VR partner with HP as well as inside-out, six degree-of-freedom input controllers.

Greg-SullivanI had a chance to sit down for an hour with two representatives from Microsoft to talk about the latest HoloLens updates, their VR headsets, as well their overall Mixed Reality strategy. Brandon Bray leads the Mixed Reality developer ecosystem, and Greg Sullivan is on the marketing team for the Windows & Devices Group. We cover a lot of the high-level mixed reality strategies as well as the low-level details for developers, as well as a wide range of topics from AI integrations with Microsoft Cognitive Services to the technical details of their new motion-tracked controllers.

LISTEN TO THE VOICES OF VR PODCAST

Microsoft has leapfrogged the augmented reality competition with the combination of having the best AR HMD with the HoloLens, a healthy ecosystem of enterprise developers, a suite of AI-driven cognitive services APIs, and a forward-looking Mixed Reality strategy. They have impeccable timing with taking a leap of faith to solve a lot of really hard problems in order to have created the HoloLens in the first place. Bray admits that there’s still a lot of remaining problems to be solved with the limited field of view, but that there was a tradeoff for being able to even create a battery-driven, tetherless, holographic computing platform that you can wear on your head that can do inside-out positional tracking.

The HoloLens developer kits are priced at $3000, and so they’re targeting enterprise applications. But their VR strategy seems to be aiming for the bottom to low-end portion of the market with their $399 price point for a bundled Acer VR headset with the motion-tracked controllers. I had a chance to have some hands-on time with the Acer VR headset, and I was not impressed with the motion-to-photon latency of the LCD screen, poor quality optics, build quality, or user experience of putting the headset on.

These tradeoffs in comfort were made in order to bring the price down, but the overall experience feels like it’s a small step up from a Daydream, but perhaps on par with the GearVR or possibly even worse. The high resolution of the Acer VR headset makes it one of the best VR HMDs to read text in and the inside-out tracking works pretty well with occasional jutter. But the LCD screen is not a low-persistent screen that seasoned VR veterans have grown used to, and so the resulting DK1 or DK2 era blurring when turning your head makes it feel worse than a Gear VR. But as long as you’re not quickly moving your head around, then you’ll minimize the motion sickness triggers.

The 6DoF motion controllers are inside-out and Bray said that they rely upon a sensor fusion combination of having a direct line of sight with the front-facing cameras on the VR headset, IMU sensors, and inverse kinematic probabilities. There were not any prototypes available for testing, and so I don’t have any direct experience with how they actually work. But I do have some concerns with their approach based upon my experiences with other line-of-sight controllers such as the Leap Motion. With the Leap Motion, you have to hold your hands up so that they can be seen by the cameras on the HMD, which will likely require developers to specifically design applications that optimize for this constraint.

This limitation of the input controllers may mean that it could limit existing room-scale Vive and Rift VR experiences from being easily ported. If existing Vive or Rift applications aren’t a good experience on these lower-end VR HMDs, then there’s going to be a huge gap of content to drive consumer adoption. If this lower price point is going to attract more consumer-grade users, then they’re going to need content. If custom entertainment content is need, then I doubt that the Microsoft enterprise developers are going to generate a lot of compelling and entertaining content.

But it could be that Microsoft isn’t concerned about having a library of entertainment for regular consumers of these VR headsets, and maybe they’re more interested in creating data visualization and enterprise applications. But if that was the case, then why not create something on par with the Vive and charge enterprise prices? Most of the mobile VR content designed for a 3DoF controller hasn’t been nearly as compelling as the full room-scale and 6DoF content. These Microsoft VR headsets look to be in yet another realm of quality and performance that’s slightly better than mobile, but a lot worse than the best high-end content.

If Windows Mixed Reality VR headsets are going to go anywhere, then there’s going to need to have content that’s compelling and drives adoption. Will these VR systems meet the needs of whatever Microsoft has decided is their target market? If it does, then all of this discussion is moot. But if not, then we’ll have another platform that is fractures the developer ecosystem and is left without a critical mass of compelling content.

Overall, I’m really impressed with Microsoft’s holistic approach to mixed reality. The HoloLens is the market leader for head-mounted AR right that’s actually being deployed into production. They are positioned to really own the enterprise and professional AR market as they create more integrations between Windows Mixed Reality, their cloud hosting, and AI-driven cognitive services.

There’s a lot of long-term promise in tetherless VR with inside-out tracking, but the early Acer VR prototypes are disappointing and risk fracturing the VR ecosystem in potentially needing specially designed experiences in order to really use the strengths of the platform.

Here’s a number of Twitter threads with more thoughts and impressions from Microsoft Build so far

Live tweets of first day keynote of Microsoft Build Conference

Thread with highlights from the HoloLens YouTube channel

Twitter Thread of Hands-On Impressions from Acer headset

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

Subscribe

Subscribe with iTunes Subscribe with RSS Subscribe via Stitcher

Follow @KentBye on Twitter