#535: An In-Depth Look at the Microsoft HoloLens & Their Mixed Reality Ecosystem

Brandon-BrayThe HoloLens is the most impressive augmented reality HMD on the market today, and their developer kit is already being deployed into production in industries ranging from architecture, engineering, design, sales, medicine, and education. Microsoft is taking a holistic approach with Windows Mixed Reality being baked into Windows 10, meaning that developers can create a single application that can run either on the HoloLens, on one of their partner VR headsets, on a Surface tablet, or as a desktop app. At Microsoft Build today, they’re announcing a new OEM VR partner with HP as well as inside-out, six degree-of-freedom input controllers.

Greg-SullivanI had a chance to sit down for an hour with two representatives from Microsoft to talk about the latest HoloLens updates, their VR headsets, as well their overall Mixed Reality strategy. Brandon Bray leads the Mixed Reality developer ecosystem, and Greg Sullivan is on the marketing team for the Windows & Devices Group. We cover a lot of the high-level mixed reality strategies as well as the low-level details for developers, as well as a wide range of topics from AI integrations with Microsoft Cognitive Services to the technical details of their new motion-tracked controllers.

LISTEN TO THE VOICES OF VR PODCAST

Microsoft has leapfrogged the augmented reality competition with the combination of having the best AR HMD with the HoloLens, a healthy ecosystem of enterprise developers, a suite of AI-driven cognitive services APIs, and a forward-looking Mixed Reality strategy. They have impeccable timing with taking a leap of faith to solve a lot of really hard problems in order to have created the HoloLens in the first place. Bray admits that there’s still a lot of remaining problems to be solved with the limited field of view, but that there was a tradeoff for being able to even create a battery-driven, tetherless, holographic computing platform that you can wear on your head that can do inside-out positional tracking.

The HoloLens developer kits are priced at $3000, and so they’re targeting enterprise applications. But their VR strategy seems to be aiming for the bottom to low-end portion of the market with their $399 price point for a bundled Acer VR headset with the motion-tracked controllers. I had a chance to have some hands-on time with the Acer VR headset, and I was not impressed with the motion-to-photon latency of the LCD screen, poor quality optics, build quality, or user experience of putting the headset on.

These tradeoffs in comfort were made in order to bring the price down, but the overall experience feels like it’s a small step up from a Daydream, but perhaps on par with the GearVR or possibly even worse. The high resolution of the Acer VR headset makes it one of the best VR HMDs to read text in and the inside-out tracking works pretty well with occasional jutter. But the LCD screen is not a low-persistent screen that seasoned VR veterans have grown used to, and so the resulting DK1 or DK2 era blurring when turning your head makes it feel worse than a Gear VR. But as long as you’re not quickly moving your head around, then you’ll minimize the motion sickness triggers.

The 6DoF motion controllers are inside-out and Bray said that they rely upon a sensor fusion combination of having a direct line of sight with the front-facing cameras on the VR headset, IMU sensors, and inverse kinematic probabilities. There were not any prototypes available for testing, and so I don’t have any direct experience with how they actually work. But I do have some concerns with their approach based upon my experiences with other line-of-sight controllers such as the Leap Motion. With the Leap Motion, you have to hold your hands up so that they can be seen by the cameras on the HMD, which will likely require developers to specifically design applications that optimize for this constraint.

This limitation of the input controllers may mean that it could limit existing room-scale Vive and Rift VR experiences from being easily ported. If existing Vive or Rift applications aren’t a good experience on these lower-end VR HMDs, then there’s going to be a huge gap of content to drive consumer adoption. If this lower price point is going to attract more consumer-grade users, then they’re going to need content. If custom entertainment content is need, then I doubt that the Microsoft enterprise developers are going to generate a lot of compelling and entertaining content.

But it could be that Microsoft isn’t concerned about having a library of entertainment for regular consumers of these VR headsets, and maybe they’re more interested in creating data visualization and enterprise applications. But if that was the case, then why not create something on par with the Vive and charge enterprise prices? Most of the mobile VR content designed for a 3DoF controller hasn’t been nearly as compelling as the full room-scale and 6DoF content. These Microsoft VR headsets look to be in yet another realm of quality and performance that’s slightly better than mobile, but a lot worse than the best high-end content.

If Windows Mixed Reality VR headsets are going to go anywhere, then there’s going to need to have content that’s compelling and drives adoption. Will these VR systems meet the needs of whatever Microsoft has decided is their target market? If it does, then all of this discussion is moot. But if not, then we’ll have another platform that is fractures the developer ecosystem and is left without a critical mass of compelling content.

Overall, I’m really impressed with Microsoft’s holistic approach to mixed reality. The HoloLens is the market leader for head-mounted AR right that’s actually being deployed into production. They are positioned to really own the enterprise and professional AR market as they create more integrations between Windows Mixed Reality, their cloud hosting, and AI-driven cognitive services.

There’s a lot of long-term promise in tetherless VR with inside-out tracking, but the early Acer VR prototypes are disappointing and risk fracturing the VR ecosystem in potentially needing specially designed experiences in order to really use the strengths of the platform.

Here’s a number of Twitter threads with more thoughts and impressions from Microsoft Build so far

Live tweets of first day keynote of Microsoft Build Conference

Thread with highlights from the HoloLens YouTube channel

Twitter Thread of Hands-On Impressions from Acer headset

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

#534: Microsoft Technical Evangelist on Presence in VR Game Design

The Microsoft Build conference starts this week, and I expect that we’ll be learning more about the HoloLens as well as Microsoft’s Windows Mixed Reality operating system. Microsoft has been evangelizing about virtual reality the past couple of years now, and I had a chance to catch up with Microsoft Technical Evangelist Kat Harris at PAX West last Fall. She’s been teaching VR 101 development courses at different conferences, and we talk about what she’s been telling game designers about maintaining presence, building immersive experiences, and how to deal with some of the biggest breakers of presence including VR locomotion and the uncanny valley. We also discuss the limits of virtual reality when it comes to haptics, uncanny narrative, the future of artificial intelligence in enabling collaborative role-playing, and the power of world building & storytelling in games like Minecraft.

LISTEN TO THE VOICES OF VR PODCAST

PAX West is great place to play social games like Johann Sebastian Joust, where the biggest aspect of this gameplay is being able to control your sense of embodied presence. A game like this would translate well in mixed reality with other co-located people, but it would be nearly impossible to translate this gameplay into a distributed virtual reality game.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

#533: High Fidelity is Architecting for VR Privacy with Self-Sovereign Identity

philip-rosedalePhilip Rosedale has been thinking deeply about how to architect large-scale, distributed virtual worlds after experiencing many bottlenecks in running Second Life. High Fidelity is taking a much more distributed approach with how it’s being developed openly in open source using Worklist.net contractors, how it plans on distributing hosting and compute resources to user’s computers, as well as using a decentralized identity based upon the blockchain. Rather than having a centralized authority for tracking and data mining an individual’s identity, they’re planning on using what’s called “Self-Sovereign Identity”, which Christopher Allen explains in great detail in his comprehensive essay titled A Path to Self-Sovereign Identity.

I had a chance to catch up with Rosedale at the 4th Silicon Valley Virtual Reality conference where we talked about distributed identity, privacy in VR, High Fidelity business model based upon sales tax, whether existing cryptocurrencies will work for them, yang and yin currencies, and their open source development process. High Fidelity is architecting a lot of the open standards for the future of the metaverse, and Rosedale is one of the most deep and profound thinkers in the virtual reality space. He’s ahead of his time in architecting virtual worlds that will be able to democratize space and disrupt travel.

LISTEN TO THE VOICES OF VR PODCAST

May 5th, 2017 also marks the three-year anniversary for the Voices of VR podcast, and this is a fitting podcast as I started the bulk of my interviews at the very first Silicon Valley Virtual Reality Conference in 2014 and I’ve been able to talk to Rosedale at each of the last four SVVR gatherings. You can check out my previous interviews with Rosedale in episodes #25, #173, and #376.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

Support Voices of VR

Music: Fatality & Summer Trip

#532: The Extremes of Mental Presence: Cognitive Enhancement, Biohacking, Psychedelics, & Transhumanism

eric-matznerEric Matzner identifies as a Techno-Optimistic Futurist who sometimes works an entire second workday within VR exploring different ways to expand his mind. He’s pushing the limits of mental presence by using nootropic supplements for cognitive enhancement from his company Nootroo, experimenting with psychedelics and VR with the psychonauts from /r/RiftIntoTheMind, exploring sensory addition & biohacking with the Northpaw device, using EEG sensors and VR to have an embodied experience of neurofeedback, teaching himself how to type with one hand with a chorded Twiddler3 Keyboard, and Rapid Serial Visual Presentation to read up to 1000 words a minute using Virtual Desktop.

I had a chance to catch up with Matzner at an UploadVR party during GDC where he shared his explorations into the extremes of mental presence. For me, I’m interested in the balance between mental & social presence with embodied presence, emotional presence, and active presence. But Matzner wants to push the limits of what types of experiences are possible as well as use VR to help him focus on his learning practices.

LISTEN TO THE PODCAST

Part of the extremes of the air element is the risk of disassociation from his body and the real world, and Matzner is openly wondering whether or not reality is going to be able to keep up with how compelling VR is to him. He’s a part of an emerging group of Transhumanist Merry Pranksters of Silicon Valley experimenting with immersive technologies and daily habits to optimize productivity and happiness.

Links and show notes:

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

Support Voices of VR

Music: Fatality & Summer Trip

#531: Tony Parisi is Leading Unity’s VR & AR Strategy beyond Gaming

tony-parisiThe keynote from Unity’s Vision VR/AR Summit yesterday showcased just how far Unity is reaching into non-gaming content when it comes to augmented and virtual reality. The person who is in charge of Unity’s xR strategy is Tony Parisi, who is a co-founder of VRML and a long-time open web advocate. Tony has long been interested in using VR for artistic expression and storytelling, and the keynote speakers highlighted the range of diversity of immersive technologies ranging from NASA JPL to car companies to the NFL to graphic novel comic books to immersive storytelling to construction to the big tech players including Facebook, Google, and Microsoft.

I had a chance to catch up with Tony at Sundance earlier this year to talk about his approach to leading VR & AR strategy at Unity, and moving immersive technologies beyond just entertainment. Anyone who knows Tony can say that he wouldn’t have taken this job at Unity if there wasn’t some long-term open web strategy involved, but he wasn’t prepared to provide any specifics on it yet. But it’s safe to assume that it’s on the roadmap, especially with the recent news that the co-creator of WebGL & WebVR, Vlad Vukicevic recently joined Unity’s emergent technology group.

LISTEN TO THE VOICES OF VR PODCAST

Here’s the two and a half hour Unity Vision VR/AR Summit 2017 Keynote

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

Support Voices of VR

Music: Fatality & Summer Trip

#530: Upload LA Opening Represents a Focus Shift From VR Tech to VR Content

freeman-masonUploadVR opened up their new 20,000 square foot Upload LA co-working space on April 13, 2017. I had a chance to catch up with co-founders Taylor Freeman and Will Mason as VRLA to get the inside scoop. Mason told me that opening up this huge co-working space in Los Angeles represents an energetic shift from the VR technology born out of Silicon Valley to focusing on content & storytelling in VR.

LISTEN TO THE VOICES OF VR PODCAST

Upload VR’s first co-working space, The Upload Collective, was opened in San Francisco during GDC in 2016, and Mason said that they quickly outgrew the space. The Upload LA space will allow them to have a lot bigger educational and training initiatives without having to completely rearrange their space for every event. Upload LA is a vertically-integrated co-working space that has access to hardware, mixed reality studio, audio mixing studio, and immersive entrepreneurs that hopes to incubate many smaller start-ups and content studios. It’s proven to be a winning combination for UploadVR considering that all of their dedicated office space was accounted for just a couple days after launching, and they’ll have floating desk space available until it runs out.

UploadVR plans on continuing to grow and expand their physical spaces to new markets, but the space in Los Angeles will be sure to hold many events, talks, and parties for the years to come. You can get a sense of the space from this launch video as well as some photos of their space after their grand opening.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

#529: VR Storytelling Highlights from Tribeca Film Festival

loren-hammondsThe Tribeca Film Festival featured over 30 different VR experiences within their Storyscapes and Immersive Virtual Arcade, and I had a chance to catch up with curator Loren Hammonds about some of the highlights of the festival program with genres spanning from live-action narrative, animated narrative, documentary, interactive installations, guided tours, empathy pieces, and even a couple of immersive theater, mixed reality pieces. The overall focus and theme that connected all of the VR pieces is storytelling, both in terms of strong storytelling execution as well as in innovations around interactive storytelling.

LISTEN TO THE VOICES OF VR PODCAST

Some of my personal favorite pieces included a immersive theater, mixed reality piece called with a live actor Dram Me Close. The Last Goodbye was an incredibly powerful tour of a concentration camp by a Holocaust survivor that pushes innovations around best practices in volumetric storytelling using photogrammetry and stereoscopic video capture. Other documentary stand-outs include Step to the Line as well as Testimony, which used an innovative non-linear structure to feature direct testimony about experiencing sexual assault. One of the best narrative shorts was Alteration, which used AI-processing techniques on the 360 video to great effect. My favorite animated short was APEX, which is the latest music video by the creator of Surge. I also had some great interviews with the creators of Blackout, Treehugger, Tree, The Island of the Colorblind, Auto, Bebylon Battle Royale, Becoming Homeless, The People’s House, Remember: Remember, Falling in Love, and Beefeater XO.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

#528: Geospatial AR Permissions with the Mixed Reality Service Spec

MarkPesceMark Pesce is a VR pioneer who has been thinking about networked virtual and augmented reality for over 20 years now. He developed the Ono-Sendai Sega VR helmet prototype, co-created VRML, and presented his Cyberspace Protocol spec at the first Web conference in 1994. This CP spec evolved into the Mixed Reality Services spec, which aims to be a distributed system that would grant geospatial permissions for mixed reality applications.

This system would be an open way of preventing AR games from being played at culturally sensitive locations, but also provide Universal Resource Identifiers to bring the open web to the real world. It could provide permissions for airspace & drones, surveillance permissions, AR game permissions, hazmat warnings, electrical and plumbing layouts, and hours of operations for buildings.

I had a chance to catch up with Pesce where he gave me a history of his work on the canceled SEGA VR helmet, VRML, and the evolution of the Mixed Reality Service. We also talk about his first ritual in VR and Technopagan explorations, as well as his thoughts on ethics in overall tech industry.

LISTEN TO THE VOICES OF VR PODCAST

Related Links

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

#527: Turning Your Torso Into an Ear: Sensory Substitution & Addition with Neosensory

David-EaglemanBack in 2015, Neuroscientist David Eagleman gave a TED talk about the potential to expand and create new senses. He showed off a haptic vest prototype that could translate audio input into an array of 32 vibrating motors that could be fed directly into your body. The signals would reach the brain and create a neural input that is indistinguishable from what the cochlea would produce, meaning that it’s possible to turn the torso into an ear. The principle of sensory substitution shows that the brain doesn’t care where the data comes from as long as it’s structured in the right format and correlated to feedback within the environment.

I had a chance to catch up with Eagleman at the Experiential Technology Conference to talk about Neosensory’s VEST (Versatile Extra-Sensory Transducer), the hard problem of consciousness and how reality is constructed in the mind, expanding and creating new senses, invasive neural interfaces to the brain from Kernel, the philosophical implications of simulation theory, and his metaphors for how he understands the relationship between the mind and the body.

LISTEN TO THE VOICES OF VR PODCAST

Eagleman co-founded a company called Neosensory that is creating this vibratory VEST, and they’re reaching out to developers to see what kind of applications this could have. He’s particularly curious about whether it’s possible for humans to create entirely new senses by feeding data streams into the body about imperceptible environmental data, or perhaps even abstracted data from the stock market.

Eagleman says that the body is not great at handling redundant data, and so people who are deaf learn to use the VEST a lot faster than someone who can already hear. Learning to understand data from the VEST completely happens on an unconscious level, which they can objectively verify happens by seeing consistent improvements through many repetitions. Eagleman hypothesizes that it is possible to create new senses, and that the expansion of our biological capabilities with technology will expand the range of human sensory experience.

There’s a number of philosophical questions about the nature of consciousness, the philosophy of mind, as well as ethical questions around the extend that this should be used. Just because we can create new senses, should we? What are the emotional and mental health tradeoffs of feeding mentally abstracted data directly into our bodies? There’s also a lot of potential benefits like could this be used to feed emotional and biometric data from other people so that we can cultivate a deeper sense of empathy and connection with other consenting adults.

Sensory replacement and sensory addition is one of the most profound implications of virtual reality technologies, and Eagleman makes the point that the extent of our experiential reality is constrained by our biological limitations. By using this type of technology from Neosensory to expand our range of sensory experiences, then it’s changing and evolving what the dynamic range of the human experience.

Here’s Eagleman’s TED talk from March 2015 talking about the potential to create new senses

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

#526: YouTube VR Wants to Find the Next Billion Dollar Genre That Hasn’t Been Created Yet

Jamie-ByrneJulia-Hamilton-Trost
At Sundance this year, I had a chance to catch up with a couple of representatives from Google to talk about what’s happening on the YouTube VR platform with 360 videos. I talked with Jamie Byrne, YouTube’s Director of Global Creator & Enterprise Partnerships as well as Julia Hamilton Trost, Google VR’s Business Development & Content Partnerships. We talked about their YouTube VR application, what they’re doing to do to empower content creators, how they 360 video as a gateway into higher end VR, and some of the potential future to add more volumetric and interactive elements to the YouTube platform in the future.

LISTEN TO THE VOICES OF VR PODCAST

Here are a number of 360 videos that were mentioned in this interview.

The Unboxing Time Machine – NES 1985

Rhomaleosaurus: Back to Life in Virtual Reality #PreviouslyOnEarth

The Dropper – A Minecraft 360° Video

Meredith Foster giving a 360 tour of her apartment

New York Times is doing a Daily 360 video

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

Support Voices of VR

Music: Fatality & Summer Trip

Subscribe

Subscribe with iTunes Subscribe with RSS Subscribe via Stitcher

Follow @KentBye on Twitter