Tony Tamasi is the senior vice president of content and technology at NVIDIA, and they recently announced the first steps towards supporting foveated rendering with Multi-Res Shading as well as VR SLI integration in Unreal Engine 4. I talked with Tony at the VRX Conference about NVIDIA GamesWorks VR and VR specific features like Direct to Rift support, NVIDIA’s research into advanced lightfield display technologies, and some of the unique graphics challenges that virtual reality has posed to GPU manufacturers.

LISTEN TO THE VOICES OF VR PODCAST

NVIDIA has been looking at VR and AR as a catalyst for new features and functionality, and even made some hardware changes within their Maxwell architecture to support features like Multi-Res Shading. They’re hoping that some of these features could provide enough of a performance boost such that VR enthusiasts could match the performance of a GTX 970 with a card that was below the minimum GPU specs recommended by Oculus.

I talked with Tony about some of the other VR-specific features that NVIDIA has been working on for both VR content creators as well as VR hardware manufacturers. Here’s an excerpt from their press release that dsce the VR features that we talk about on the podcast:

For game and application developers:

• VR SLI—provides increased performance for virtual reality apps where multiple GPUs can be assigned a specific eye to dramatically accelerate stereo rendering. With the GPU affinity API, VR SLI allows scaling for systems with >2 GPUs.
• Multi-Res Shading—an innovative new rendering technique for VR whereby each part of an image is rendered at a resolution that better matches the pixel density of the warped image. Multi-Res Shading uses Maxwell’s multi-projection architecture to render multiple scaled viewports in a single pass, delivering substantial performance improvements.

For headset developers:

• Context Priority—provides headset developers with control over GPU scheduling to support advanced virtual reality features such as asynchronous time warp, which cuts latency and quickly adjusts images as gamers move their heads, without the need to re-render a new frame.
• Direct Mode—the NVIDIA driver treats VR headsets as head mounted displays accessible only to VR applications, rather than a typical Windows monitor that your PC shows up on, providing better plug and play support and compatibility for VR headsets.
• Front Buffer Render—enables the GPU to render directly to the front buffer to reduce latency.

NVIDIA’s press release said that these features are more likely to land within the next Unreal Engine release of 4.11, but that an exact release version has not been confirmed yet.

Finally, I talk to Tony about balancing cooperation in order to make VR successful while at the same time still push forward with competitive advantages. While there have been a number of surprising collaborations with NVIDIA, they will be primarily focusing on innovating on their product features to keep pushing the limits of performance and what’s possible in striving towards rendering more and more realistic real-time virtual environments. For more information, be sure to check out their NVIDIA GamesWorks VR page.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Sony has been looking into virtual reality for a long time, and Sony Magic Labs’ Director Richard Marks tells me about this long history and the development of PlayStation VR. He helped to create the PlayStation Move, which is a 6 degree of freedom controller that enabled VR enthusiast developers within Sony a way to experiment with motion controllers within a VR experience. Richard’s division was in charge of the capabilities of high-end tracking technology and then creating prototypes for game developers to explore the game design possibilities. I caught up with him at the VRX conference where he shared more information on the 120 fps framerate, how the PlayStation VR tracks the headset and Move controllers, and what he sees as the vast potential of social gaming.

LISTEN TO THE VOICES OF VR PODCAST

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

James McCrae wants to lower the barrier to entry for creating virtual worlds with janusVR. He has a PhD in computer science from University of Toronto, and he’s created a sophisticated technology stack that allows front-end web developers to rapidly prototype interactive virtual worlds using a simple HTML-like markup language. These worlds are connected to each other through portals and provide an early look at what the metaverse might feel and look like.

LISTEN TO THE VOICES OF VR PODCAST

I caught up with James at the VRX Conference in San Francisco, and he talks about his vision and inspiration for creating janusVR. He wanted a simple markup language as well as the ability to drag and drop 2D or 3D content into a world while you’re in the world. He even has an in-world IDE where you can write code to create the virtual world while being within that same virtual world.

It’s amazing to see what type of creativity and experiences that have already started to be created within janusVR, including a replica of an A-10 airplane. There are also users who make videos of their guided tours of different janusVR destinations, as well as popular social hangout locations like The VR Bar.

It’s still pretty early for any type of open standards and single technology stack for determining how the metaverse is going to be built, but janusVR is helping lead the way by releasing some of their tools under the MIT license. High Fidelity has also been open sourcing their tools, but there are currently a lot more active janusVR virtual worlds that have already been creating.

James has a vast vision for what’s possible with the metaverse, and his approach has been to just dive in and start making it happen. You can check out this video to see what’s already possible with janusVR:

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Presence Capital is a new early-stage venture capital firm that’s focused on funding virtual reality and augmented reality startups. I had the chance to interview one of the co-founders Amitt Mahajan at the VRX Conference about this new fund and how he plans on applying the lessons he learned from the causal, social & mobile gaming market to VR.

LISTEN TO THE VOICES OF VR PODCAST

Amitt Mahajan was a part of the early development of mobile and social gaming as the CTO of MyMiniLife, which was technology that was acquired by Zynga and used within FarmVille.

Amitt says that there’s a lot of parallels between what happened in social and mobile platforms, and what they expect to happen within VR. For example, in the early days of mobile gaming, the free-to-play business model was still evolving through in-app purchases and ad network models. But now that these have been established and proven to work, then we have the benefit of hindsight as we move into mobile VR games.

Amitt says that it comes down to user expectations when thinking about the differences between causal mobile games and a room-scale VR experience. Not only are the price points a lot different, but there is not going as much tolerance for ads and in-app purchases in a room-scale VR experience because it’s seen as a more premium and high-fidelity experience where you pay more upfront and don’t think about micro-transactions.

Another huge lesson that comes from mobile gaming is how it’s the early movers who are able to create a platform and network of users that can be used to release a series of games and not just a single release. A good example is King who were building Facebook applications like Candy Crush, and then cross promoting their mobile applications as well as new titles. Venture capital firms like Presence Capital are less interested in funding one-off games, but will be looking towards funding strong teams who are able to create a sustainable series of titles that build of one another.

Amitt talks about what he looks for in funding companies, how he sees advertising may play out in VR, and some of his favorite VR experiences including Epic’s Bullet Train and Oculus’ Toy Box demo showing the potential of social VR. He’s also really excited about the potential of productivity apps in VR as well as how VR can revolutionize education and training. Finally, he sees that there will be a lot more market opportunity for augmented reality in the long-run, even though it may be another year or two before that market will be ready for consumers.

Presence Capital announced an inaugural fund of $10 million and has already invested in VR and AR companies including Harmonix (makers of the game Rock Band), Baobab (story and character-driven cinematic experiences in VR), Waygo (Automatic translation of Asian languages to English), as well as some other unannounced companies.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Shauna Heller worked as a developer relations specialist focusing on non-gaming and non-entertainment experiences at Oculus for a year and a half before leaving to start her consultancy Clay Park VR. She’s had a lot of exposure to the leading non-gaming VR experiences and understands what the how the full landscape of VR is shaping up and what the early adopters of VR are doing. I had a chance to catch up with Shauna at the VRX conference to get more insights for how VR is going to go beyond this initial push of gaming and entertainment.

LISTEN TO THE VOICES OF VR PODCAST

Spending a year and a half at Oculus in developer relations has given Shauna Heller some unique insights into how the rest of the world is thinking about VR. She has seen a lot of interest in VR across arts and culture, aerospace, architecture, automotive, education, enterprise, medical, music, publishing, retail, sports, themed entertainment, and movie visual effects. Those are just the early movers in the VR space, and she says that this is really just the tip of the iceberg since she’s hearing about a new application of VR every day.

She left Oculus and started Clay Park VR in order to help advise and educate clients about the broader issues around VR who are not sure what to do with VR and how to engage with this new medium. She also is able to provide some unique feedback on what’s been happening within the broader market across all of the different industry verticals.

In terms of her personal interest in VR, she’s a huge proponent of educational VR experiences and sees it as one of the most important potentials of VR. Not everyone is a book learner, and she sees that VR could help us unlock the full potential of our learning capabilities and knowledge in a way that could profoundly change humanity.

Some of her favorite VR experiences include Apollo 11 VR experience as well as First Flight, which is about this history of the Wright Brother’s airplane that was commissioned by the Smithsonian. She wants to see more educational experiences like these that are able to educate without over-indulging, overstimulating or overreaching.

One of the things that I’ve noticed in educational VR experiences is that they tend to rely heavily on narration or infographic text popping up. The thing that I wonder is how VR can go beyond what a film can do, and what the best use of the virtual reality medium will be to provide an embodied educational experience that goes beyond what instructional videos can do. Shauna agrees that this is a tricky balance as to how to balance the environmental visual information, any audio narration, and then any additional text or information provided on the screen. There’s not a set formula for that perfect balance, and so it’s a process of experimentation and knowing a good balance when you see it. It’s an ideal that even her most favorite experiences have yet to fully reach.

One of the biggest mistakes that Shauna has seen in experiences is when developers try to use all 360 degrees to no effect. Requiring viewers to quickly look all around them even if it’s not really compelling to do so can annoy or strain the viewer. The stronger developers have been finding ways to best use this new immersive medium with meaningful applications. She also recommends spending more time on implementing a spatialized audio design since it’s crucial for taking a VR experience to the next level.

Finally, Shauna says that now is the time to get into VR development. It’s still so early in the development of VR that we’re still in the process of unlocking new pathways of discovery with the medium. Ultimately, she sees that VR is going to enable us to engage, enlighten, educate and explore. There’s going to be applications of VR in nearly every aspect of our society, and she’s looking forward to helping guide companies along the way with her work at Clay Park VR.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Brian Blau is the research director in the personal technologies group at Gartner specializing in mobile apps and video game software in the consumer domain. He’s also been analyzing the market for virtual and augmented reality for the past five years, but has been involved in VR since the 1980s. Gartner is a technology advisory service and so they’re in the business of predicting the future. A lot of their predictions and reports are not released to public, but Brian was able to hint at some previously made predictions as well as compare their analysis with some of the other analysts who have been estimating the overall market size for virtual and augmented reality.

LISTEN TO THE VOICES OF VR PODCAST

Brian is in the business of making predictions about the future, and so this was a really fascinating interview to get some insights about the significance of how VR and AR is going to change our relationship to technology and other people. The big point that he wanted to emphasize is that immersive computing represents an entirely new user interface and human interaction design that is way different than anything else that we’ve seen before.

He says that the transition into immersive computing is significantly more different than the transition from typing on a desktop computer to touching a smartphone or tablet. Also, this new immersive computer paradigm is going to change everything from how we interact with technology to how we interact with other people.

It was also fascinating to hear how some other emerging technologies are going to combine with VR and AR. He sees that virtual personal assistants, like we saw in the movie Her, are going to start assisting us on our daily tasks and perhaps start doing things on our behalf. Then when you combine a virtual personal assistant with computer vision technologies and the Internet of Things, then you start to create what they call “ambient experiences.” He expects that we’ll start to have rooms with sensors so that these artificial intelligent programs can help us with a lot of different things in our daily lives.

In terms of making some VR and AR market size analysis, Brian couldn’t share Garter’s latest predictions. However, he was able to provide some older predictions that have been made public as well as compare their predictions to some other analysts. About a year ago, Gartner predicted that there would be 25 million HMDs by 2018, but they’ve since updated their prediction to be more than that. Brian also said that there are other analysts who are saying that the VR market is going to be $100-200 billion within the next 4-5 years, but Brian says Gartner’s prediction is more conservative than that. He also says that the compound annual growth rate of the VR market is expected to be well over 100%, which indicates that VR will continue to be a high-growth market.

Gartner does a combination of top-down research in to the supply side and major HMD manufacturers, as well as bottom-up research by doing surveys to see what type of experiences that the market is going to be demanding using VR. They’ve been producing reports on the competitive landscape of the three major HMD manufacturers, but now they’re starting to report on some of the use cases ranging from training, prototyping, design, and beyond. What they’re seeing is that the demand for VR is pervasive, and that the media coverage for VR trending really high.

One of the things that Gartner invented was to the hype cycle journey of different distinct periods of hype and overblown expectations that new emerging technologies go through over the course of five to 20 years. After an initial “innovation trigger” period, then there’s a period of “peak of inflated expectations” that can be seen as the height of the hype cycle. The media goes crazy with a lot of hopes and dreams of the promise and potential of the new technology, and then reality often sets in and there’s a “trough of disillusionment”. It’s only after passing through that reality check period that technology starts to climb back up the “slope of enlightenment” and then into the “plateau of productivity.”

Brian maintains the virtual reality dot on this hype cycle graph, and he says that it’s just about to cross over from the “trough of disillusionment” and into the “slope of enlightenment.” VR has had a long history where the peak of inflated expectations probably hit in the early 90s when the promise of the potential of VR was about 20 years ahead of where the technology and ecosystem of content creators needed to be. From this perspective, then we can start to see a lot more pragmatic and practical applications of VR being talked about in the media as it’s slowly accelerating towards it’s plateau of productivity. VR has had a long journey relative to other technologies, and Brian says that they’ve artificially slowed down the normal 5-20 year hype cycle journey for VR to more accurately account for where it’s currently at.

It’s interesting to note that the augmented reality dot is lagging behind the virtual reality dot as it’s just starting to really enter in the bottom of the trough of disillusionment. Brian estimates that AR is at least a year behind where VR is at, and perhaps more.

Brian has also covered the evolution of cell phones and the smartphone market, and compares and contrasts what was happening there with how he expects the VR and AR markets to unfold. There’s a lot that is different with the infrastructure that was already in place with cell phones and Apple’s role as playing the market hero, but there’s also a lot of similar parallels in terms of the importance of a software and app ecosystem that differentiates the phones. He expects to see a VR and AR HMD battle and an ecosystem battle that will be continuing to evolve and play out.

Finally, Brian sees that we will have access to these wearable computing AR and VR devices that really match the human perceptual system for the next-level of immersion and how we experience each other through technology. Right now we have a pretty good sense of what’s real and what’s not real. And Brian says that AR and VR are going to blend that line, and it’s going to fool us. And we’re going to really love it.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

The Night Café is an amazingly surreal immersive VR experience of walking through a space that’s inspired by a painting by Vincent van Gogh. I speak with Mac Cauley about his inspiration and process to in creating The Night Café, which won both the Platinum and Community Choice awards in the experience category for the Oculus Mobile VR Jam 2015. It’s one of my favorite VR demos to show people in order to introduce them to VR because it really creates a sense of presence in a place that’s really only possible in VR. Mac talks about his design process for The Night Café (be sure to check out his two detailed behind-the-scenes blog posts #1 & #2), his experience of using Oculus Medium, as well as what he’s working on next.

LISTEN TO THE VOICES OF VR PODCAST

The Night Café: An Immersive Tribute to Vincent van Gogh is available for free on the Gear VR store as well on the Oculus Share website.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Maureen Fan is the former vice president of gaming at Zynga, but she’s always had a passion for animation. Maureen’s mind was blown open the first time she experienced VR because she saw that the future of animation was going to be within Virtual Reality. She left Zynga about six months ago, and within a couple of weeks met her Baobab Studios co-founder Eric Darnell who directed Antz as well as wrote and directed four Madagascar films. They quickly assembled a team of other animated film veterans and put together a 2-minute trailer that helped them to secure $6 million dollars led by Comcast Ventures with participation from HTC and Samsung Ventures as well as others.

LISTEN TO THE VOICES OF VR PODCAST

Baobab Studios co-founder Eric Darnell has written a number of animated feature films, and has already written a number of different scripts for animated shorts that take place within immersive virtual reality. The first production from Baobab Studios is Invasion! which features an alien and bunny as the main characters. They rendered out two-minute 360-degree video which can be found on Milk VR on the Gear VR, but Maureen says that they have also produced a more interactive version that can be run on the Oculus Rift.

HTC was also an investor into Baobab Studios, and Maureen says that do intend on creating animated shorts that are best suited to the VR medium implying that they are indeed also working on creating room-scale animated experiences.

One of the open challenges that Oculus Story Studio has talked about at Oculus Connect as well as written about on their blog, is the challenge of balancing cultivating a sense of presence with narrative. Maureen doesn’t think that they necessarily have to be in conflict with each other and says that they have some ideas for how they’re going to find a balance between these, but didn’t provide any concrete specific details as to how or what they plan on doing.

Another big open question that Baobab Studios will have to address is what the revenue stream is going to be for producing short animated 360-degree videos as well as some more interactive immersive stories. Maureen says that one thing that she will be bringing to the table from her experiences at Zynga is a dedication to testing and using objective behavioral data from the audience in order to find a sustainable business model. Zynga was a pioneer of causal gaming, and was driven by testing and finding a sustainable minimization strategy based upon a free-to-play model and different ways to either speed up gameplay or extend the experience in different ways. Baobab Studios has a number of different hypotheses for a viable business model that they will be testing as the ecosystem of distribution starts to get fleshed out.

Maureen says that the underlying motivation and mission statement of Baobab Studios is to bring out your sense of wonder and to inspire people to dream. They hope that their shorts can bring out that childlike sense of wonder and awe in people who watch their shorts, and they’ve been able to recruit an impressive team of creatives who are excited to translate their visions into actual virtual worlds, characters, and stories. They still have to discover and form the cinematic language of VR given the new constraints that the audience will be shaping a lot of the experience based upon what they’re looking at and where they’re looking.

For Maureen, she spent some time at Pixar, but wanted to have more control and freedom that a start-up would provide and decided to go to Zynga. She also spent some of her free time working with some friends on the animated short called The Dam Keeper, which ended up getting nominated for an Academy Award. But after seeing virtual reality, she knew that she wanted to start her own company to start to chase after her lifelong dream of bringing animated stories to life. They’re starting with short films, and the full Invasion! experience should be ready sometime in the first quarter of 2016.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Pete Moss is Unity’s “VR Dude” in that he’s been a long-time believer and advocate in the cross section between VR and gaming and how the Unity game engine could play a significant part. He talks to me about his dual role of content creator and dogfooding the Unity engine as well as evangelizing VR to different communities and collecting experience prototypes from different game jams and developers from around the world. Pete’s passion for the potential of virtual reality is clear, and he’s the most excited to see how artists use VR to express themselves given that VR provides completely new pathways into the brain.

LISTEN TO THE VOICES OF VR PODCAST

As part of Pete’s job, he has been going around to different conferences around the world to keep tabs on where the industry is going. He’s starting to talk at more non-gaming gatherings such as different events in Hollywood as the film industry is starting to look into how to best use this new immersive medium for storytelling and entertainment. But he’s also been available at different game jams as a Unity expert as well as witness to the imagination and creativity for how different developers have created

One of the examples that he provided was being at the Boston game jam with an early prototype of Valve’s HMD and hearing Ben Throop talk about his desire to create a soccer game where you head a soccer ball into the goal. Ben went on to finish that prototype, and eventually have Headmaster be selected by Sony to be a release title for the PlayStation VR, which is having their PlayStation Experience this coming weekend.

Here’s a video of Pete helping to launch soccer balls to Valve’s Chet Faliszek and Aaron Leiby with the first working prototype for Headmaster

I really see Unity as the “lingua franca” of these immersive technologies. It’s pretty clear that the majority of VR experiences have been created within the Unity game engine, and Unity will also be a key to bringing immersive experiences to AR systems like Microsoft’s Hololens, Meta, and Magic Leap.

The PR firm representing Unity followed up with me after this interview, and passed along some of these other statistics in terms of how widely Unity is being used.

• Unity accounts for 95% of all virtual reality content developed for platforms such as the Samsung Gear VR, Oculus Rift, HTC Vive, and Sony PlayStation VR
• On the Oculus Gear VR Store, Unity accounts for 88% of the games
• 174,000 games (not just VR) were made with Unity last quarter
• 5 million people in the world are currently using and creating on Unity
• 465 engineers are working on making a better 3D engine every day

Unity’s biggest competitor in the VR space has been Epic’s Unreal Engine, and I wouldn’t be surprised to see Crytek’s CryENGINE start to move towards more of an open and free model based upon what was hinted to me at the Virtual Reality Intelligence gathering. But whether or not these numbers end up being 95% or 85% of the market share, it’s clear that Unity is a significant player when it comes to providing the content creation tools for VR game and experience developers to manifest their creative visions.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Sophia Dominguez is going all in with virtual reality, and she talks her journey of making virtual reality a career. She’s the author of the weekly AllThingsVR newsletter, and she has been working as the entrepreneur in residence at the Rothenberg Ventures River Program. She was inspired by M. T. Anderson’s Feed inspired her to get into technology in order prevent a sci-fi dystopian future, and eventually traveled to a number of countries showing people Google Glass for the first time in order to counter the negative press it had been receiving. Sophia is really interested in social VR and she shares her experience of the Toybox demo from Oculus Connect 2 as well as her vision for her own social video viewing experiences using WebVR. Sophia has a lot of passion for the potentials of these immersive technologies, and is making a lot of strategic moves to be involved with VR for a very long time.

LISTEN TO THE VOICES OF VR PODCAST

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.