Aaron-LemkeAaron Lemke of Unello Design talks about his VR development strategy where he’s prototyping and quickly iterating and refining a VR experience first for Google Cardboard, then updating it for Gear VR, and then increasing the graphical fidelity to create a release for the Oculus Rift.

The tagline for Unello Design is “Transcendence through Virtual Reality”, and so a lot of Aaron’s experiences are very relaxing and meditative. He’s probably most well known for Eden River, where you simply float down a river within VR and lean side to side in order to stear. He adds his own musical compositions, and then uses VR as a sort of music video and immersive experience to give people a feeling that goes beyond what a 2D medium can accomplish.

Some of his other experiences include Zen Zone, which uses the concept of mirror neurons to be able to watch a digital avatar in front of you and then focus on the highlighted areas of your body. Other experiences include Opera Nova, Nebuland, and Lunadroid 237. Aaron has also been exploring the possibility of using some of his experiences within a medical context for pain management.

Aaron also recently participated in the Austin Vive Jam, where he created a snake game that is kind of like operation for a room-scale maze where you’re trying to avoid touching virtual walls. Given that a Lighthouse tracked room is only 12 feet by 9 feet, then their design intention was to create an experience that turned people around and disoriented them so much that they forget where they’re located in the real physical room. By all accounts, they were able to achieve that.

Aaron also talks about some of the other highlights from the the Vive jam as well as from the SVVRCon.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Tomas-MariancikTomáš Mariančík (aka Frooxius) is one of the most innovative and mad genius VR developers that I’ve met. He is probably the most well known for SightLine: The Chair VR demo, which uses the mechanism of change blindness to cut between different scenes.

The version that’s on Oculus Share is an updated version of Sightline that he created for the Oculus Game Jam back in 2013, which destroys the concept of object permanence and creates an experience that you can only have in VR. You see a table, look away, and then look back and the table is gone. It’s a surreal experience to be within a world that is so volatile, which is what really excites Tomáš. He loves to explore realms that are only possible to do within VR.

Tomáš also talks about some of his work with incorporating light fields into 3D scenes within Unity, which allows him to include a lot more complicated geometries than he would otherwise be able to. Here’s a demo scene with lightfield diamonds where “the diamond image rendered using Blender’s Cycles pathtracer into a lightfield and then new views are synthesized and blended with the scene goeometry within the Unity scene.”

Tomáš also recently won a Broze prize within the Oculus Mobile Game Jam of 2015 with the educational experience called Neos VR, which Tomáš starts to allude to when talking about some of his future plans around building out an interactive educational metaverse.

Tomáš also created some innovated 3DUI with the Leap Motion in VR Comenius, which is an educational framework prototype for teaching about anatomy.

Tomáš is definitely a VR developer to keep an eye on, and you can track his latest experiments on his @frooxius Twitter account and his latest experiments on his YouTube channel.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

mel-slaterMel Slater is an ICREA Research Professor at the University of Barcelona, Spain, where he leads the Event Lab. Mel and his fellow Event Lab researchers have been doing some pioneering work in defining the key components of presence as well as the exploring the sense of self within virtual environments.

According to Mel, the illusion of ‘presence’ within VR can be broken up into two major components. First, the Place Illusion is when your perceptual systems are fooled into believing that you’re in another place through more technologically driven factors of low-latency head tracking, a fast framerate, and having accurate 1:1 tracking of your body. The other component is the Plausibility Illusion, which is when you perceive the virtual world as being coherent in it’s construction and it accurately meets your expectations for how it behaves and reacts to your actions. Here’s more details from the abstract of Mel’s paper titled “Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments”

In this paper we address the question as to why participants tend to respond realistically to situations and events portrayed within an Immersive Virtual Reality (IVR) system. The idea is put forward, based on experience of a large number of experimental studies, that there are two orthogonal components that contribute to this realistic response. The first is ‘being there’, often called ‘presence’, the qualia of having a sensation of being in a real place. We call this Place Illusion (PI). Second, Plausibility Illusion (Psi) refers to the illusion that the scenario being depicted is actually occurring. In the case of both PI and Psi the participant knows for sure that that they are not ‘there’ and that the events are not occurring. PI is constrained by the sensorimotor contingencies afforded by the virtual reality system. Psi is determined by the extent to which the system can produce events that directly relate to the participant, and the overall credibility of the scenario being depicted in comparison with expectations. We argue that when both PI and Psi occur, participants will respond realistically to the virtual reality.

Mel describes the Place Illusion as being governed by the perceptual system, and that the Plausibility Illusion is more of a cognitive function. When it comes to breaks in presence, the Place Illusion is more resilient to temporary glitches or technological disruptions. As long as the system can recover technologically to issues such as latency, framerate, or graphical fidelity, then subjects can regain their sense of presence of being within in another world.

The Plausibility Illusion however is a lot more sensitive because it’s a cognitive function. Once there is something that is not coherent within the rules of the environment that violates the expectations of the subject, then it’s a lot harder for the experience to recover from this type of break in presence.

The Event Lab has also done a lot of work in investigating the Virtual Body Ownership Illusion and what’s required to create it, as well as it’s impact on your attitudes, beliefs, and behaviors. Here’s an overview view that explores some of these “Positive Illusions of Self in Immersive Virtual Reality:”

Another really interesting experiment from the Event Lab was looking at how to create the illusion of Time Travel. Subjects were put into a virtual museum with full body tracking so as to create a strong illusion of body ownership. They were then presented with a moral dilemma scenario that was based upon decisions that they had already made. The subjects were then brought back a week later to watch their previous actions, and then have an opportunity to intervene on the actions of their previous self.

Subjects from the experiment reported that it felt like they were time traveling because they were able to objectively witness their previously embodied avatar’s actions. More information about this study can be found here:
A method for generating an illusion of backwards time travel using immersive virtual reality—an exploratory study

This time travel illusion has a lot of interesting implications for the future of social interactions and experiences within a VR environment. Imagine hanging out with friends or family, and then being able to revisit those experiences after it had long faded from your memory. Perhaps you’d gain new insight about yourself, or be able to go beyond nostalgia and be able to relive experiences with people who are no longer alive.

Finally, Mel talks about how VR has the potential to be a very powerful tool to be able to change your sense of self. VR has the capability to put you into another body and give you another point of view and perspective, which can then give you a new perspective on your own life. No other medium has the capability to do that, and the ultimate potential of being able to do this is largely unexplored and unknown at this point.

If you’re interested in these types of questions about how VR can the sense of ourselves, then be sure to follow the publications listed on The Event Lab website, keep an eye on The Event Lab YouTube channel for some of their latest results and research, and take a look at the new Frontiers of Virtual Environments publication that Mel is editing.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

jason-jeraldJason Jerald has been working in virtual reality for 20 years now, and he’s celebrating his 20th VR anniversary with the launch of an epic textbook called: The VR Book: Perception and Interaction Design for Virtual Reality. It’s nearly 500 pages and contains over 300 academic references covering the psychological and technical considerations for creating a comfortable VR experience.

The VR Book focuses on human elements of VR, such as how users perceive and intuitively interact with various forms of reality, causes of VR sickness, creating useful and pleasing content, and how to design and iterate upon effective VR applications. It is not just for VR designers; it is for the entire team, as they all should understand the basics of perception, VR sickness, interaction, content creation, and iterative design.

In this interview, Jason gives an overview of some of the chapters and sections that are covered in the book including perception, how to minimize adverse health effects, creating VR content, a list of interaction patterns, iterative design strategies, and where VR is headed.

We talk a bit more about the importance of fidelity in VR experiences, the virtual body ownership illusion, and how he thinks about the different fundamental components of presence.

VRBook

The VR Book will be launching on August 9th at SIGGRAPH next week, and will eventually be available online for purchase here. It’s not available for online purchase yet, but you can sign up to Jason’s e-mail list to get informed as to when the book becomes available. It’s sure to be a vital collection of all the latest research for what is known and not known about creating comfortable and compelling VR experiences.

If you’re going to be at SIGGRAPH next week, then check out Jason’s VR Book presentation on Monday, August 10 at 4:00pm in the Immersion Dome at the VR Village (South Hall G). He’ll also be signing books on Monday, August 10th at 5:15pm at the SIGGRAPH bookstore where hard copies will be available for purchase.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Tom-ImpallomeniSocial VR experiences and telepresence applications are going to be one of the most compelling use cases for virtual reality, and watching sports events within a VR space with your friends has a lot of potential. Virtually Live is taking a really innovative approach to showing live sports events within VR in that they’re not using 360-degree cameras. Instead they’re using optically-tracked statistical data from Stats.com and then transposing it into a virtual arena. They’re able to re-create the positions and movements of all of the players and the ball in a 3D space, and therefore recreate the experience of going to a sporting event with your friends.

Tom Impallomeni is the CEO of Virtually Live and he was at SVVRCon showing off a demo of their technology in action. They’re starting with soccer, but planning on expanding to other sports presumably including baseball, basketball, and tennis as is shown in their trailer for their company.

Virtually Live – Intl. from Metaverse Holdings on Vimeo.

Part of the advantage of using optically-tracked data is so that they can create a virtual camera and put it anywhere to have the best perspective, even start to give you the perspective of what a specific player might have been able to see.

Tom says that we’re still a long ways away to being able to completely recreate what it feels like to be at the stadium with tens of thousands of people, but that they’re getting closer and closer. They’re able to create a shared social space with your friends that goes beyond watching it a game on a 2D screen.

It’s also still a bit of an open question of how realistic their VR recreation will feel, especially when there’s a lot of emotion, body language, and body movements that may be too nuanced for cameras to be able to track at such a long distance. But Tom says that the technology is continuing to improve, and that they’re even looking at tracking facial expressions and limb tracking as a part of their R&D initiatives.

They’re also able to visualize the raw data from stats.com in a way that is improving it. There’s a lot of noisy data from the raw feed, and they’re able to apply some algorithms to help to improve upon the data that’s being collected. Using VR to visualize the data is helping to provide feedback and improve upon these existing optical tracking techniques.

Some of the use cases that Tom sees are going to be compelling is to be able to first be able to share a virtual environment with your friends. He also imagines that it might be really interesting for people to watch a game from their favorite seat in the stadium if they’re using to seeing the games from that perspective. They’ll also be able to create views that were impossible, and so I’d imagine that a tabletop view may be a perspective that’s really new and likely to be super compelling.

Tom says that they also plan on fusing live footage into the experience as well and so there will likely be a way to watch the action in a more traditional way and then have the VR environment be able to recreate the feeling of being with at the stadium, in a room with your friends, and have the sports commentary all piped into whatever combination best suits your preferences.

At the moment there aren’t clear broadcasting rights to data-driven reconstructions within VR, but I imagine that there will be new licensing frameworks if this type of approach starts to catch on.

Virtually Live has some really interesting and innovative technology, and so I’ll be curious to see where these types of reconstructed VR experiences will go in the future. I think there’s a lot of interesting use cases beyond just sports, and it could be applied to many other live performances and concerts, especially when there and better and better ways of capturing action with arrays of 3D depth sensor cameras and other emerging technologies.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

howard-roseVirtual Reality can make you feel like you’re in another world, and it turns out that this can have quite a therapeutic effect for people experiencing chronic pain. VR pain management has been researched in clinical studies for over a decade, and it’s been shown that VR can help significantly reduce pain, relieve stress, and build resilience.

Howard Rose has been working in VR for over 20 years, and his company DeepStream VR is focusing on building VR applications for the healthcare market. They’re focusing on pain management applications for burn victims, wound care, deep tissue injections, and a number of other different conditions that require hospitalization.

They’re able to modulate the VR experience using biofeedback sensors like heart rate monitors and galvanic skin response in order to maximize the amount of pain relief that you need. They create a pain profile, and then use the feedback as inputs to help maximize the focus and level of immersion. They’ve worked on classic VR experiences for pain management including SnowWorld VR, and they’re also creating a new pain management experience called Cool with more interaction as well as a labor and delivery VR application. Some crucial elements of creating a compelling and effective pain management VR experience include having a wide field-of-view above 60 degrees, visual flow, and engaging interaction. They also found that feeling vection and motion down a river created a more compelling experience.

VR pain management compares favorably to other techniques, and does especially well when paired together with opioids. Because pain medicine can be so effective, then there can develop an over-reliance and subsequent addiction issues. However, if VR is provided an option, then it can engage and develop a resilience in patients in a way that distracts them from the pain, but also prevents them from relying too heavily on their morphine drip.

Howard also talks about some of his lessons learned for making grounded VR experiences, as well as some of his experiences with the River Accelerator program.

Finally, DeepStream VR is ultimately trying to go beyond the normal health consumption model to a model that empowers people to become a health producer by using our innate abilities to heal ourselves. He sees that VR can help people become more independent, and to more easily integrate wellness into our lives in a way that’s more accessible and affordable. DeepStream VR is doing this by creating the VR experiences and pioneering the approaches that enable this type of paradigm shift.

Here’s a TEDMED talk by Howard Rose from September 2014.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

eric-greenbaumEric Greenbaum was at a New York Virtual Reality Meetup when someone from the financial industry came up to him with some ideas for how to use VR to analyze stock market data. They decided to collaborate on QuantVR and develop a prototype demo of a stock market platform and data visualization tool that was shown at the Silicon Valley Virtual Reality Expo this spring.

Greenbaum’s business partner has a lot of experience within the financial industry, and they’ve using that insight to create different 3D visualizations of financial data in order see patterns within the data. They created a Ticker Tube view that fills up your entire field of view in VR with data from 300-400 stocks, but they believe that VR has the potential to go beyond just adding a lot more screen real estate for 2D data. They believe that 3D immersion has the potential to unlock new insights and democratize financial analysis that make the market even more competitive by smaller players.

Greenbaum’s business partner says that automated trading algorithms account for as much of 70% of the volume of trading, but that a lot of these automation tools have been developed in the absence of visualization tools. QuantVR is able to slow down and visualize the millions of transactions and actions that happen over a course of a second, which allows them to gain insight into how dumb some of these trading robot algorithms can be sometimes. Again, discovering these types of inefficiencies can translate into real dollars by finding a way to be more efficient.

Technology has disrupted the financial industry, and there’s been a lot of complexity introduced that has created a lot of fear due to that lack of understanding. So not only can there be applications for financial companies, but also for the government in order to improve their understanding and oversight of what’s happening.

One of the 3D visualizations that QuantVR created is the Stockscape view, which shows the market capitalization of the highest traded stocks as a city-scale landscape. You can hover around to the tops of different buildings representing the market cap size, and look around to compare the relative sizes from within an immersive environment. Scale is something that QuantVR has been experimenting with, and it’s still a bit of an open question as to whether having a city-scale or miniaturized-scale is more efficient.

Stocks move to common risk factors and so QuantVR has created different logical groupings of stocks to be able to watch trends of how those stocks are changing over the course of the day. There haven’t been a lot of 3D visualization tools, and so most of the financial analytic approaches have defaulted to a 2D representations.

QuantVR believes in the power of exploring what 3D has to add to analyzing data, and if they’re able to gain new insight into the market, then that could easily translate to having a competitive advantage in generating revenue for clients.

They’re actively beta testing their product with a high-profile client, but it seems as though the financial companies are going to be pretty tight-lipped about how these new immersive technologies might provide them with any specific competitive advantages.

Will 3D data visualization tools be the future of financial analysis? Or are there too many inefficiencies with the occlusion issues that are introduced into data visualization? That’s yet to be seen and QuantVR is on the forefront of exploring these issues. Yet I’d don’t honestly expect to get a lot of answers to these questions from Wall Street considering how much money could be at stake given what those answers might be.

The VR data visualization academics that I talked with at IEEE VR believe in the promise of 3D data visualization and analysis, but it’s usually data that is tied to actual 3D geometries — like correlating earthquake data onto a map of the earth. So the benefits of abstract data visualization with an immersive 3D environment are not immediately clear. Sometimes 2D charts and graphs are actually far superior and more efficient than trying to view the same data within a 3D context.

However, one thing that the academics agreed upon is that data that has a time dimension has a more clear use case for representing that change over time on the Z-axis. So stock market data certainly has a lot of potential for how that time data could be represented within VR.

Finally, QuantVR does sees that VR has the potential to democratize data analysis within the financial markets beyond the few, very elite quantitative masterminds. Their goal is to break down complex problems into a symbolic representation in an immersive environment that becomes intuitive for a user to understand a fuller spectrum of the dynamics of the marketplace. They hope that it’s going to make the market more competitive and ultimately better for all investors, and they’re looking forward to help making that happen with the power of VR.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Ben LangBen Lang has been covering the VR ecosystem for The Road to VR since 2011, which is before the Oculus Rift kickstarter even launched in 2012. I had a chance to interview Ben at last year’s Silicon Valley Virtual Reality Conference 2014, which was really the first large consumer VR conference. So it was nice to be able to catch up at SVVRCon again to reflect upon how much the VR industry has grown over the last year, but also where he sees VR is headed.

Over the course of 2 years, the Silicon Valley Virtual Reality meetup has gone from a monthly meetup of 60-120 people to the San Jose Convention Center with over 1400 attendees from around the world. It’s only been about a year since Oculus released the DK2, and since that time Ben hasn’t spotted a DK1 at any VR-focused event.

Ben says that the enthusiasm about VR from the community is palpable, and that it’s important to remember that this popular resurgence of VR was catalyzed by a grassroots movement of people who wanted to make it happen. It was a corporate marketing push, but it started with people on the ground believing in the potential of this medium, and evangelizing it to their friends.

It’ll be very soon when consumers will be able to purchase a VR system that previously would have cost then tens of thousands of dollars. But yet, there will still need to be a grassroots sharing of VR on a one-to-one basis. Ben says that having a VR advocate to share their enthusiasm of VR is part of the transformative impact of having your first VR experience. People aren’t necessarily seeking out their first VR experience, but instead it’s coming from recommendations from other people that they know.

Marketing VR is a challenging problem because overhyping the potential of VR can actually have a negative backlash of setting the expectations too high. One example of this is the Samsung ad advertising the Marvel Avengers VR experience by Framestore that shows all sorts of hand-tracking, implicit haptic feedback, and interactive room-scale immersion that the Gear VR — or any consumer-level VR equipment — is capable of doing today.

Ben advocates using truth in advertising as a long-term strategy to grow the market, rather than using overhyped promises that are going to cause people to become disillusioned and disappointed about what VR is actually capable of. VR is a hard to convey through a 2D medium, and so the best way to demonstrate the power of VR is to have an actual experience of this immersive medium.

As for the future, Ben sees that there’s a lot of potential for VR experiences that make you feel different emotions. He cites some of the experiences that VRSE.works have created that get people tuned into their emotions, and that there’s a huge opportunity for more of these types of experiences.

He also sees a huge potential to extend fictional worlds that we have emotional connections to from various stories. It could just be either a meditative, emotional, scary, or creepy experience that you have to within an environment that is inspired by your favorite book or movie could be really powerful. Storytelling in VR is going to go to the next level when we are able to have interactive conversations with characters driven by AI.

Finally, Ben hopes that the VR community can maintain the tight-nit and collaborative nature even when there may be a lot of competition for the same dollars.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Agata-SocciniAgata Soccini is a Research Fellow Università degli Studi di Torino at Thales Alenia Space. She talks about how virtual reality is used to train astronauts, visualize galactic data, as well as analyze thermal data about space ships.

Specifically, she was at the IEEE VR conference giving a talk about doing aerothermodynamic analysis of the Intermediate eXperimental Vehicle (IXV) within VR. The IXV is an European Space Agency (ESA) experimental suborbital re-entry vehicle (spaceplane prototype) to validate European reusable launchers, and she says that when you match up the actual data to the the actual location on the space ship that even a layman could start to interpret the results. Overall, they were trying to see how well the data from their simulations matched up with the real-world data.

Agata also talks about some of the other procedures that astronauts need to be able to perform, and how they use Haption haptic devices to simulate interacting with zero-gravity objects. There aren’t a lot of astronauts in their program, but there’s been enough feedback to know that these simulations have been very effective to training the astronauts to go through these procedures.

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.

David-WhelanOne of the most common reactions that I hear after giving someone their first VR experience is “Oh my God! This is going to change everything. Imagine how this will change education!” David Whelan & Mike Armstrong are doing just that with Lecture VR, which is a social multi-player edutainment platform that they’re building starting from the Riftmax Theater social VR technology stack. Their goal is to recreate the educational system that was portrayed in the sci-fi novel Ready Player One.

David is probably most well-known for his Virtual Reality Reviewer website, but after watching and reviewing hundreds of VR apps he’s starting to create his own experiences with Immersive Education company.

One of first experiences that they are creating is The Apollo 11 VR experience that was successfully funded through Kickstarter.

Mike-ArmstrongDavid and Mike talk about their vision for creating both live lectures in VR as well as pre-recorded lectures that may have a live interactive Q&A session afterward. David says that they completion rate for the online open courseware classes are actually pretty low, and they’re hoping that recording the more of the body language of the lecturer will make it more immersive and engaging.

They’re also hoping to create a much more social learning experience. Whereas a lot of open courseware classes have online chat forums, they’d like to have ongoing classes where the students have a chance to interact with each other in real-time.

To me the educational opportunities with VR is one of the most exciting potentials for what VR has to offer, and the team at Immersive Education are certainly on the leading edge of creating a platform for social learning experiences. It’ll be exciting to see how their efforts will evolve, especially when I see reactions to their Apollo 11 experience like this one from YouTuber “Hoopermation”

Become a Patron! Support The Voices of VR Podcast Patreon

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.