This is the 100th episode of the Voices of VR podcast marking 33 hours of interviews with the leaders of the consumer VR revolution over the past 10 months.
I wanted to celebrate the 100th episode with an interview with Sébastien Kuntz because he helped inspire me to start this podcast. Sébastien has been working in virtual reality for 13 years doing everything ranging form training simulators, 3D engine development, and producing a middleware VR solution with MiddleVR.
I first discovered Sébastien’s work during the IEEE VR conference last year because he was tweeting about different presentations talking about the academic community’s response to the Facebook acquisition. Here’s a couple of examples of his tweets that captivated my attention:
— Sebastien 'Cb' Kuntz (@SebKuntz) March 31, 2014
— Sebastien 'Cb' Kuntz (@SebKuntz) March 31, 2014
I wanted to hear more from Sébastien and attendees at IEEE VR, but there weren’t any consumer VR publications covering what was happening in academia or with VR researchers. In fact, there was hardly any coverage from any publication of last year’s IEEE VR conference beyond tweets from attendees, with the most prolific being the ones from Sébastien.
Because of this lack of coverage, I decided to start my own podcast. I reached out to interview a couple of other attendees of the IEEE VR conference including Eric Hodgson and Jason Jerald. I also really wanted to hear more from Oliver “Doc_Ok” Kreylos who was a respected commenter on the /r/oculus subreddit, and also happened to be working in VR within an academic context.
The Voices of VR podcast was born and seeded with these more academic insights and perspectives into VR, and so I’m really looking forward to being able to travel to France at the end of March to cover the 2015 IEEE VR conference.
Sébastien has a great blog on VR with summaries of a lot of interesting VR research. He pointed me to Mel Slater’s research into virtual bodies, and this video summary of Slater’s research into the Positive Illusions of Self is one of the most fascinating VR videos that I’ve seen:
I also came across some quotes about presence in VR from Mel Slater on Sébastien’s blog that inspired me to write an extended essay about it in VR that I’ve referenced in a few tweets:
— KentBye (@kentbye) February 1, 2015
A number of people have asked me where that excerpt was from, and it’s from an unpublished essay that I’ll share here so that people can more easily link and reference to it.
THE FUTURE OF IMMERSIVE VIRTUAL REALITY
Just as the Internet and mobile computing have changed every aspect of society over the course of the last 20 years, Virtual Reality is poised to have a similarly pervasive effect on our lives.
The types of visceral experiences that VR can provide are truly unique, and on the whole constitute a new communications medium that will amount to the Gutenberg Press of the 21st Century. Felt experiences will be able to be captured and shared just as books were able to capture and share information and knowledge. Just as new insights into perspective catalyzed breakthroughs in Renaissance art, then adding a new immersive dimension to computing will likely spur a similar revolutionary change in what types of experiences that can be shared through virtual reality.
In order to investigate what types of new doors and interactions that VR will be able to provide to society, then it’d be helpful to first dive into the key components of what constitutes “virtual reality.”
WHAT IS IMMERSIVE VIRTUAL REALITY?
There’s a pretty big difference between 3D virtual worlds that are experienced through a 2D screen, and a completely immersive, virtual reality experience. There have been 3D environments in computer games since the early 90s, and interactive virtual worlds like Second Life since 2003. While immersive virtual reality includes some components of virtual environments, it also provides other elements that are completely new and different that transcendent what is possible through a 2D medium.
Sébastien Kuntz defines Immersive Virtual Reality as “the science and technology required for a user to feel present, via perceptive, cognitive and functional immersion and interaction in a computer-generated environment.”
There is a sense that you’re transported into another world where your unconscious parts of yourself are fooled into believing that the computer-generated reality is an actually real. Within the VR community, this is widely referred to as the experience of “presence.”
VR researcher Mel Slater defines two key components that are necessary for people to have a realistic response to a virtual reality environment. He says, “The first is ‘being there’, often called ‘presence’, the qualia of having a sensation of being in a real place. We call this Place Illusion (PI). Second, Plausibility Illusion (Psi) refers to the illusion that the scenario being depicted is actually occurring… when both PI and Psi occur, participants will respond realistically to the virtual reality.”
Some of the elements that trick your perception into believing that you’re in another place are real-time interactions like head tracking where your physical movements are mirrored within a virtual 3D environment to the point that they match your expectations. Even though your rational mind may realize that you’re not in another world, your unconscious perceptions and primitive limbic mind will react as if the environment was real.
The second component of plausibility is when your cognition is fooled. As Kuntz says, “Everything that happens is coherent. You actually believe you’re there, your actions have a credible impact on the virtual environment and your sensations are affected by it.”
Attaining a sense of “presence” is the ultimate goal of a VR experience, but there is no fixed set of ingredients that reliably produce it. What is known is that there are a combination of components that together constitute “virtual reality,” but taken individually are merely a part of a virtual world.
If I were to boil down presence to an equation, then I’d say it’d be this:
Presence = Place Illusion + Plausibility Illusion
Sébastien and I talk about how the Crescent Bay demo really only had half of this equation with the Place Illusion. Without input controls, then there’s no ability to feel like your actions have an impact on the world and makes it more difficult to achieve the illusion of plausibility. So for the both of us it wasn’t able to achieve that full sense of immersion and presence.
This interview happened at Oculus Connect last September, and so neither one of us had seen the latest Valve Vive demo yet. Based upon the reactions to Vive, it’s clear that having an accurate tracking and input control system takes VR presence to the next level. We discuss our understanding his presence as well insights from his extended experience in VR, as well as his reaction to the Crescent Bay demos.
- Middleware abstraction of the input trackers to use a wide range of input controllers. Also be able to determine things about the user
- Input controllers and input devices and how should developers should approach implementing input controls, and natural hand interactions
- What is presence has two levels of being immersed cognitively, and then there’s a lower-level of immersion at the subconscious perceptual level.
- Virtual body being transferred to a virtual world for virtual therapy for phobias. You can also have marketing studies within VR because people act naturally. Engineers can test the ergonomics of physical designs & architectural spaces
- Connecting the space to your interactions. You can simulate products and spaces at scale.
- Don’t want to use VR to escape reality, but use it to improve reality. Use VR to test an assembly line with a virtual body. You can training simulations where you can actually practice the gestures that you need to do. You can have therapy towards phobias. You can use it to build empathy for others.
- Crescent Bay VR HMD was the best that he saw up to the point of the Oculus Connect. The input device was missing from the experience.
- Lightsaber demo with Sixense built even more presence. VR presence is about both Immersion into the Virtual world as well as Interaction with the virtual world where your actions make a difference.
- Reflection of the VR community and how much that it’s changed over the past couple of years. It’s easier to see your own body and collaborate with other people in a CAVE environment. Having your avatar in VR and play around with your identity, which will make it easier to collaborate.
- Telepresence within VR and social experiences in VR with haptics. Experiment within a virtual bar where two avatars are arguing, and they look at you and ask you to weigh in. You can tune into emotions with their body language in these virtual environments.
- Most compelling VR experience where you are put into the body of wounded soldier, and your legs have disappeared and you’re waiting to die. Then an avatar comes to help you, and people smile are really grateful. Presence wasn’t broken because interactions were limited. If a virtual hand goes through a wall, then that also breaks cognitive presence, which is harder to maintain because it’s like a house a cards. It takes a long time to rebuild this sense of presence at a subconscious level once it’s broken. Have to make sure that your brain accepts the rules of these virtual worlds, which is more difficult.
- No bodies were in the Crescent Bay demos, and you were standing. Want to be able to track your entire body. Sitted experience is a limit due to the hardware and what you can do with creating a sense of presence. Trying to recreate a whole reality. Can’t simulate everything yet because we don’t understand everything yet. Realistic rendering makes the mind demand that everything else is realistic including the sound and haptics, otherwise everything will fall apart.
- Showdown demo where you’re moving through the space through slow motion. Felt like a ghost, and so anything could happen. That helped him accept it. If he could interact and see interactions within VR, then it’d be even more believable.
- Really liked to interact with the alien in the Crescent Bay demo. Also really like the T-Rex coming at him because it was beautiful and scary.
- New Consumer VR vs. Old VR communities. Misconception with VR is that everything VR died 20 years ago, but it’s still being used in professional situations as well a lot of VR research. Need to do new research with the new hardware, and great to see Facebook and Oculus are investing in new research. Working with the human mind and perception and not just hardware. We don’t understand how we perceive the environment.
- Controlling perceptions to take us into another world, is that just escapist of going into fantasy worlds. VR can be used to improve reality. Stanford research where you see you virtual avatar and how nicer avatars can improve your sense of self-image and confidence. You can transform your self-image. It can also be used to escape reality as well. It’s up to society to decide. TV is probably worse than VR because it’s more passive.
- Crescent Bay demos were pretty passive and not very interactive. Adding a tracked joystick is easier compared to achieving a low-level sense of presence. Then we can start a new frontier of how to interact with a 3D interface
- Mel Slater’s research into the impact of virtual self, and how your brain accepts a virtual body. Your brain accepts it in seconds shen you hands move. Experiments of putting your virtual body into different ages and races, and it reduces racism because it builds empathy.
- 3DUI book by Doug Bowman. Go to IEEE VR conference. There are a lot papers online. They’re friendly and they want to help the VR community.
- Last IEEE VR conference happened right after the Facbook / Oculus acquisition. How could Palmer create a VR HMD, and the academic community couldn’t do it? Too reductionistic in the academic community and they have to be cautious and make incremental progress.
- The Metaverse will change the world because it’ll provide a new way to communicate with your friends. People will recreate reality in VR because that’s what happens with a new communications medium. Unlock our brains to embrace the new possibilities, and looking forward to being a part of the VR development community.
I’m looking forward to many more episodes of the Voices of VR, and I hope that you’ve been enjoying them.
— Sebastien 'Cb' Kuntz (@SebKuntz) September 20, 2014
Theme music: “Fatality” by Tigoolio