I had the chance to give the keynote at the Silicon Valley Virtual Reality Conference where I provide some historical context for virtual reality tracing the evolution of communications mediums in order to contextualize my elemental theory of presence. It traces the history of the science and philosophy, and shows how VR is providing a direct experience of how much of our reality is subjectively constructed with our entire bodies. Hopefully this talk will help contextualize where we’ve been and where we’re going with VR, as well as a holistic framework to doing experiential design.
LISTEN TO THE VOICES OF VR PODCAST
Here’s the video of the talk:
And here’s the slides from my presentation:
Here’s the link to this presentation on Facebook if you’d like to share it there.
Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye and welcome to The Voices of VR Podcast. So on today's episode, I'm going to air the keynote that I was able to give to the Silicon Valley Virtual Reality Conference. It was an opportunity for me to speak to the entire VR community and I wanted to provide some historical context of the evolution of the different communications mediums and how they're leading into VR and how we're kind of shifting from the information age to the experiential age and some of the deeper philosophical implications of holistic thinking that is required in order to do experiential design. And I think it also provides some context as to why I've been looking back to the ancients in order to use this elemental theory of earth, air, wind, and fire to be able to use these primary metaphors in order to describe the qualitative experience of VR, but also as a framework to do experiential design. So that's what we'll be covering on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. Today's episode is brought to you by VRLA. VRLA is the world's largest immersive technology expo with over 100 VR and AR experiences. They'll have tons of panels and workshops where you can learn from industry leaders about the future of entertainment and storytelling. I personally love seeing the latest motion platforms and experiences that I can't see anywhere else. Passes start at $30, but I actually recommend getting the Pro Pass so you can see a lot more demos. VRLA is taking place on April 14th to 15th, so go to virtualrealityla.com and get 15% off by using the promo code VRLA underscore Voices of VR. So this keynote happened on May 30th, 2017 at the San Jose Convention Center at the Silicon Valley Virtual Reality Conference. So with that, let's go ahead and dive right in. All right, so my name's Kent Bye, and on May 19th, 2014, I got in front of the Silicon Valley Virtual Reality Conference, and I announced to the world that I was going to start the Voices of VR podcast. And over the next day and a half, I did 45 interviews with a lot of the early pioneers of that first consumer gathering. Since then, I've done over 700 interviews with many different leaders in the virtual reality space, published about 520 of those so far, and I've just been trying to organize all the different things that people are actually doing in VR, what's actually happening, and just seeing how this medium of virtual reality is evolving. And one question that I ask everybody is, what is the ultimate potential of virtual reality? And as I hear all these different answers, I start to categorize them in my own mind, in the realm of, the domains of human experience for virtual reality. And these kind of describe different contexts under which you may use virtual reality, everything from entertainment to enterprise in your career, medicine, home, family, education. And so for me, what I see happening is that there's all these different silos in each of these different contexts. And that each of these different people that are focused on each of these areas have their own language for describing the same thing. So we have these silos. And that in some ways, virtual reality is providing almost like this transfer learning language so that people can start to collaborate within the context of virtual reality. We're all working with hacking the senses and be able to use human experience in order to collaborate with each other. So today I just wanted to orient us in space and time to give you a little bit more historical context of virtual reality for how I see it, for where it's been and where it's going, but also talk a little bit about the elemental theory of presence that I've been working on. So let's go way back 20,000 years to the first cave paintings, where you can imagine just sitting in a cave with some light and you're completely immersed and you see these images. Just imagine what kind of direct experience you would have that would be different than anything else you've ever experienced before. And then we go from there to hieroglyphics. So we start to see this evolution of visual communication that has a little bit more semantic structuring to it. But still, it's kind of atomized in a certain way. And it wasn't until we got the alphabet that we started to really go into this level of abstraction. Not only with being able to have letters able to form words and thoughts that go beyond our direct experience, but to use the level of language. And you might be able to learn about the world based upon something that you may have read. So we're moving from a world where you used to learn about the world by your direct experience to reading about things with language. And to me, I see this in the context of the left and right brain, the left brain being very linear, the right brain being nonlinear. It's almost like a CPU and GPU. But the left brain is very objective, it's quantitative, and the right brain is very subjective and qualitative. And so if we skip forward to 1454, that was when the Gutenberg Bible was first published, and we have like this kicking off of this revolution of printing. So books kind of represent this democratization of information. You're able to capture information and knowledge and widely distribute it in this kind of mass production way for the first time. And so what started to happen is you have people like Ficino, who started to go back to the ancient Greeks to translate a lot of Plato and Hermetic texts, and he started this Plato Academy and was teaching a lot of these Renaissance painters about some of these old ancient thoughts. And this really inspired them to start to try to encompass the pure forms of beauty and truth into art. And then that same thread of elegance inspired Copernicus to go from what was essentially this geocentric, Ptolemaic system that was very confusing to something that was much more elegant with the heliocentric model. So you have this switch from geocentric mindset to heliocentric mindset, which really spurred the Enlightenment. And from that Enlightenment, we also have this birth of the modern man. So, we're starting to get into the left brain and science and objectification of the world, and the thing that happens is that we have this emphasis on the left brain. For the last 500 years, our culture has been driven by the left brain objective in the quantitative world, and there's been a diminishing of the image, the diminishing of the qualitative subjective realm. And so this is sort of encapsulated by Descartes. You have this mind-body dualism, this split, where the mind and body are actually separate. You have your inner life, but you also have your outer life, your objective. And Rick Tarnas, in The Passion of the Western Mind, as well as The Cosmos and Psyche, he really says that this is a turning point in our history, that this is really like the beginning of what you could say is a 500-year journey of man and his hero's journey, that there's a separation, that separation being the split between the mind and body. and that we're kind of in the middle of this ordeal and initiation, and we're trying to resolve a lot of issues in our world and our culture. So he sort of casts it as these dual myths. Both of these are happening at the same time. On the one end, you have unending technological progress, but on the other axis, you have this spiritual and ecological crisis. And these are both happening at the same time. So just to flesh that out, you could go from the printing press to the computer today, and you just see this exponential growth of unending innovation and change with technology. But yet if we look at what's happening to our bodies, we're kind of like in this realm where we're becoming more dissociated and we're kind of like disconnected from our bodies. We don't have a good sense of what's happening in our internal subjective life. And we have this fundamental question is, is it actually making us happy? And not only that, we have this ecological crisis. So everything that we're doing in technology is kind of out of harmony with nature. A lot of the things that we're doing in this world right now are not in harmony with the Earth, and it's just creating this ecological crisis. So on one hand, we have this technological progress, but we have this spiritual and ecological crisis at the same time. So this Descartes moment, you have this split between objectivity, subjectivity, the mind, body, and science, and spirit. So let's skip ahead to the 19th century. We have the return of the image with the camera. 1890s, there's this amazing confluence of all sorts of different things, from film to camera to the transmission of power to the development of electromagnetic waves, which then leads, a couple decades later, to broadcast radio as well as broadcast television. We have, essentially, with electricity, the birth of this electronic age. And that drives then to computers. So then have computers do all these calculations that we used to do by hand, and now they can take care of it. And in 1962, you have Ivan Sutherland, who decides that he wants to create a graphical user interface with a sketchpad. And he just starts to draw on a computer. And he's, in 1965, wrote this paper called The Ultimate Display, where he wants to treat the computer as a window into a mathematical wonderland, where he can just see the pure elegance of mathematical form. So that inspires him to build the very first virtual reality headset with a sort of Damocles in 1968. Then in 1969, we have some of the first transmissions of the internet. So we have the beginnings of interconnecting these computers to not only distribute information, but also to distribute images and videos instantaneously all over the world. So we're moving from this information age to the experiential age. And with that, we have computers. So I would say that a computer is kind of like the Gutenberg Press of this era. It's democratizing experience in a way, just as the Gutenberg Press democratized access to information and knowledge. So we have virtual reality technologies that are able to basically hack our senses and to put us into these crazy worlds where we have these experiences that our mind doesn't necessarily make a differentiation between what's real and what's happening in these synthetic realities. which then gives us a direct experience of like, what is reality? And so what I see is happening here is that it's actually moving us back towards this right subjective qualitative brain. We're actually realizing how much of our reality is constructed within our minds. And that there's actually a huge subjective component to it. It's not something that you're objectively seeing. So we have this shift back to the right side. So we have the principle of embodied cognition that essentially says that we don't just think with our mind, we actually think with our entire bodies. We don't really actually know how we're constructing the reality, but we have a sense that it has something to do with the holistic system of our body, our emotions, everything. And not only that, but our environment. And that's something, a unique affordance of virtual reality. So this is a graphic from Robin Hunneke, where she showed at VRDC. At the very highest level, you have this abstractions. That's the mind, and that's sort of the left brain. But at the very bottom, you have direct experience. That's sort of like virtual reality is able to mimic all these things to your senses. And there's so much of our perception that is happening below our conscious awareness. And so with virtual reality, we're able to give these rich, direct experiences, which allows us to learn and retain information in such a deep and more intense way. So, what I see personally is that with virtual reality we're starting to have this convergence of objectivity, subjectivity, the mind and body and the science and spirit, whereas we have this Cartesian split that happened in the Renaissance and the Enlightenment, but now this new Renaissance that is happening is this return to holistic thinking. So I would say that experiential design requires holistic thinking. If you're going to be creating virtual reality experiences, you have to think about all different dimensions of what makes a good experience. So just like Ficino went back to Plato to get inspiration, I'm today also going back to Plato to look at some of these Neoplatonic ways of thinking, which is why I would say that looking at the elemental theory of presence is sort of the thing that I've been really focusing on a lot lately. So the air element being social and mental presence, of being able to communicate with other people. Active presence, that's any time you're expressing your agency or will within an experience. Virtual reality is actually putting your body into media for the first time, so you actually have the sense that you have an embodied presence within virtual reality. And then emotional presence, I think, is probably the one thing that is probably the most overlooked in our culture overall, but there's a huge component for emotions and being really engaged within an experience. So we take this framework and just apply it. The ancients actually said that all these four elements are happening all the time. You can't necessarily just isolate them. It's just maybe a center of gravity or so. So if we kind of think about that and say, what is the center of gravity of some of these existing mediums? We can look at gaming and say, it's a lot about making choices with your mind and actually exerting your will into an experience. So it's a lot about agency and mental presence. Film, you know, it's actually encompassing everything, but it's also a passive experience where you're really receiving a story, and it's really ultimately engaging your emotions. And then virtual reality is putting your body into the experience for the first time. So I just wanted to go through each of these a little bit just to see how I kind of see how these are playing out. So with active presence, it's all about agency. They're going to have gameplay mechanics, and there's going to be a lot more intuitive interactions. Whereas in a lot of games, there's a lot of abstractions that you're pushing buttons, but now you have the ability to express your agency in an intuitive way. So there's going to be all sorts of new 3D user interfaces, conversational interfaces. There's a lot of error element there as well. But the fact that you could make something happen in a VR experience by saying something is going to be a new way of you expressing your agency. So think about that for mobile VR, what that means to be able to not have your hands in the game. But if you're able to talk to a VR experience, you can actually do a lot more there. There's going to be a lot new creation tools, live performance, peripherals, I think, to exert your agency, the tools, essentially, and interactive learning. When you look at the education, there's this pedagogy of constructivism, which is meaning that you're actually engaged in the process of constructing your own meaning. So that's a lot of the fire element. So social mental presence, you have the scene coherence that you actually believe everything that's actually happening. And I think the big thing with the social part is the Uncanny Valley, so making sure that your avatars actually feel real. Otherwise, it's going to break your sense of presence. And game developers often talk about mental friction in order to have engagement. So the combination of natural language processing with artificial intelligence is going to allow you to actually speak and interact with these experiences. And that is going to deepen your sense of presence. So a lot of choices, a lot of the mind is about education, cognitive enhancement. Got some interviews coming up with people who are doing like consciousness hacking in different ways. So neuroscience, also the brain computer interfaces. This is something that's actually coming up a lot lately. So you'll hear about Elon Musk and his neural lace and doing more invasive brain control interfaces. I think this is actually going to happen. Kernel is going to start with addressing dementia and Alzheimer's, but eventually it's going to get into the realm of cognitive enhancement. And I think over the next 50 years, I'm going to see a lot more invasive technologies. But the question I would have is that given this model of thinking holistically, can you really just hack into the brain, write neural code, and implant memories? Or do you actually need the full body? Is it actually better to have a VR experience where you're actually engaging your full body in your existing perceptual systems? Or are we going to have something that has these invasive brain control interfaces? I'm personally skeptical that they're going to be able to figure that out. But it's something worth considering. So in terms of embodied presence, you have the invoking of the virtual body ownership illusion. That means that whenever you're moving all of your limbs, you actually feel like your body is in VR. Because we don't have a really great IK, we're going to get more and more of our body into the experience. And as soon as we have our feet and our hands in the experience, it's going to take the level of presence to a new level. Looking at embodied cognition to see how that can change our sense of learning. It's going to completely revolutionize how we think about education. Hand presence, our avatar, haptics are just going to keep getting better and better. So all of our sensory perception has to do with the body. And I've even done some interviews with David Eagleman talking about sensory replacement. So he's got these buzzers on his body that are taking audio. And for people who are deaf, they're able to actually mimic the signals that they would get from the ear. And they're able to turn your torso into an ear. So, this idea that you could do sensory replacement is an interesting idea, but again, I'm sort of skeptical about the limits of that. Biometric data. There's a lot of issues about privacy around data you're getting from your body. This is entering into a new realm. This is something I've been talking a lot in my podcast recently. Body language and a sense of place. These are all things that are going to be communicating in a way that are creating this sense of embodied presence. And finally, if we look at emotional presence, it's all about story and character and the colors that you use are creating this ambience. Music is a huge part of engaging your emotions in a subconscious way. You have facial tracking and empathy, and I think there's actually a strong connection between our emotions and our memory. But also looking at symbols and intuition and the soul and the collective unconscious, all these sort of more esoteric realms, again, I think that emotional presence is the thing that we understand the least, but I think it's That's a huge part of creating an experience that's going to be engaging. This is the framework that I've been using as I do interviews. I am parsing out and I go through experiences. I'm seeing these different levels of presence. That's how I conduct my interviews. It's also seeing where there's different blind spots. Overall, when you think about experiential design, this is a framework that you can use to start to develop your VR experiences. So I'm continuing to do the Voices of VR podcast, and I'm working on my book, The Ultimate Potential of Virtual Reality, which I hope is coming soon, and also the Voices of AI. So I'm going to be continuing to be unpacking a lot of these different issues and whatnot. And the people that are in this room are really going to be the ones that are going to be creating the future. And so hopefully I've been able to give you some tools to be able to help not only understand what you're creating, but also to create the most immersive experiences that you can. Thank you very much. So that was the keynote that I gave at the Silicon Valley Virtual Reality Conference, where I was trying to give some historical context to VR and to give a myth and a story as to what we're doing with VR, where we've been, and where we're going, and some of the deeper philosophical implications of VR. And if you're listening to it here for the first time and you want to look at it some more, check out the slides. I highly encourage you to check out the links in the show to actually watch the video or to be able to scroll through the slides that were presented since it is a lot of visual information that's also being presented here. And it was also just an opportunity for me to talk about some of the philosophical implications of VR, because one of the things I've been finding in talking to all these different experts in these different domains is that they're kind of very focused on their silo and don't necessarily are thinking about other dimensions, let's say the body or emotions, if they're a neuroscientist, just as an example. And so looking at this Neoplatonic framework of the four elements allows me to say, OK, well, let's try to look at all these different dimensions on a little bit more equal footing, because in order to create a really immersive VR experience, you have to think about things holistically. And I think that the elemental theory of presence can be a really useful framework for experiential design. But I think this speech in particular is going to be one that I'm going to be pointing people to to help get them ramped up and contextualized as to where we've been and where we're going with VR, but also a mythological story that we can think about the deeper meaning and purpose of trying to connect our inner subjective lives with our outer lives through the medium of virtual reality and human experience. So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends. And you know, my Patreon has been a crucial part of me being able to do this type of high-level thinking of having the luxury of going to all these different conferences and talking to all these people. And I really do rely on this Patreon to be the foundation of my coverage and my support and my livelihood. So if you enjoy this podcast and this information as a service to the community, then consider becoming a donor. Just a few dollars a month makes a huge difference. So donate today at Patreon.com slash Voices of VR. Thanks for listening.
