Jeremy Bailenson founding director of Stanford University’s Virtual Human Interaction Lab, and his latest book Experience on Demand traces his academic journey through virtual reality. It’s an intellectual memoir that focuses on his personal work in VR, and the insights that VR provides into human communication dynamics, as well as the impact of VR on our identity, empathy, education, medicine, and our ability to understand complex issues such as global warming and our impact on the environment. (UPDATE: Becoming Homeless: A Human Experience is now available on Steam.)
I had a chance to sit down with Bailenson to talk about his journey into VR, the major insights that VR has provided into human communication, and how STRIVR, the company he co-founded, is moving from training elite quarterbacks in the NFL to landing major corporate training contracts including training Wal-Mart employees. STRIVR is gathering one of the most robust data sets for using VR for education and training, which is enabling them to build statistical models to make connections between unconscious biometric gaze data and the process of learning. We also talk about how AI and machine learning will help build powerful models for biometric data, but also some of the privacy implications of this data as well as what we know and don’t know when it comes to the risks and dangers of virtual reality technology.
LISTEN TO THE VOICES OF VR PODCAST
This is a listener supported podcast, considering making a donation to the Voices of VR Podcast Patreon
Music: Fatality
Support Voices of VR
- Subscribe on iTunes
- Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So Jeremy Bailenson is the founder of Stanford's Virtual Human Interaction Lab, and he's got a new book out called Experience on Demand. And it's kind of like a memoir of his academic journey and work within virtual reality, trying to test the edges of what's known and what's not known. So Jeremy went to Stanford to start the virtual human interaction lab in order to push forward what's possible from a communications perspective with virtual reality. And so he's been very focused on a lot of like body transfer and how much of your body language can you communicate, what are the social interactions that you can have, what's the potential for interactional synchrony and mimicry, some more technical bits about communication within VR. And then after he's gotten tenure and moved on into having a little bit more freedom to explore what the potential of VR is, he started to look into issues of empathy, as well as the larger issues like global warming and climate change, and how can you use virtual reality in order to communicate the complexity of some of these larger issues to people so that they can actually change some of their consumer behaviors. So I had a chance to read the book and sit down with Jeremy to talk about his journey into virtual reality and some of his major work that he's done, as well as where he's going in the future. So that's what we're covering on today's episode on the Voices of VR podcast. So this interview with Jeremy happened on Friday, January 5th, 2018 at Stanford University in Palo Alto, California. So with that, let's go ahead and dive right in.
[00:01:44.554] Jeremy Bailenson: My name is Jeremy Balanson. I'm the founding director of Stanford's Virtual Human Interaction Lab. My PhD is in cognitive psychology, and I switched fields in the late 1990s. I switched from psychology to I took a postdoc where I learned how to build from a hardware standpoint and how to program from a content standpoint virtual reality. And at the same time, instead of asking cognitive mind based questions, think about bigger, more social implications type questions at UC Santa Barbara. In 2003, I came to Stanford and I founded the Virtual Human Interaction Lab. And here, what we do is we build VR, but we build it not for the sake of it. We build it to test to see how VR affects the mind. We run psychology experiments on people while they're inside virtual reality. And we study and build applications to try to help people become better.
[00:02:35.048] Kent Bye: Great, and can you talk about the first time that you actually experienced virtual reality and that moment that you decided that this was the thing that you really wanted to do?
[00:02:44.117] Jeremy Bailenson: For me, there's a few threads that get there. The first thread is the book Neuromancer, which is one of the two texts that I signed in my VR class here at Stanford. And I read Neuromancer in high school, probably didn't really get it. For those that read Neuromancer, I always encourage a second read because it really pops on the second read. In 1999, I was in graduate school and I was doing cognitive psychology where I was running experiments to see how the mind worked, for example, how people formed categories, how they did reasoning. And it just wasn't for me. There was lots of other people who were in the space and a lot of them, frankly, were better than I was at that. So I was kind of floundering to find my academic home, and I reread Neuromancer at that time, and something just really clicked. You know, with my study of how the mind worked and the methods and the techniques I'd picked up with programming, I thought it would be really neat to shift careers to go to virtual reality. Now, to directly answer your question, the first time I tried VR was in 1994. I was interviewing for graduate school at Berkeley, and I had an extra day where I was kind of wandering around San Francisco. And at somewhere down on the Embarcadero, there was an arcade that was showing a game called Dactyl Nightmare. And at the time, Dactyl Nightmare, I don't know the exact frame rate, but it was really low frame rate. The latency was, you know, probably measurable in seconds as opposed to milliseconds. That's an exaggeration, but probably hundreds of milliseconds. The tracking was off, but it was still so mind blowing. It kind of stuck and made me think that this is something that I need to consider.
[00:04:11.748] Kent Bye: So yeah, the name of the lab that you started here at Stanford is the Virtual Human Interaction Lab. So you're focusing specifically on people, on humans. And so you're coming from a background of psychology and looking at issues of communication. So maybe you could talk a bit about some of the early experiments that you were starting to do and some of the questions that you were trying to ask in terms of what you could learn about the nature of humans in communication and psychology through the medium of VR.
[00:04:38.090] Jeremy Bailenson: So the early work here at Stanford, a lot of it was driven by this quest for what we call tenure. And getting tenure anywhere is hard. At Stanford, it's so hard that it's almost laughable when you arrive here as a young professor and you're surrounded by Nobel Prize winners. And the quest that you're on is you need to discover something that changes the field. And I still don't think I did that. But what I chose to focus on was a theory called transformed social interaction. And what we were asking was basically, in a world where we are communicating via avatars, where all physics can go off, where you can change how you look, you can change the way eye gaze works, you can break physics off, you can literally be at three places at one time, in a world where there are no social rules, that's half of the equation, The other half of the equation is, Kent, as you know, with VR, the brain treats it as real. When you told me that you'd done The Void, I just recently did this mind-blowing demo by Nomadic VR, which is similar. It's utterly overwhelming, and we treat it as real. I wasn't studying presence in the sense of, you know, how real does a pit feel, or does it feel like that alien's a real alien? What I was studying is, in a world where you could break all social rules and literally change the nature of human communication, And that same world where the brain treats avatars as fairly realistic in the sense that we respond to avatars, even when they're breaking social rules as real, what does that do to communication? And let me ground that in some examples. Some early studies that we did, one was about non-zero-sum gaze or augmented gaze. In the real world, I can only look at one person at a time. But in VR, because we're networking avatars and you can tweak the outgoing tracking data, you can literally be a teacher in a class and looking at 10 students at once. Each student can believe she's the only one receiving your direct eye gaze. And what you can do is augment your ability to make social connections with people. And some of our early studies would show that when you had somebody implement augmented gaze compared to a normal tracked avatar that can only look at one person at a time, the overall social situation improved where you'd be more persuasive or you'd have better learning. And we had dozens of studies like that. For example, you could change your face to subtly subconsciously make your face look like someone else, or you could mimic someone's gestures so that there's this kind of nonverbal connection that we call synchrony. What we studied when I arrived at Stanford was how can you study the social world, the classic communication phenomena like interactional synchrony, like mimicry, things like eye gaze, like the bread and butter landmarks of nonverbal communication. How does that change in a world of avatars?
[00:07:16.780] Kent Bye: Yeah, and one of the effects that you had mentioned that was really interesting is that, you know, as you start to embody these different avatars and characters, you start to almost like subconsciously start to adapt those characteristics that you may be projecting or assigning to those avatars. So what is happening there when you start to embody avatars or characters that are different from you? Why is there this change of your behavior?
[00:07:39.218] Jeremy Bailenson: So the recent work in this domain is called body transfer. And one of my academic heroes, Mel Slater, he does a lot of great work in this area where he studies the neuroscience behind when you wear an avatar, what happens in the brain from a mechanism standpoint to make it adapt. Before that work was popularized by a number of neuroscientists. Nick Yee, my former PhD student, he coined something called the Proteus Effect. And Proteus was a Greek god who could change his form. Nick's work early on, and we started doing this work in 2003, asked the question, when one wears the body of an avatar, how does that change the way they behave? Now, the framing, the theory for this is a theory called self-perception theory. Self-perception theory, decades old now, states that, in general, the social world is hard, and humans, as they walk around the world, a lot of the cues we take on how to behave in a given situation comes from ourselves. In other words, of course we react to others, the place, what other people say to us, but it turns out that we also look to ourselves for cues. For example, when you dress up and you wear a suit and a tie, that's going to give you a bit of confidence. And as you're figuring out things to say or subconsciously a tone of voice, simply by looking down and subconsciously picking up these cues of how you're dressed are going to change the way you behave. One of the more famous studies by Valens in the 1960s, he had males look at Playboy centerfolds, and they were hooked up to a physiological monitor while they were looking at the centerfolds. What he did is for half of the models that the men were looking at, he would increase the heart rate on the sensor. In other words, for half of the models, the males would see an accelerated, exaggerated version of their own heart rate. In other words, they would see their heart beating faster than it actually was. Later on, he'd have them rate the attractiveness of all the models they looked at. And what he demonstrated in this early study was that the models that were paired with a higher heartbeat were rated more attractively. The subjects in that study literally interpreted their own bogus heart rate as a cue for them to figure out if the model was attractive. So that's probably one of the more famous early studies. So what Nick did, he came to the lab. He was the first PhD student that I ever had the privilege of working with here at Stanford. And we built the virtual mirror, the virtual mirror, you walk up to the mirror, you move around physically, your avatar moves synchronously with you in the mirror. Of course, those who are playing VR games, we see this all the time now in games, it's a great way to get you connected with your avatar. So the first mirror we built here was in the year 2003. And what we would do the general paradigm is you'd walk up to a virtual mirror, you would Do what the neuroscientists now call body transfer you'd move around for a few minutes and you would see yourself in the mirror And we typically start where you are your own identity say I'm a white male my avatar would be a white male We would then switch where you would become a woman or you would become a different race or we would change your age In fact, the first study we did was about ageism where college-age students walked up to a mirror and they saw themselves wearing avatars that were senior citizens likely in their 60s or or 70s. So half of the experiment is you'd do body transfer, you'd become older in the virtual mirror. The other half is you turn around and we would network a second person into VR with you, who was a part of the study. And there'd be a conversation or social conversation to reinforce your new avatar identity where you would We experienced some level of discrimination. In this instance, it was a job interview in Nick's first study with Ageism, and some of the questions were about your memory, and it was very clear that your new identity was being targeted as you had this conversation. And in that first study, what we demonstrated was that there was less implicit bias after the experiment ended. Meaning all people in their 20s are a little bit ageist. It's not a bad thing to say about people. We just have stereotypes about how we view the elderly. It's not that we're bad people. It's just associations, subconscious ones that we have. When you become older in the mirror, these subconscious biases get reduced.
[00:11:45.072] Kent Bye: Yeah, and another thing in the research that I see here at the Virtual Human Interaction Lab is on empathy, both with humans, but also empathy with, you know, expanding this fear of compassion out to either animals or, you know, into the earth with different issues around climate change. And so, You talk about in your book, Experience on Demand, how there's a couple of different models for, there's a cognitive model for empathy, there's an emotional model, and then there's perspective taking and working with different professors here around empathy. I'm just curious to hear from looking at the issue of empathy, both with humans and animals and beyond, what you've been able to kind of gain insight into the nature of empathy after doing it through the lens of using virtual reality.
[00:12:28.596] Jeremy Bailenson: So the empathy story, I'll tell a bit of the sociology behind how this happened. When Nick and I first did this original work, it was really more about the pure science of it. So we were fascinated by the question, if you wear an attractive avatar, do you become more confident? If you wear a taller avatar, will you negotiate more aggressively? It was really about just being fascinated by the question, do people adapt to their avatars? In 2010, the lab's research started to shift a bit and part of that was driven by the fact that I'd received tenure and publication in top-notch journals wasn't as important as doing work that I was really just utterly excited to do and work that I thought was really going to help the world and people, not to use a Silicon Valley cliche, but we really wanted to to feel good about the work we were doing. Not that we didn't feel good with the other work, we want to feel great about it. So we took a lot of the work that we'd been doing, which was this basic science stuff, and said, well, how can we apply this? And there's a really fun story where a woman came to the lab. Her name was Deborah Bay. She was a project manager, program officer for the Robert Wood Johnson Foundation. And She got a lab tour where we showed her the virtual mirror. And we showed her some of the empathy studies where you'd become a cow, and some of the studies where you would experience hardship. And what she said to me, she said, Jeremy, I believe your findings. But there's two things that that I'm not convinced of. One is does it work on real people? And what she meant by real people is most psychology studies are published with small samples of sophomores at a college. And we wanted to expand that. And so she challenged us to use larger samples, more diverse samples demographically. The second thing is that she said, I want to see how long these effects last. I believe that you can bring someone in the lab have them become a different race and experience some traumatic event. And 20 minutes later, they're going to demonstrate behavior change. How long does that last? And so what we are about to publish, it's not out in the journals yet is a two study paper that's a massive endeavor run by Fernanda Herrera, my PhD student who's doing the great work here. And this work is, study one is over a thousand subjects where Fernando went to museums and to flea markets and shopping malls and senior citizens' homes. It's the largest, you know, I can't know this for sure, but as far as anything I've seen, the largest, most diverse data set in the history of VR where we've got lots of people experiencing these treatments and studying not only things like presence, how real they felt VR was, but how effective the empathy manipulations were on causing them to change attitudes and behavior. That's study one. In study two, we also had a large sample, not as large, this was north of 100 subjects, where we looked at the same people right after they experienced the empathy demo, two weeks out, four weeks out, and eight weeks out. And in both studies, let me take another step back sociologically, when Deborah Bay, the program officer from Robert Wood Johnson, challenged us to really, and the context here is, it's a really popular narrative to say VR causes empathy by myself and by others, and I think a lot of us, we just want it to be. But what this project sought to do was to be devil's advocate and to really come up with the most rigorous control conditions you could to really start with the notion that maybe it doesn't cause empathy, and try to design a study that really, if we did find an effect, that would be something that would be fairly convincing from a doubting scientist. And we chose to collaborate with Jamil Zaki, who's a professor in psychology here, who's one of the world's experts on empathy, has written, in my opinion, the best, if anyone wants to understand the psychological science behind empathy, in 2016 he published a paper in Psychological Bulletin that I recommend to everyone, it's online on his website. So we paired with Jamil, who, I don't want to say he was a VR doubter, but he's not a VR guy, and he certainly didn't come in with an assumption that VR was going to blindly cause empathy, and he was a really good partner to have in this work, because for that first large-scale study with a thousand subjects, we chose three different control conditions, and all three were chosen, you know, maybe one was chosen to be a straw man, meaning a pure control where you didn't expect much empathy, but the other two control conditions both were very carefully crafted chosen ones that should be inducing empathy. And so now circling back to the results of these two studies, what we demonstrated, so the studies themselves were what happened with the subject experience was a 10 minute journey that Kent, I know you did at the Tribeca Film Festival. that we're going to release on Steam on January 30th for the world to do called Becoming Homeless. And Becoming Homeless, it's based on a psychologist here at Stanford, his name is Lee Ross. And Lee Ross decades ago coined a theory called the fundamental attribution error. And in psychology, what this means is that when something bad happens to someone else, and you see that, typically the normal human response is to blame that person's character or blame the person themselves. When something bad happens to us, on the other hand, we don't blame our own character, we blame the situation. So to sum that up, when others experience a bad event, we blame them. When it happens to us, we blame external factors. And this is just in general how humans tend to interpret negative events. What we wanted to do was come up with a simulation to reverse that, in particular as it relates to those who are homeless. And we created this journey that is based on a lot of due diligence, working with organizations that help support the homeless, doing dozens of interviews with people who were homeless, watching documentaries, really well-founded from just the truth and the actual sociology side of homelessness. The journey, you start out in your house, and you get fired from your job. You can't make rent. You try to sell items in your house. You get evicted. You then are living in your car. You're trying to survive living in your car, but that's illegal. Eventually, you end up having to sleep on this bus. And it's a journey that's pretty intense and arousing that's designed to highlight the situational factors, basically saying that not all homeless people are homeless because it's their fault. What we measured from an outcome variable was whether or not people would sign a petition. In other words, we weren't going to just ask people how they felt. There's nothing wrong with doing that. But in my lab here at Stanford, we always strive to measure behavior. And what we did is we would hand them a petition. It was on a clipboard. It was outside of VR. And the petition was supporting an initiative, a referendum, that basically said, are you willing to have your own personal taxes increased to build more affordable housing so that homeless people can get off the streets? And we measure the probability that they are going to sign this document. In both of our studies, immersive VR caused more people to sign this document compared to control conditions that were really carefully crafted.
[00:19:05.142] Kent Bye: Yeah, yeah, I had a chance to go through that experience at Tribeca and at the end I did sign the form. I didn't know it was sort of like a fake form and the other experience that I did back at Sundance last year from Condition One was this experience where it's a 360 video where you're going into a chicken farm and you're like seeing the conditions there and you're almost getting like this guided tour of the conditions there of chickens that are in these cages and in this specific case they were breaking the law and they weren't supposed to be in these cages and so at the end they showed the chickens in the cage-free environment and like it actually changed my consumer behavior. such that when I had the option to spend a little extra money to buy the eggs that were treating the chickens more compassionately, then that's what I've been doing since then. And so I think that's the challenge of these big, large issues is to kind of boil it down to that choice or that decision of that consumer behavior that each individual can feel like they're having some control over the overall issue, whether it's global warming or whether it's eating meat. And so I'm just curious to hear from your perspective of like trying to create virtual reality experiences that give people an embodied experience of something that is very large and complex, but kind of boils it down to the essence of something that they can actually do and change their behaviors.
[00:20:23.535] Jeremy Bailenson: So this is part of the challenge with, I like to honestly talk about the limits of our work as well as the benefits. And let's talk about climate change for a little while as a context here. In my opinion, VR is spectacular for moments, teachable moments, aha moments. The name of the book is Experience on Demand. And I think when VR is done well, we treat it as an experience, both in terms of how we talk about it, the studies coming out of my lab and others that show that the brain reacts in ways that are similar to actual experiences. I think of it as an experience. Climate change is really complex in terms of the science behind it, understanding how you can prevent it, understanding how it's going to affect you. There's so many parameters of space and time. You know, when you think about this general paradigm of using technology to help people understand experiences better, a lot of the times we think back to gamification and using these serious games or learning games to help us understand. And when you're trying to do systems level research to understand, well, over time, if I used this much fossil fuels, and then everyone in my town did, then how's that going to affect the overall climate change? That's really good for video games, maybe not as good for VR. I really think that VR is really good for intense moments. Anyone who's listening who's actually built VR experiences know how brutal it is. And I'm not saying that when you put $100 million into it the way that EA is doing it for video games now, we can't get there. But where we're at right now is I believe that if you're trying to teach a really complex system, I think a lot should be done by reading, a lot can be done by video or more interactive 2D displays. And then you put on the VR helmet for a really intense moment, like this is what your hometown is going to look like if we keep being this way. And I can tell you a recent example. We just went to Palau and Palau is a small island in Micronesia. It's spatially the size of Portland, probably smaller than the size of Portland. but due to nautical law, it owns a chunk of the ocean roughly the size of France. Palau is famed for two things, the most stunningly gorgeous coral reefs that you can fathom, just unbelievably great, but also one of the nations that is directly threatened by climate change, both centrally due to sea level rise, because the island is not that high, but also this network of islands has 20,000 citizens roughly, and takes about 100,000 tourists per year. The tourists come to go scuba diving and snorkeling mostly. There's not that much to do on land from a tourism perspective. So these reefs are critical for the island's survival. Now climate change is eventually going to make the entire network of islands be underwater, but in the shorter term and not that far away, due to the rising temperatures and the acidification of the waters, the economy is going to be ruined because no one's going to come see the reefs. So I had the privilege of flying to Palau with two great scientists, Rob Dunbar from Stanford, who's a climate change scientist, a number of others. We basically went to Palau, we went on a boat for about 10 days straight, and Tobin Asher and Elise Ogle, who are full-time staff in my lab, we tagged along. We were there for two weeks, about five, six hours a day on the boat, and we would go not scuba diving, but snorkeling on these spots that would basically show the before and the after of climate change. They would show what happened to storm damage, what happened due to farming where areas got cleared and there was damage and acidification. Now Palau, they can't do anything about climate change. In other words, they can't reduce their carbon footprint and that's going to make a difference. There's only 20,000 people that live there. So they are forced to do something called climate change adaptation. They can only adapt to the consequences. And so what we did there is we filmed, you know, we have so much footage, we're still putting it together. This is going to be, by the way, at the April 2018 Tribeca Film Festival, there's going to be one of our two entries there. And I'm really excited about it. It's going to be an educational piece about climate change adaptation. But what we did was we captured all these touch points. And I'm going to tell you about one of them. There's a place called Soft Coral Arch. And Soft Coral Arch, it's stunning. It's basically, for those of you that have dove tropical sites before, coral reefs, most of them are in the sun. Because of this rocky arch, it's this massive coral reef in the shade. And because of the shade, the coral just grows differently and there's different species. It's this long, flowery. And so we got there and we filmed the scene really early in the morning. It was pristine and just gorgeous. We got this amazing underwater 360 footage. Then at about 8am all the tourist boats started hitting and these tourists weren't scuba divers, they weren't trained snorkelers. What they would do is they would all jump off a boat wearing snorkel fins and a mask and they would all grab onto a raft with their hands on the raft and their feet facing outward like a sun. And they would float through the arch. Now most of these people couldn't even swim, and what we caught with the underwater footage, and when I say caught, this is not a hard thing to catch, this is the norm there pretty much during every hour of daylight, were these tourists kicking their legs, smashing the coral off, running into the walls, and you could literally see the coral getting just knocked off the wall. Tobin and Elise pulled an all-nighter, and they stitched all the underwater footage together, and they put together maybe a 45-second demo. And the 45-second demo, maybe it was 90 seconds, was being in the Arch Wallace Pristine, the next day we had the privilege of going to present to the legislators of the nation of Palau. They've got senators and house delegates. We probably had about half of all the lawmakers, or more than half, maybe two-thirds of all the lawmakers. Imagine getting half of the US Senate and half of the house here to experience VR to understand climate change. And what we did is we'd have a person come in, and they would experience their underwater scenes. Now, this is important for the lawmakers of Palau because culturally there, they don't scuba dive, and they don't snorkel. And in fact, of the 12 legislators that I put through the VR experience, only one had ever been snorkeling or diving before. There's a culture of fishing and swimming in shallow waters, but they don't like to go. So most of them hadn't seen the treasures that are fueling their economy. And what our demo was, is we put them in the helmets and we would have them just look around Soft Coral Arch and just kind of marvel at how great it was. And they liked that part. Then we would have them look down at their feet at this particular peak of orange coral. While they were looking down at their feet, we would swap the 360 video to the one with all the tourists there. And the tourists, you can't see them initially. And then we'd have them look down from their feet at this orange piece of coral and then look up at the water above them. And what they would see is this monstrous cluster of 20 tourists clinging to rafts, kicking their feet, knocking the coral off. And the reactions we got were just unbelievable. ranging from we need to make this VR demo mandatory for every tourist whoever comes here to go snorkeling to we need to put this in our schools. And if you want to read about this, I wrote a piece for National Geographic. I need to update it because they just passed a new law about tourism in Palau about how to kind of manage that. And so this is a long-winded answer to your question. And I guess I'm describing my process of building this nice piece for Tribeca Film Festival in a few months. there's a story about what climate change adaptation is. So one thing Palau can do to adapt to climate change adaptation is to help manage tourists, to teach them not to destroy the reefs. The second thing they can do is to have more protected areas where no one can go, for example, to do overfishing. The third thing they can do is to stop commercial farming as often, because when you do commercial farming, you cut down the mangroves. The mangroves prevent sediment from going to the water. The sediment destroys the reefs because of how they interact with the coral. So there's probably other things they can do to adapt to, but the three main things that they can do is to stop cutting down the mangroves, have more protected areas and to help tourists. Back to your question, which is how do you use these interactive systems to teach people? As we're designing this VR experience, I believe it's a home run to show somebody the before and after of the tourists cutting down the reef, the sediment when you cut down the mangroves. So the demo that we're doing for Tribeca, when you were going to have them physically cut down a tree using their hands and a haptic device to cut down the mangrove and then they're going to go in the reef and see actual footage where the sediment is high and you can't see anything and there's no coral versus ones where the mangroves are there and all the life still exists. So I believe you are spectacular for those moments. Now, if you're trying to teach somebody about the complexities of how ocean acidification is caused and how it's going to affect people, I'm not convinced VR is the holy grail on that sense. To me, if I had the president in the room and I was trying to convince him about ocean acidification, I'd try to teach him about the science on a chalkboard and then I'd show him the really intense visceral stuff in VR.
[00:29:10.262] Kent Bye: Yeah, I noticed in the experience that's on steam around the ocean acidification that there's actually some moments as a user you have a chance to touch the exhaust pipe and then you see the carbon dioxide come off and then you have the chance to place flags on snails and the before and after and so it seems like you are engaging the user into these immersive and interactive experiences that's not just completely passive, that you're actually engaged in doing these little tasks that aren't really meaningful in any way. It's just sort of like to get you engaged within the story that you're telling. And so is this sort of inspired by principles of embodied cognition such that, you know, when you're actually kind of making choices and taking action and being in the context of an environment that it's actually more of a rich experience that you remember more vividly?
[00:29:56.867] Jeremy Bailenson: The ocean acidification experience is a demo that we have used to study embodied cognition now for years. So the experience, it's about, you know, eight minutes long, you do a number of tasks, but you're basically walking around a four by four meter space, and you are doing a number of activities, you're counting the number of sea snails that are there, you're putting your hands over vents and feeling the bubbles that's designed to show you about how carbon dioxide affects water, we're encouraging you to move around. Now, over the last few years, we've run 1000s of people through this and collected data, we've done it in high schools, we've done it in college classes, we've done it at big film festivals, for weeks at a time, we've got 1000s of data points here. For every person that's gone through the study, we've got two things. One is the outcome measure, the outcome measure could be a test to see how much they learned about acidification, it can be putting their name on a form, do they sign a form to support conservation efforts or to receive more information? That's the outcome measure. On the input measure, we've got their tracking data, how much they've moved, how much of the space they've explored, how many snails they chose to tag in the species count. And we just submitted a four study paper that's not published yet. But the overall theme of this paper, with thousands of subjects going into what I'm about to say is, In general, there's a correlation between the amount of movement one does in VR in these scenes and how engaged they are with the outcome measure. Now, it's correlational, so we don't know if it's people that love the environment also are going to sign this form at the end and they choose to really explore the space, so we can't prove causality in this instance. But in general, what we're finding in these studies is that the more movement there is, the more intense movement, the more exploration, the more likely people are to show engagement in your behavioral measure later on. The theory, one of the theories we use to ground this is called embodied cognition. Embodied cognition states that, of course, we know that the mind is in the brain, but it turns out that your body also has an impact on how we think, on cognition, on attitudes, and in general, the way the body moves during these experiences is, in our opinion, which is based on a decent amount of data at this point, it's really critical to look at how the body moves when you are examining outcomes of a VR experience.
[00:32:17.072] Kent Bye: Yeah, well, and of course, with the work with Stryver that you've been doing is, you know, all about allowing the quarterbacks to kind of be in the context as well. So it's the environmental as well. It's not just our body, but also sort of the context of the environment. You're kind of responding to what's happening in the environment that changes your way to think. And I had a chance to talk to Michael Casale a couple of times the last time, talking about how you're now sort of going into and teaching Walmart employees and kind of taking these techniques of embodied cognition and training through virtual reality, but to be able to teach different social skills. And maybe you could expand on that evolution from training high-end quarterbacks into sort of Walmart employees and what the common thread there is into this driver system.
[00:33:01.982] Jeremy Bailenson: So Stryver is the most stunning data set of VR that I've ever gotten to see in my life. And it's just been amazing watching it unfold. So the company began with Derek Belch, my master's student in 2014. Derek was an undergrad who took my classes in the year 2005. His master's thesis was using VR to train Stanford quarterbacks. So he was a master's student in my lab while he was also an assistant coach on the Stanford football team. And we had this amazing access where we got to build and test these demonstrations on the Stanford football players. And what we were teaching the players was not how to kick or throw or run. They already know how to do that, but to give them extra mental repetitions. So it turns out in American football, even the best teams, the best NFL teams, they only get a little bit of time on the field doing 11 on 11, which is the entire defense playing against the entire offense. They don't get much actual scrimmage or practice. And so what we decided to build was just a trainer where they got to relive complicated moments and to practice mental rehearsal. So the quarterback in American football, before the ball is snapped, he goes to the line of scrimmage. And in those second or two or three before the ball is snapped, he's got to look around the defense and he's got to recognize a pattern. What defense are they throwing at me? Is this going to be a complicated blitz? And then he's got to make a decision based on that pattern recognition. It can be let it roll, which is keep the original play. He can kill, kill, kill, which is go to the next play down the queue. Or he can, for example, move a running back over to pick up a blocker. So those are three parameters that a quarterback has. All of this happens very quickly. He has to recognize a pattern, make a decision. Now, I could talk for a long time about how this evolved in football. Coach Shaw at Stanford saw how well it was working after three weeks. He made it mandatory for the players to use. Towards the end of that season, we had a breakout season that no one expected. The coach and the quarterback of the Stanford football team, both attributed the use of VR as one of the many reasons they succeeded so well that season. Derek graduates January 2nd, 2015, forms a company called Stryver on January 3rd. Like many Stanford students, they leave, they form a startup, and to everybody's surprise, a few months later, there are many NFL teams signed for multi-year contracts, and this thing takes off. The special thing about this from my perspective as a professor is for the first time in VR history, and you know, I could be wrong here, but I am fairly in tune with a lot of the academic research and a lot of the applied research done in the military. We have a data set of not dozens and not hundreds and not thousands. We literally have, just in football, over 100,000 plays watched. I mean, think about that. That's a lot of VR experiences of people going through these simulations and making decisions. So for all these players, we know when they're using VR. We know how long they've used it. We know where their head has turned, where they've looked with the pitch on roll data while they're looking around the scenes. We know where they've paused. So we've got all this tracking data. Football is such a big business that every single player, when they play on the field on Sundays for pros or Saturday if it's college, every single player receives a grade for every single play, regardless of whether or not he's touched the ball. So for those that are familiar with machine learning, you may know the common adage, garbage in, garbage out, which is you've got to have really great training data in order to make things work. We've got the best training data in the history of VR. We've got all these use cases for every single training session. We know when they've used it, where they looked when they used it, what time they used it. It's an amazing data set. And then we can formulate an output layer based on how they did on the field, even if they didn't touch the ball. And what I've got to witness with Michael Casale as our brilliant data scientist doing the real work here is building these really intense statistical models of how can you define engagement, how can you figure out whether learning is going to transfer from VR to the physical world based on not dozens of cases, but just thousands and thousands of cases. it's been just utterly stunning to watch us build the statistical model to show how learning works in VR and sports. But then it turns out, every skill that I just mentioned, which is being in a rousing situation, have to look around a space and then make a quick decision. Every single person who's got a job has to do that. And we'll talk about Walmart, there's about five to 10 big companies using it now, some of which I can mention on this, some of which I'm not allowed to mention yet. Walmart is the biggest company in the world. And the way that Walmart finds out about this, it's again, the sociology of this stuff is always fun to talk about. One of the football teams we work with is Arkansas, the college team. And one of the main brains at Walmart in training, he happened to be visiting the coach of the Arkansas team that day. And he saw this, he said, huh, I think there's a way we can use this to help Walmart. And the journey with Walmart, it starts in one single training facility. So Walmart has these things, they're called the academies. And the new CEO of Walmart has come to the lab. And in my opinion, it really gets it as a really forward thinking guy. He decided that every single Walmart employee should be able at any point to get in her car and drive to a training academy to become better. And so they strategically have built 200 of these things, all of which you can drive to if you work at Walmart. And the point is you don't need to be a fancy person who owns a jet to be able to go to one of these. Anyone can drive there. So we started at one of these training academies, and we ran this qualitative, we'll call it qualitative, because there was enough data to really do the kind of math that makes Michael Casale happy. We put VR in one of these training facilities. So what is the kind of thing that you would train on? Well, we would put you behind a deli counter, and you would have to figure out how to do your job better. So is there a sharp knife out that a customer can cut herself on or employee can cut himself on? Is there a long line of people and you haven't I checked somebody in general look around, recognize patterns and make decisions and do actions and it turns out it works I mean so let me take a step back here and VR is not for everything and one of my pet peeves with a lot of VR simulations is they don't involve using the space itself. And so I can always say if somebody is in VR and you're watching them wearing the helmet and all they're doing is standing still and looking straight ahead, that thing probably didn't need to be in VR. And so with all the, why it works so well with quarterbacks and other football players is that they have to look around the field, look all the way behind them, above, down to see these patterns. And what we tried to do was to leverage those scenes for Walmart as well. So one of the Stryver executives, his name is Logan, He read the entire back-to-back Walmart training manual, and of the thousands of scenes we could have chose to build, he chose about 10 that were really suited well for VR, which is use the space, look around, super intense and arousing, and have to make a quick decision. The deli counter was one of them. Probably. So Kent, you mentioned that you've done some pretty scary planks in your time. I have too. I think the scariest thing that I've done to date in VR is experiencing what Walmart calls holiday rush. Others call Black Friday, which is lots of customers really angry coming at you, not physically, but verbally and getting in your personal space and giving new employees the opportunity to experience that and teaching them the right strategies while they're going forward. So we started one Walmart Training Academy and got really good results from, you know, the exit surveys from the employees and from the trainers. They really liked it in terms of the engagement and the learning. What we then did is we went into 30 of them and we put a lot more VR simulations in 30 of these training academies and people typically come for about a week and they would experience these simulations and lots of people rowing through them so you're starting to get large quantitative sets of data now. And Walmart was forward-thinking enough to pair the 30 training academies that use VR with 30 that hadn't, matched to roughly give you a good comparison set. to do some statistics later on. And what we demonstrated was better training, better transfer of the skills learned, and just in general, better results in the academies that had used the VR compared to the ones that didn't. We are now in all 200 of them. And so just in the year 2017, you know, north of 100,000 Walmart employees got to use VR and to train to be better at their jobs. And You know, I've been doing VR for 20 years now, and it's really neat to see not just fancy labs like Stanford or other places or executives of companies or quarterbacks, but just anyone gets to use it. And it's really neat to see a use case that a works and is really well received and appreciated by the people using it. But from a scientist standpoint, boys, this is a fun data set. I mean, it's. hundreds of thousands of use cases of people who are learning their skills in VR and then we can later on pair their VR training with performance evaluations that are given to them anyway. It's just really neat.
[00:42:10.312] Kent Bye: Yeah, in talking to Michael Casale, I was sort of asking him questions about, you know, evaluation or how you know what you're doing is successful. And it seems like that, you know, you're going through and you're you're teaching people these things. But there's also like this most qualitative assessment, this performance reviews is one way but I noticed that you teach a class on the philosophy of science, and I think a lot about the limits of reductive materialism, about being able to objectify things, and how virtual reality as a medium actually has a potential to correlate our inner noetic subjective experiences into some sort of... If it's in a virtual environment, you're able to correlate that objectively to whatever someone may be looking at. And so there's these ideas of neural phenomenology, or just phenomenology in general, of being able to look at somebody's direct experience and see how that is impacting either their choices or behaviors. And so from your perspective of looking at the higher philosophy of science, does VR provide new avenues to be able to expand what data are even be able to be collected? And if that has like deeper philosophical implications for what is considered real or that you can back up empirically?
[00:43:19.239] Jeremy Bailenson: In the book, chapter eight is called Reverse Field Trips. And it's about using VR in the classroom to give people really amazing educational experiences. And one of the neat things about VR in terms of providing people learning experiences, you can track their movement. Now, what would you do with that? So from a professor standpoint, when I teach a class at Stanford now, I have maybe six or seven data points that contribute to someone's grade. I've got their midterm, their final, how brilliant they were when they made comments in class, their discussion section, attendance, and again, how brilliant they were in discussion. You know, on these large lecture classes, you get some data points. Now, imagine the entire class were taught in VR, or if some portion were taught in VR and they got to do some experiences, some labs inside of VR, even if they were using the commercial systems today, you'd have 18 degrees of freedom, six on the head, six on each hand, X, Y, Z, pitch on roll for each appendage. If you're recording the tracking data, even at 30 frames a second, which we can do at 90, if you're doing the math in your head after a 15 minute field trip, your data file is so big now you can't even open in Excel. It's a massive data file. And so a number of our published papers in the lab show that if you're trying to predict somebody's learning transfer, how well they know material, how well they're going to do, that what I call the digital footprint is really predictive when you take as input to a machine learning algorithm the tracking data from the learning session and the outcome measure, some measure of assessment. So the question that I pose to my students when I teach these VR classes and when I teach the philosophy of science class, when I've taught that, If you're an employer and you want to figure out, should I hire this Stanford graduate? You've got grades or you can have all of somebody's digital footprint, all of their data while learning. The question for you as a future employer, what would you rather have? Somebody's GPA coming out of Stanford or all of the tracking data for while they were learning the material? And pretty much every student who's taken my virtual people class and actually has seen what you can do with these predictive models will say, if I'm trying to figure out if somebody's actually going to be good at their job, the grades are much less important given this climate of grade inflation than literally knowing every single move they'd made while they were learning this stuff and then later on being assessed. Now, getting back to what I think is the gist of your question, from the student's perspective, I then say how many of you guys would rather forego your grade in this class right now with the midterm, and let me stick a Connect on you, and we'll use the tracking data because this is pre, we're not putting them on helmets in class right now. And how many of you would rather me figure out your grade based on your tracking data than the midterm and the final? And I've yet to someone raise their hands. And I agree with them. And one of the reasons is that students have gotten very good at taking midterms and finals and good for them. And that's a system they're comfortable in. But all of us have got this. unsettled, are we unsettled? Are we wary? Tracking data says a lot about you. And, and when you're in VR, you've got this double edged illusion, which is, I forget the physical world is there. I forget how my body is moving. That's the nature of presence. I forget I'm even in VR. But in the history of all data tracking of all ways that we can figure out what someone's doing, VR is perhaps the most detailed in terms of every single move you make, whether it's a a slight lean forward of three centimeters, or if it's a quick head pivot to kind of look in a certain place, it's all recorded. So there is an aspect of, geez, we should think about the nature of privacy in VR. One of the people who's my thought partners on this is Tom Wheeler, who was the former head of the Federal Communication Committee under President Obama, who is the champion of net neutrality. He's come to the lab and experienced VR and he'd actually experienced it long before he came to my lab. He's one of the few government folks that I've met at high places that was quite well versed in VR. And one of the things he and I talked a lot about was how do we think about privacy in this world of VR? And I don't have an answer for you. I can tell you that I'm thinking about it and I do talk to a lot of government folks and I'm helping them think about it too.
[00:47:31.423] Kent Bye: Yeah, well, that sort of speaks to some of the other themes that I see in your books is that, you know, there's the positive aspects of virtual reality, but there's also the potential dangers, whether it's through surveillance of corporate surveillance, or government surveillance, or in regards to privacy. but also just how much we don't know about this technology and what the long-term effects are, the effects of violence in VR, being able to see things on a 2D screen that we're very used to doing, and when we translate that into VR, it's a whole other ballgame in terms of stimulating our physiology and giving us this embodied experience that could actually either re-traumatize ourselves or give us new traumas. And so I'm curious to hear from you, what do we know about the potential dangers and risks of VR in contrast to all of the other amazing potentials that we also know about?
[00:48:20.873] Jeremy Bailenson: So my my thinking on this continues to evolve and I want to be careful on how I answer these questions Let me say first and foremost, and I sincerely hope you don't edit this part out I'm a firm believer in free speech and the US Supreme Court has ruled that Even really intense violent content is free speech and I do not believe that free speech should be regulated So I hope that stays in this podcast because it's important that that gets out there that being said I think we should consider what we want to do in VR really intense violence in VR, it feels a little different than it does on video games. And I applaud a lot of the game designers who are finding a way to walk the line to allow us to do what we like to do in video games, which is shoot and run around and to be heroes. without giving us muscle memory and giving us this brain experience of doing actual violence to people. So let me give you an example. I recently just got to do the demos from Nomadic VR, which is one of these location-based VR scenes where you're running around and it was fairly incredible. They had nice uses of fans and heaters and passive haptics. It was pretty cool. They made the choice of the bad guys, because you did have a VR gun that you were holding or a physical gun that you were holding, The bad guys in this were drones, drones in the sky that you were shooting. And to me, I feel like that's a good way to walk the line on the violence, which is that you're shooting basically clay pigeons that are robots. And I felt pretty good about that. You know, without getting too much into the desensitization of violence debate, because I think that's an important debate to have, but maybe we shouldn't go into that here. There's a separate level that VR offers. So when you think about the classic argument about video games and violence, that tends to be about desensitizing people that they become more likely to think about violent outcomes or more willing to do violence, given scenes, and lots of scientists have addressed that. I want to talk about something different, which is simply the muscle memory and the skill learning. I mean, so we talked about Stryver. I am really confident that VR teaches people better than other media. I've watched it. I've watched it with athletes. I've watched it with companies like Walmart. It just works more efficiently and the motivation to engage is higher. It's a neat tool. I also, my lab has been funded by various organizations of the U.S. military, and I'm proud to get to work with our soldiers, to have our soldiers become better at their jobs. What worries me is when citizens are going to have access to training simulations that better teach them how to succeed at violence. you know, so we can joke about the matrix where Neo learns how to do Kung Fu in 10 seconds with an upload. I don't think we're going to be anywhere near that for quite some time. However, the US military has been funding VR research for as long as there's been VR research because it's an effective tool to train soldiers. When citizens have access to great violence trainers on how to shoot and how to do combat. That's something that I think we need to think about.
[00:51:19.759] Kent Bye: Yeah, and so for you, what are some of the biggest open questions that are really driving your research forward from this point on?
[00:51:27.629] Jeremy Bailenson: So there's a lot of open questions that I'm not going to address with my research. And so let's stay in this negative thread for one second. I mean, if you think about VR overuse, you know, there's some fun Guinness Book of World Records, one fellow stayed in VR for over 24 hours. There's two German scientists, psychology professors, where one watched the other while he stayed inside of VR, I think it was for 24 hours. And that's an academic paper that's published. So there are some scientists that are studying that. But at some point, you can't do the science. So if you think about the argument that the cigarette companies had, the tobacco companies had, for quite some time, they were claiming that there was no causal relationship between smoking and cancer, because no one had ever run the controlled study where half the subjects were forced to smoke two packs a day for a couple decades, and half weren't. No scientist is going to do that. One of the questions that makes me curious is this question of, what's too much? What's too much to use? In my lab, we have basically a 20-minute rule. After 20 minutes, take the helmet off, look around, touch a wall, have a sip of water. You can go back in if you want, but remind yourself of where you are. No one has academically studied what are the effects of, you know, reality blurring, what are the effects of just changes to how you think and behave when people are using VR for hours and hours per day, days on end. And so that's a question that remains in my mind, but not something I'm going to do in the lab just because it just doesn't, it just doesn't sound like work I want to do. One thing we're starting to do in the lab now is to focus a bit more on augmented reality. We just published our first paper in augmented reality that's dealing with how people perceive virtual others, avatars and agents that you see using AR. So AR, it's a field in academia where there's been a ton of energy in terms of studying the perceptual issues behind it, studying how to design the displays such that people can have a good visual experience. There's been some studies on how to do learning a little bit, but what there hasn't been much of is the basic type of psychology studies I've done and my colleagues have done for quite some time on AR and The study we just published on AR, we're talking about social presence. So when a virtual character is integrating in your room with you, yet that character doesn't abide by physics, he walks through chairs, he can be a ghost, basically, how does that affect you compared to one that walks around chairs that abides by physics that can really register with the items in the room? And that was more of a kind of nuts and bolts question about how realistic character design should be done. The study we're doing now in the lab asks, what is the effect of an AR experience on your memory of a place? So if you think about for a second, close your eyes and think about the best thing that's ever happened to you and the worst thing that's ever happened to you. For both those events, you can probably really viscerally imagine the place. And if you went back to that place, you would think about that event. If the event was the best or worst thing that's ever happened to you, that place is arguably changed for you based on the associations you have with it. Now, in AR, an augmented reality, as we know, we are going to be putting virtual objects and events in physical scenes. Those physical scenes are going to be places where you are every day, your office, your home. I would venture to say that most of the things people will design for AR are not boring events. In other words, the reason why we go into VR and AR is because the events are intense. They're amazing. They're things you couldn't do otherwise. And so this study that we're doing right now is designed to ask the question, how does an event in augmented reality change your subsequent attitudes and behaviors towards the place?
[00:55:02.945] Kent Bye: Interesting. And finally, what do you think is kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:55:13.149] Jeremy Bailenson: So I'll answer that in two ways. I think that the best use case that we've seen so far, hands down, is in training. There's a reason why Stryver has gone from just Derek and I to 65 employees right now and just lots of companies that are using it. It's just the right way to do the job. But my dream application is communication. And so my early work as we began this interview is really about social interaction and communication. I want us to go back. When you watch a movie from the 1970s, you'll often see doctors smoking a cigarette while they're operating on a patient. And you look at that movie and say, what? Can you believe that guy smoking a cigarette in a hospital, like in the room with a patient? You watch that and you say, I can't believe anyone ever did that. I want us to go back 10 years from now and look at the commute. Can you believe people used to pile into these metal boxes and then drive behind one another for an hour each way just so they could sit down at a desk and type on a computer? If you think about 40,000 deaths in the United States last year, 40,000 deaths in car accidents. Think about the productivity loss, the road rage. just how obviously the damage to the environment. It's unbelievable that we're commuting to work. Now, there are some events, of course, that you need to be there for face to face important meetings, etc. And I'm not trying to say we shouldn't travel, we shouldn't go visit loved ones, we shouldn't go out to bars and hear music. I'm saying that there is a large portion of travel that is not essential. And I'm very depressed to say that there was an amazing call by ARPA-E, which is the research wing of the Department of Energy, that a number of us helped the government write, which was probably 10 to 20 million dollars that were going to go towards scientists who are going to help figure out how to perfect the technology to make it so that you actually didn't need to commute. And this was a call last year called FACES, F-A-C-E-S, and in it we were asking professors and business leaders to write grants to apply for this project where we can help to figure out how to perfect the systems that can perfect eye contact and the right posture and presence so that you can really feel like you're in the room with other humans in real time the way that the video conference just doesn't get you. And I'm sad to say that we'd gotten really far. We decided on some people that were going to get funded, I believe, and the current administration has taken away that research money. So this dream that I've had for quite some time, a lot of us have been doing it one group at a time. We had for the first time ever had the government putting their muscle behind the Department of Energy, who says, basically, we can't be 11 billion people as we're going to be fairly certain on this planet and have everybody driving and flying everywhere. That's just not going to work from an infrastructure standpoint. So let's think about the future. So I'm hopeful that anyone listening who might work for the government might resurrect the Department of Energy's ARPA-E call to help travel get eliminated by having VR replace some of it.
[00:58:09.608] Kent Bye: Awesome. Is there anything else that's left unsaid that you'd like to say?
[00:58:13.693] Jeremy Bailenson: I want to thank you for the work you're doing. I listen to your work and it's really good to have someone who does due diligence, who's got thoughtful interview style and keep it up.
[00:58:24.065] Kent Bye: Awesome. Thank you so much. Thanks. Thank you. So that was Jeremy Balanson. He's the founder of Stanford's Virtual Human Interaction Lab and the author of Experience on Demand, which is available on January 30th, 2018. So I have a number of different takeaways about this interview is that first of all, I was really struck by Jeremy's path into virtual reality and some of his driving motivations and then how that's changed over time. I think A big part of him getting into VR was that it was new and different, and it was an area where he could really innovate and really push the boundaries of what's known in terms of human communication. Very interested in some of the social dynamics of what it means for people to communicate to each other, but also the process of body transfer and these different effects like the Proteus effect of, you know, how does your self-perception change when you're embodying these different characters? If you have different races or if you're dressed differently, how does it actually change how you are interacting within these different scenes? And so it started with a lot of these fundamental communications questions. And when it comes to different conferences that Jeremy goes to, he goes to the major communication conferences. A lot of his research that he's doing is kind of pushing the boundaries of what's known in terms of, you know, either human communication or these interconnected systems within our body, like empathy, for example, is something that has a lot of different trajectories and vectors for how to look at it. And virtual reality is actually a technology where you can kind of bring them all together and start to look at these issues in a new way. I think at some point, though, it was clear that Jeremy became much more interested in some of these larger issues like climate change and the environment. And how could you really use virtual reality to make a big difference in the world? And he sort of moved from research focusing on some of the technical bits about what we know about the process of human communication into how can we both explore these very complex issues, but to give people an embodied experience so that they can actually change in their consumer behavior. So this process of embodied cognition, I think, is a key component of a lot of what they've been trying to research. And they found that in order for you to really be engaged within an immersive experience, there's a correlation between how much you're moving around and how immersed you're feeling. And so if you look at some of the experiences that they've created, then you kind of have to do some of these actions in order to be interacting with the environment. They're not actually doing anything. I mean, they could still tell the story without them having you participate in that way. But having you participate actually gives you much more engaged into what's actually happening. And I found that as well with the ocean acidification experience, where I'm kind of planting flags around all where I see the snails and then the next scene when there's no snails you have that embodied experience of like well I'm not going to actually plant any flags here and it's just a message that can be embedded within your body a little bit more because you actually took those actions and then the lack of doing those actions just makes that moment of communication that much more powerful. It was also really fascinating to hear him kind of peel back some of the curtain on Stryver. I know that with companies like Stryver that a lot of this type of information of what data they have, what they're able to find from the data that they do have, is something that's usually pretty locked down and pretty proprietary. Because Jeremy is looking at it from an academic perspective, there's not a lot of other data sets that he's had that much access to. And so to me, it's really fascinating to look at both the football players as well as what they're doing with Walmart, where they're starting to capture all this biometric data in terms of where people are looking at, what they're paying attention to, and to start to do these statistical models to be able to model the process of learning. And I think that is a huge thing for virtual reality in general, but specifically for Stryver, who's got access to these huge datasets and also applying different machine learning techniques to be able to discern these huge swaths of numbers into deeper meaning and to have some sort of way of actually connecting the dots between these movements and the process of learning. And I think, overall, the big takeaway here is that starting within these high-performance athletes within the National Football League, but those same situations of being able to be in an arousing situation, look around a space, and to be able to make a quick decision. And so within virtual reality, you're able to tie all those things together. You're being immersed within an environment. You're able to have some situational awareness. And then you have to make a decision and take action on that. And those are the same principles within a quarterback within the NFL as to what people have to do if they're an employee at Walmart on Black Friday. And they were able to prove to Walmart that this was effective to be able to go from a small pilot study of 20 to be spread out to pretty much all of their different academic training centers across the country. So to me, this is huge news, as well as the fact that they've got some other big corporate clients that he wasn't at liberty to disclose yet. But it just is showing that virtual reality is kind of making this inroads into the enterprise training market and that training is kind of like the big killer app for virtual reality and that it starts with the NFL but is moving into more the enterprise market. So to me this is one of the affordances of virtual reality that is going to make it persistent. It's why the military invested so much money in virtual reality back in the 60s and all the way up until today. So it's been over 50 years of investment because of the power of the training applications within virtual reality. And I think with Stryver, it's a little bit of like being at the right place at the right time and to be able to push forward in the technology and to have it really kind of take off the way it is. I was really also interested in hearing some of the more philosophy of science implications of VR. And I think when talking to Jeremy, he's somebody who's an academic. And so when you read this book, The Experience on Demand, you get very much of a here are the facts, here's the statistics. It's a lot less of a first person phenomenological experience of Jeremy, of him going through his academic career. And it's much more focused on what he's been able to kind of prove within the domain of the academic knowledge around virtual reality. What are the sort of, you know, ethics around whether or not you're going to be able to look at some of these unconscious processing in the process of learning? And are you able to, you know, have access to all these data sets? Right now, there's a lot of implications around privacy. And, you know, what he said is that his students were kind of hesitant to be able to be like, either you're going to take my grades that I got in class, or you're going to take this whole swath of data that is basically the map to my unconscious. And if you can unlock that with machine learning algorithms to put different models around that, I think people are just a little bit more hesitant to be like, yeah, go ahead and have full access to my unconscious data. And you'll be able to make your decision based upon that. And I think that is rightfully kind of like a little bit more scary for people to take that leap. But that was some of the issues in the philosophy of science class that he was talking about. To me, one of the things that Jeremy said was that we know that the mind is in the brain, and I would actually disagree with that, that that's sort of a point of view that consciousness is emergent from neurology, and I think that that's actually an open question, that we don't know if consciousness is maybe fundamental or universal, something that's more... Some of these other philosophies, like panpsychism, or transcendental idealism, or Eastern philosophies, or even Platonic idealism, I think that there's a lot of open questions in terms of what the nature of consciousness is and how phenomenology and subjectivity is going to be bridged into objectivity. That, I think, is something that is still a bit of an open question as to how that's going to play out. The principle of embodied cognition, I think, is interesting because it does start to show that we don't just think with our brains and our mind, but we think with our entire bodies, and that the context and the environment that we're in actually also has a huge impact for how we think. And so these concepts of distributed cognition and embodied cognition start to question that metaphor of the brain as a computer and that our mind is inside of our brain. So Jeremy's academic journey within virtual reality actually covers a lot of these big parts of the ultimate potential of VR. And that Jeremy is much more on that side of saying that part of the ultimate potential is that when you take off the headset, you actually be much more connected into reality. And that he sort of agrees with Jaron Lanier in that he's doubtful that virtual reality is ever going to be able to really fully trick your mind always. What Jaron said is that there's a bit of an arms race in that each time the technology makes these different incremental improvements, our mind is going to be able to adapt and kind of know the difference. But Jeremy's covered everything from like identity and body transfer to changing consumer behaviors to the communications implications, educational implications, how VR can be used for PTSD and pain reduction and different medical applications and Empathy at Scale is a big project and these principles of embodied cognition and training and education, as well as the potential to use VR to be able to really make a big difference in the world, especially around a lot of these environmental issues that he really gets excited about in terms of the potential for using VR to be able to educate people about these larger complex systems of the human impact on the environment and climate change. So that's all that I have for today. And I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends and consider becoming a donor. This is a listener supported podcast, and I do rely upon your gracious donations in order to continue to bring you this coverage. So you can donate today at patreon.com slash Voices of VR. Thanks for listening.
