Mel Slater is an ICREA Research Professor at the University of Barcelona, Spain, where he leads the Event Lab. Mel and his fellow Event Lab researchers have been doing some pioneering work in defining the key components of presence as well as the exploring the sense of self within virtual environments.
According to Mel, the illusion of ‘presence’ within VR can be broken up into two major components. First, the Place Illusion is when your perceptual systems are fooled into believing that you’re in another place through more technologically driven factors of low-latency head tracking, a fast framerate, and having accurate 1:1 tracking of your body. The other component is the Plausibility Illusion, which is when you perceive the virtual world as being coherent in it’s construction and it accurately meets your expectations for how it behaves and reacts to your actions. Here’s more details from the abstract of Mel’s paper titled “Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments”
In this paper we address the question as to why participants tend to respond realistically to situations and events portrayed within an Immersive Virtual Reality (IVR) system. The idea is put forward, based on experience of a large number of experimental studies, that there are two orthogonal components that contribute to this realistic response. The first is ‘being there’, often called ‘presence’, the qualia of having a sensation of being in a real place. We call this Place Illusion (PI). Second, Plausibility Illusion (Psi) refers to the illusion that the scenario being depicted is actually occurring. In the case of both PI and Psi the participant knows for sure that that they are not ‘there’ and that the events are not occurring. PI is constrained by the sensorimotor contingencies afforded by the virtual reality system. Psi is determined by the extent to which the system can produce events that directly relate to the participant, and the overall credibility of the scenario being depicted in comparison with expectations. We argue that when both PI and Psi occur, participants will respond realistically to the virtual reality.
Mel describes the Place Illusion as being governed by the perceptual system, and that the Plausibility Illusion is more of a cognitive function. When it comes to breaks in presence, the Place Illusion is more resilient to temporary glitches or technological disruptions. As long as the system can recover technologically to issues such as latency, framerate, or graphical fidelity, then subjects can regain their sense of presence of being within in another world.
The Plausibility Illusion however is a lot more sensitive because it’s a cognitive function. Once there is something that is not coherent within the rules of the environment that violates the expectations of the subject, then it’s a lot harder for the experience to recover from this type of break in presence.
The Event Lab has also done a lot of work in investigating the Virtual Body Ownership Illusion and what’s required to create it, as well as it’s impact on your attitudes, beliefs, and behaviors. Here’s an overview view that explores some of these “Positive Illusions of Self in Immersive Virtual Reality:”
Another really interesting experiment from the Event Lab was looking at how to create the illusion of Time Travel. Subjects were put into a virtual museum with full body tracking so as to create a strong illusion of body ownership. They were then presented with a moral dilemma scenario that was based upon decisions that they had already made. The subjects were then brought back a week later to watch their previous actions, and then have an opportunity to intervene on the actions of their previous self.
Subjects from the experiment reported that it felt like they were time traveling because they were able to objectively witness their previously embodied avatar’s actions. More information about this study can be found here:
A method for generating an illusion of backwards time travel using immersive virtual reality—an exploratory study
This time travel illusion has a lot of interesting implications for the future of social interactions and experiences within a VR environment. Imagine hanging out with friends or family, and then being able to revisit those experiences after it had long faded from your memory. Perhaps you’d gain new insight about yourself, or be able to go beyond nostalgia and be able to relive experiences with people who are no longer alive.
Finally, Mel talks about how VR has the potential to be a very powerful tool to be able to change your sense of self. VR has the capability to put you into another body and give you another point of view and perspective, which can then give you a new perspective on your own life. No other medium has the capability to do that, and the ultimate potential of being able to do this is largely unexplored and unknown at this point.
If you’re interested in these types of questions about how VR can the sense of ourselves, then be sure to follow the publications listed on The Event Lab website, keep an eye on The Event Lab YouTube channel for some of their latest results and research, and take a look at the new Frontiers of Virtual Environments publication that Mel is editing.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:11.975] Mel Slater: My name is Mel Slater. I'm an e-career professor at University of Barcelona and also at UCL. And we've been doing virtual reality to study body ownership illusions for the last 10 years or so, where we put people in a virtual environment with real-time head tracking and real-time motion capture. They look down at their real body, but they see a virtual body replacing the real body. So basically we've been looking at what are the factors that induce into people the feeling that the virtual body they see is their body, and also if they do have that illusion, what are the consequences for their own attitudes and behaviors.
[00:00:50.640] Kent Bye: Yeah, and it seems like the sense of self is actually pretty slippery when you go into virtual reality in the terms of being able to change attitudes or behaviors and Maybe you could talk a little bit about when you're in VR, what you're able to measure in terms of changing attitudes, behaviors, and spontaneous actions.
[00:01:07.979] Mel Slater: Yeah, so there's various different measures that can be used to this, which is one is simply behavioral. How do people behave differently under different circumstances in the virtual reality? So if you have one body compared with another type of body, does it affect your behavior? afterwards you can ask people various standard questionnaires about their illusions while they're in the virtual reality you can measure things like their physiological responses to threats to the body and so on so there's quite a battery of actually even we've used and other people have used EEG which is measuring people's brain signals so for example we find that When your virtual body is threatened you have cortical brain response that is similar to what would happen as if your real body was threatened.
[00:01:56.458] Kent Bye: Yeah, and it seems like you're able to change racial biases as well as in terms of, you know, putting people into avatars of a different race. You know, how are you able to measure that in terms of what happens when you put somebody into an avatar of a different race and how you're able to measure how that changes their biases?
[00:02:15.075] Mel Slater: Yeah, so what we've done at the moment is use a standard tool that comes from Harvard called an implicit association test. where the kind of rapid associations you make between different categories indicate a degree of bias. So, so far we've used those kinds of tests. I wouldn't go so far as to say we've shown that it affects implicit bias. We have some evidence to show that, but we really need more studies and longer term studies to make this a more firm result or in fact show that it's not true.
[00:02:47.829] Kent Bye: In terms of presence, what are the components of presence that you'd say are pretty key in terms of creating this sense of being in another place?
[00:02:55.615] Mel Slater: I think there's two key components, which I call place illusion and plausibility. So place illusion comes from using your body to perceive in the way that you do normally. So by moving your head, by bending down, by reaching, by turning around, this kind of thing. And when you use your body, to perceive in a natural way and the virtual environment responds in a more or less natural way. The simplest illusion for the brain to adopt is that this is where you are. The place depicted by the virtual reality is the place you're in. So I call that place the illusion, the illusion of being in a place. It's a feeling, you can't really describe it to anyone else. And it is an illusion in the sense of you know you're not in that place. but nevertheless you have the illusion that you do. It's not a belief. So if you really, really believed you're in that place, there's no distinction from being in a real place. So that never happens in virtual reality. It's only an illusion. The second one I call plausibility, which is the extent to which, not that you're in the place, but that you have the illusion that what's going on is real. So it's a dynamic illusion. You see some events, and do you take those events as real? So when you have both of those things together, the illusion of being in the place and this plausibility illusion, then that produces a response in you as if you were actually in the real place doing those things, interacting with other virtual people and so on. So I think it has those two main dimensions.
[00:04:28.165] Kent Bye: And in terms of the plausibility illusion, what are the key components of doing that? In terms of, do you need to do fully tracked or, you know, what are the things that you found that you need to add to a VR experience in order to create this plausibility?
[00:04:42.038] Mel Slater: So, plausibility, I think at least has three components and probably more. One is that the environment responds to your actions. So, for example, if you walk towards a virtual character and they back away from you, this is an example of actions in the virtual world correlated with your actions. Second, that there are events in the world that are addressed personally to you, like a virtual character waves at you or smiles at you. And the third one, which is the most difficult of all, is that if the virtual environment is supposed to depict something of which the subject in the real world knows about, then it has to follow certain rules that would be expected to happen in the real world. One example of that is that we did one scenario where we looked at people's reactions to a fight between two other people in a bar, a fight about football, soccer, and subjects afterwards told us it wasn't too real because a fight like that would never happen in a bar that looked like this. So this is an example where you have to be credible in terms of what you're depicting to the participants, that it has to follow certain basic expectations that they would have in that kind of situation. So you have to have a lot of domain knowledge to do that right.
[00:06:01.181] Kent Bye: I see. Now it seems like the sense of presence is something that can be broken and is it like a house of cards that once it's broken can you bring it back or what are the components of breaking presence and then bringing it back?
[00:06:13.108] Mel Slater: So presence is a perceptual illusion. So for example, it's based on things like moving your head and seeing updates to the images and using your body to move around and so on, that kind of presence, this place illusion component, if that breaks it comes back because you just have to go back to, so for example, you're in a virtual environment, you look very close at an object and you see the pixels, your presence breaks, the illusion of being in the place breaks. Then when you move away from it and you just carry on as you did before, it comes back again. So this illusion of place, of being in the place, it comes and goes. Whereas the plausibility illusion, our experience is once you lose it, it's lost. So, for example, you're interacting with virtual characters and you start to realize they're doing repetitive actions again and again. Once you realize that, you lose interest in those characters. Plausibility is more of an illusion that once it's more cognitive once it breaks once you lose interest You've lost it. It doesn't come back.
[00:07:12.876] Kent Bye: But the place illusion is more Stable in the sense that you can break in it reforms quickly Yeah, and it was interesting that you'd mentioned that it's hard to have a control condition when you look at your body and having an identification of your body and it seems like when you're in VR and you have a one-to-one tracking then you have this sense of body illusion and maybe just describing like how far you can kind of change the identification with these avatars.
[00:07:39.482] Mel Slater: So basically when you put on a head-mounted display and you have full head tracking and body tracking when you look down to your body you'll see another body instead. So statistically, in our whole lives, whenever we've done that action, we've seen our body. And what's interesting is the brain seems to say, OK, I'm looking down, I'm looking towards my body, I see this, so this must be my body. It's really fast. in adapting to these kind of changes. Then if on top of that you move and the virtual body moves with you, again this is further evidence to the brain, well this must be my body. So these illusions are pretty quick to start and it doesn't need a lot of time. It happens in a few seconds or it doesn't happen at all with some people.
[00:08:24.381] Kent Bye: Yeah, and it seems like you're able to kind of distort and extend the hands of a body and still people identify that. You know, what's the limits of what you've been able to see in terms of how you can distort and change the body and people still identify with it?
[00:08:37.150] Mel Slater: Well, we don't know the limits, but for example, we have one experiment that was carried out by Constantino Kiltani where we made one arm up to four times longer than the other arm in virtual reality. So up to three times longer the arm was still incorporated into the body representation. Four times longer it was lost. Actually we don't know if it was lost because the arm being so long is kind of the vision of it degrades. But certainly you can get away with quite big asymmetries and you still maintain the body ownership. You can have bodies quite different from your own. So thin men can have a fat body. Men can have a female body. Females can have a male body. The brain just seems to accept all of these because this body ownership perceptual illusion is like a fundamental perceptual illusion that arises because of multi-sensory integration across different domains, vision and haptic or vision and motor and so on. Once you have that kind of multi-sensory evidence that this is your body, the brain seems to accept that very easily.
[00:09:45.072] Kent Bye: In some of the funding, I've seen that, you know, the word morality or that there's some sort of, you know, exploring issues of morality within VR. And I'm curious what that means to you and how that plays out.
[00:09:54.759] Mel Slater: So this was a specific project where we were using virtual reality to put people in moral dilemmas and then just seeing how they respond. Yeah.
[00:10:03.190] Kent Bye: Okay, and it seems like this this time travel experiment of creating a sense of illusion of time travel was creating a moral dilemma and you know people going back and seeing this and so I'm curious of like what was the feeling of time travel illusion what that actually felt like for people?
[00:10:19.936] Mel Slater: It's very difficult to describe an illusion. Actually, I can't describe it. It's just that People report that they've gone back in time, and the critical thing seems to be seeing their previous self doing the actions that it did before. So, when this happens, it's like the brain says, okay, I'm back where I was before, this is my previous self, who did all the wrong things, now I'm going to try and do the right things. But I can't explain to someone what an illusion feels like, because it's something very subjective.
[00:10:49.731] Kent Bye: I see. And have you experienced it yourself? Yes.
[00:10:52.292] Mel Slater: What was it like? Well, it was like time travel. I can't explain it. It's like a feeling, very strange to see your previous actions and words being acted out by a body that you see and you remember having done that. And now you see it externalized on someone else. But I can only say it's like a strange feeling.
[00:11:13.939] Kent Bye: Yeah. And finally, what do you see as the ultimate potential for virtual reality and what it might be able to enable?
[00:11:21.622] Mel Slater: Well, in terms of changing the self, virtual reality can be very powerful because basically what it can do is to take you outside your own body, which is your own preconceptions about yourself and about how you are. By being in another body, you can have a different point of view that comes with that body from that different perspective and therefore reflect on your own life and your own being. So it can give you experiences that you can't have in any other way. So what the limits of that, I don't know.
[00:11:52.429] Kent Bye: Yeah. Great. Well, thank you so much. You're welcome. Thanks. And thank you for listening. If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash voices of VR.
