VR Presence researcher Mel Slater is fascinated by what makes the medium of virtual reality unique and different from other communications mediums. He says that VR activates our sensorimotor contingencies in a way that fools our brain that we’re transported into another world, and that what is happening is real. He breaks this into two primary illusions where the Place Illusion answers the question “Am I there?” and the Plausibility Illusion answers the question “Is this happening?” These two illusions happen inside of your mind and are very difficult to study, but Slater has develop an experimental research protocol that draws inspiration from color theory research the combination of the objective spectral distribution as well as an individual’s subjective perception of color.
After seeing well over a thousand VR experiences, I started to cultivate my own ideas about an Elemental Theory of Presence that describes different qualities of Embodied Presence, Social & Mental Presence, Active Presence, and Emotional Presence. Slater says that these different qualities of experience are more related to the content of the experience, and that they’re not unique to virtual reality. You can be just as emotionally engaged with a movie or a book as you are with a VR experience, and so looking at how the content contributes to his conceptualization of presence isn’t an interesting research question trying to figure out what’s unique about VR.
LISTEN TO THE VOICES OF VR PODCAST
My elemental theory of presence is more of an elemental framework for experiential design that’s isn’t unique to VR, but it has been useful in helping to understand the component parts of a VR experience. Slater is primarily interested in researching the objective and measurable dimensions of a VR experience that contribute to the place illusion and plausibility illusion that he sees are the primary factors of the subjective feeling of presence. I personally don’t believe that you disregard the role content in how it helps cultivate a feeling of presence, but I acknowledge that it’s a difficult thing to study in controlled academic research environment. There is not a universal formula for what combination of content and experience ingredients that will help you achieve a sense of presence whether you are in VR or not. There are limits to predicting the degree to which a piece of content will resonate with someone, and the successful approaches are usually market-based solutions that big data collections of behavior to drive the content recommendation algorithms at Amazon, Netflix, Facebook, and Google.
My interview with Slater explores the threshold of the boundaries of his theory of presence as I try to understand it through the lens of my own elemental framework of experiential design. He cites NASA’s Stephen Ellis who once said that any good theory of presence will provide a series of tradeoffs that allows you to make choices amongst features that are within the same “equivalence class.” Slater’s approach to presence focuses on the objective features of the VR system while my elemental theory of presence focuses on the qualitative aspects of the specific content. Slater says that it’s a completely valid approach, but that it’s just completely different than what he’s interested in looking at. This conversation clarified for me the differences between objectively controllable VR hardware & software variables and the specific content of a VR experience. I think that both contribute different things to the subjective feelings of presence, and experiential designers will have to take into account both the objective features of the VR hardware and software as well as the specifics of the content in order to create the qualities of presence that they’re striving for.
Want to discuss more about this podcast? I’m having an experimental discussion about it with my Patreon members here.
Become a member of the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye and welcome to The Voices of VR Podcast. So Mel Slater is one of the leading researchers in the virtual reality community when it comes to presence. So virtual reality is a unique medium in the way that's able to transport you into a completely different world, and you believe that those worlds are happening. And that is what is really fascinated Mel in terms of trying to research the different components of the technology that are leading to this feeling of presence. He's broken it up into two component parts, which is the place illusion, which is answering the question of, am I there? And then the plausibility illusion, which is looking at, is this happening? So Mel has a very specific definition of presence and how he thinks about it. And I started to have different realizations of the importance and contribution of the content to the different levels of active presence or social and mental presence or emotional presence or embodied presence. And these are all different dimensions of presence that are coming from the content that's being developed. And this is something that is completely different than what Mel is interested in looking at. He's interested in looking at the objective qualities of the virtual reality hardware, the field of view, the spatial audio, and he's looking at all those things and trying to determine what are those factors that are able to be objectively controlled and measured and studied when it comes to cultivating presence. So this is something that I learned in the process of this interview. I think that we are kind of talking about these different layers and levels of both the objective qualities of presence, but also the content. And so I had an opportunity to kind of like stress test my own theories and conceptualizations around presence with Mel. And I think that there was just a lot of disagreements and discussions around how he thinks about it and also what he's really interested in researching. And that while some of the questions I'm asking are valid, they're just more of an experiential design framework for content developers rather than looking at the fundamental capabilities of the VR technology that is enabling these different qualia of presence. So we'll be covering all that and more and doing this deep dive into VR presence on today's episode of the Voices of VR podcast. And so this interview with Mel happened at the IEEE VR academic conference that was happening in Los Angeles, California from March 18th to 22nd, 2017. So, with that, let's go ahead and dive right in.
[00:02:29.201] Mel Slater: Hi, my name is Mel Slater. I'm at ICRIA, University of Barcelona, and also part-time at University College London. I work in the area of virtual reality and many different aspects of it, and more recently we've been studying the relationship between the representation of your body in virtual reality and how this affects your attitudes, behaviors, cognition, and so on.
[00:02:56.246] Kent Bye: And so you're probably best well known for your presence theory of the the place illusion and the plausibility illusion and you were talking yesterday and one thing that I found really interesting is that you were looking at color theory and how people describe color as a Approach of describing something that is really dependent on someone's internal qualitative subjective experience so maybe you could describe a little bit of what the approach that color theorists have done to describing color and how you are using that as a metaphor to think about how people think about plausibility and presence.
[00:03:29.205] Mel Slater: Yeah, so in color theory if we think about what color is Color is a spectral distribution that describes how light is emitted or reflected from a surface. So, when people perceive the color, of course, they don't just soak in that spectral distribution and say this is red or whatever it is, but there's a complex relationship between the reflected and emitted light and the way the visual system in humans works. And it's an active system, it's not passive. Humans bring, well I guess humans and all animals, we bring to our perception our own expectations, our own particular physiology and so on and so on. So it's a very active process, it's not passive. And also it follows from this that the perception of a colour is very subjective and qualitative. So if you say to someone, describe what it means by red, it's impossible to describe. It's a feeling, what philosophers call a qualia. So there's a physical basis of colour, and then there's the subjective sensation of colour. But nevertheless, colour is something which is pretty well quantified, in the sense that you can describe a colour as a particular combination. Any visible colour can be described as a particular combination of, say, red, green and blue primaries. How is this done? And this is pretty objective. It works across people in general. So the way it works in color theory is that you have a target color which is displayed on a screen and then the perceiver has three guns, red, green and blue guns say, and they have to manipulate those three guns that are shining together at the same spot until they reproduce in their mind the same color as the target. So if you do that over many people, you get average values of red, average values of green, average values of blue, and they say, okay, this color is represented by red, green, and blue. Now, if you move to virtual reality, there's an analogous situation. There is the physical side, which is, let's say, the hardware and software and the corresponding capabilities that the virtual reality system can deliver in an objective sense. And then there's people's perceptions and feelings that they get when they use that system. So the first is like the spectral energy distribution and the second is like perception of a color. So there's the analogy. So what we try to do is follow through on that analogy as a measurement method. So suppose we have some particular virtual reality system which is made of a number of manipulable components like field of view can be changed, resolution in principle can be changed, the kind of lighting that there is, whether it's flat shading or radiosity or ray tracing, all of these things can be changed. So imagine you put someone in a virtual reality where everything is at its possible best. And then you say, pay attention to your feeling of X, which X might be their feeling of being there, or their feeling that this seems to be really happening, not just virtual. So pay attention to your subjective feeling, whatever the research is interested in, like presence. Then start from some low-level configuration of all those factors, like low field of view, bad lighting, et cetera, et cetera, and manipulate those factors in turn, one by one, until you reproduce the same feeling you had when everything was at its full level. So again, if you do that across multiple people, you can say, well, the feeling of the best environment, the best setup, was reproduced when there was X percent of the field of view, and Y percent of the resolution, and Z percent of the lighting, and so on. So this is the way that we do it. So it's kind of like mapping on the ideas from colorimetric theory to measuring aspects of subjective experience in virtual reality.
[00:07:33.283] Kent Bye: Yeah, because one of the things that you were saying is that whenever you give someone a questionnaire, that may be numbers that a three for them may be a different three for someone else. So you're able to maybe come up with a more way to kind of standardize the system of trying to figure out these trade-offs between what is really ultimately the most important. You had this quote from Stephen Ellis. Maybe you could expand on what he said and sort of what you're able to do functionally in terms of these cost-benefit.
[00:08:00.156] Mel Slater: Again, it matches with color theory. In color theory, there's metamers where the same color is perceived even though the spectral distributions underlying those colors are different. So in other words you look at several objects they've all got different spectral distributions but an observer will perceive them as the same color. So it's the same idea here that you could have trade-offs in virtual reality whereby, let's say, a bit more field of view would compensate for a bit less resolution, or a bit more resolution would compensate for a lesser degree of realistic lighting. So a long time ago in the 90s, Stephen Ellis from NASA, he wrote a paper saying that any good measure of presence should be able to have these trade-offs which he called equivalence classes where you could get configurations of systems which are different but nevertheless give rise to the same feeling of presence.
[00:08:57.780] Kent Bye: Right, so the basic idea is that there may be different dimensions as you're doing experiential design. You have these trade-offs of limited time and money. And if there's a number of different factors that are pretty much essentially the same in terms of cultivating presence and experience, then you're going to have a choice to do these trade-offs and really focus on the thing that is really cultivating the deepest sense of presence. I guess one assumption that you're making in the process of this experience is from your own direct experience trying to figure out what you think is kind of all the different dimensions of what could be the best level of presence that you're able to give and then try to then quantify it or break it down into its component parts. So it's this fundamental challenge I see is like what you're doing with this research is trying to take something like the plausibility illusion something that's fundamentally qualitative, but yet break it down into component parts so that you can start to determine the fundamental components of it as you're doing experiential design.
[00:09:52.468] Mel Slater: Yes, that's correct. So I can't know what your illusion of plausibility is, and I can't know what somebody else's, but I can know that you decided that this particular configuration was a match to your feeling of plausibility, and that was the same as somebody else's. So it's like if in color theory, they could say, well, I'm going to give you a questionnaire. How red is this on a scale of one to seven? You might say five and someone else might say three and someone else might say two. You can't really compare those numbers because they're... They relate to each individual's experience differently. But what you can do is to say, I say it's red, and he says it's red, and she says it's red, and so on. So this is the same thing as we're doing. We're looking for matches, the point when a configuration feels the same as some other configuration. And this you can do across many, many people. And the only thing you care about is they say, yes, it matches. You don't care about is it on a scale of one to seven or something like that.
[00:10:55.761] Kent Bye: It seems like a methodology that I think is going to help really refine these different fundamental components of these dimensions of presence. Richard Skarbez had a presentation where he was giving his ideas about how to start to break down plausibility. And one of the things I found really interesting is that he said the place illusion is pretty well settled in terms of the technology stack that's required in order to really create a good virtual reality experience. And so we have a lot of the objective parts of VR that has, you know, kind of described by that level of immersion with the place illusion. But yet the plausibility illusion of making sure that that scene is coherent is sort of like this next frontier of research that you're actively looking in. Richard Scarbez starts to come up with three fundamental ways of breaking that down with the correlation of whether or not your actions that you're expressing your agency are interacting in a way that feels believable. The scene coherence is something that makes sense with the context of the situation we're in and that everything else kind of makes sense in that. And then also this self-reference which is kind of like this sense of embodiment within the experience so that you have the virtual body ownership illusion that's invoked. And I think one that wasn't talked about that we talked about is maybe a little bit more of the emotional engagement of the space. Is it engaging your emotions in any way? So that's sort of Richard's first take of trying to break it down. For you, what is your approach of how you're starting to think about starting to break down plausibility into these other parts?
[00:12:26.279] Mel Slater: Can I just clarify one thing? We're not breaking down plausibility or place illusion. What we're breaking down is on the more physical side, we're saying, Let's go back to the color analogy. Different wavelength distributions can give rise to the same perception of color. Or the same wavelength distribution for different people can give rise to different perceptions of color. So when you analyze color theory, what you're really doing is you're not classifying, well, in a way you're classifying the perceptions, but you're not breaking down that perception of red. You're understanding what are the components in the physical side that contribute to that feeling of a certain color. Likewise here, we're not trying to break down the meaning of plausibility. We're not trying to break down the meaning of being there or place illusion. What we're trying to understand is what are the components in the VR system that might lead people to say, I have a match to the best plausibility I've ever experienced in this system configuration. So for me, the plausibility or the place illusion are unitary concepts that themselves, or unitary illusions or subjective feelings that themselves can't be broken down. What we're doing is we're finding what are the contributions to that feeling? What are the technical, if you like, the objective things that are actually tangible that contribute to different possibilities of having that feeling.
[00:13:55.609] Kent Bye: I see, that's interesting. So after hearing about your place illusion, plausibility illusion, I started to kind of use that as my framework of presence. And then as I was going through, at this point, over a thousand different experiences over the last three years at all the kind of latest consumer VR conferences that I've gone to, started to have different dimensions of presence that I started to kind of break down in my own subjective experience and I started to try to Categorize them to go beyond the just the place illusion and plausibility and and just to start that up I'm curious how you kind of fit like social presence within your model of the place illusion possibility illusion.
[00:14:32.899] Mel Slater: Yeah, so in a way I don't Let's see. What is the basic thing that? So if I think about how do I think about this illusion of being there, for me it's based on sensory motor contingencies which means that If I try to perceive the virtual world and I perceive it using my body much as I do the real world, so in other words, I turn my head, I bend down, it's in 3D stereo. As I move close to an object, the object changes in my retina, it changes in size exactly the same as it would in reality and so on. When you have all those things going on, the brain always wants to have a hypothesis about what's going on. The simplest hypothesis for it to make under those circumstances, when I perceive in VR using the same methods that I perceive in reality with my body, then the brain's simplest hypothesis is, this is where I am. So if I happen to be looking at a person and I move my head around and they change in my perception the same way they would in reality if I was looking at a person in reality, then my brain says, okay, that person is there. So that's one side. On the other side, if I look at that person and I go up to them and I see they're doing repetitive motions over and over again and they're not aware of my presence and I can say anything to them or do anything and nothing ever changes, It doesn't change my presence. That object is still there. But what that affects is my plausibility. I'm no longer interested in it. This is not really happening. That's not a real person. So this is the kind of classification that I can use. Now, when it comes to like issues about emotional engagement, Instead of presence or plausibility, you could take that as your output variable and use exactly the same technique. In other words, you could have a number of factors that you think may influence degree of emotional engagement. Like, again, it could be things like the particular behaviors of the avatar, all the objective things that you can program a virtual reality system to do. You might come up with, say, ten things. And you can let users or participants vary those 10 things until they say, ah, this particular configuration that really engages me emotionally. So to me, that would be a separate issue, a separate output variable from plausibility and place illusion. The emotional engagement would be another such output variable. As I said in the talk yesterday, you could use this same methodology for many different things. I happen to be interested in place illusion and plausibility, but if you're interested in the emotional effect it has on you, then you could use the same technique where that's your output variable. If you're interested in how much engagement people have with an environment, again, you could use the same technique where that is your output variable. But by the way, things like emotional engagement intrinsically don't have anything to do with virtual reality because you could use the same technique on a movie you're watching on a screen. How much does it emotionally engage you? You could use exactly the same techniques. So why I'm interested in plausibility and place illusion is that I think, especially place illusion, it's something really unique to virtual reality that you can't get in any other medium.
[00:17:50.368] Kent Bye: Yeah, I agree with you and I think that I've sort of been in the process of formulating my own framework and trying to kind of figure out how my framework merges with yours, if your framework is a subset of my framework or the way I think of it fits in yours. So let me just sort of describe what I'm thinking here. I feel like, if I go back to the four elements, earth, air, wind, fire, that the earth element is the body, and that's the embodied sense of presence that you have, and that could be the virtual body ownership illusion, and it could actually be where you're talking about both a subset of the body as this place illusion and plausibility illusion, such that you feel like you're actually immersed and it makes sense, right, and from that body's perspective. But I also feel like, you know, the water element is the emotion, so it's like the emotional engagement. So is there a story? Is it some sort of, like, something that's really moving your emotions and being engaged in a certain way? And then you have the fire element, which is your agency. It's your interaction within the environment, such that you're able to participate. And you're kind of testing the plausibility, like, by saying, okay, if I move this, is it reacting in a way that I expect? Does it have a good physics engine? Is it reacting in a way that I would expect it to, given the will that I'm exerting into this experience? So then finally, there's the air element, which is sort of like the social mental presence. So anything that's related to, is it coherent in a way that you make sense of the world? But also, are you having any social interactions and communication and talking? So there's these different levels that I've had where I have an experience like the Oculus Toy Box demo, where I feel like I have this deep sense of social presence. because there's somebody else in the scene or I'm able to have this high level of active presence because I'm able to participate in this live physics environment that feels real and I have a sense of embodied presence because I have my hands in the game but it's not a full avatar so it's just my hands and not my full body so it's a limited invoking of a virtual body ownership illusion. And then the emotions come from a little bit of a transfer from like this social interaction that I have. There is just sort of like this fun that we're having in the context with this other human being. So, you know, when I saw the Toy Box demo, that was like, oh, wow, this is the deepest sense of presence I've ever felt. What were the fundamental components? So, you know, as I am talking about this, I don't know if you have any thoughts of kind of reflections on that.
[00:19:56.802] Mel Slater: Yeah, I think that it's just a different framework. They don't really intersect. It's like, you know, let me give you an analogy. We're sitting here in front of a table, and as I move my head around, and I move, and I can look under the table and over the table, I can go and look around the other side of it. This table is here, and I'm here with it. For me, being there is a purely perceptual illusion that you can turn off, for example, by shutting your eyes. What I care about the table, whether it affects me emotionally, is a completely different thing. It's nothing to do with my sense of being there. It's my subjective opinion of how the table affects me emotionally. And that could be the same as if I'm looking at a real table, a table in virtual reality. a table in a photograph, a table in a movie, a table on television or whatever. So it's quite a different thing. I'm specifically and only interested for myself, I'm not saying what other people should do, I'm specifically and only interested in this perceptual illusion that I am in this place, that's it. And I'm only interested in the perceptual illusion that the events taking place in this place are really happening. So whether the events have an emotional impact on me is a completely separate question, because that depends on the context and what the events are. So what you're describing is many different components of an experience, ranging from it's emotional, it's interactive, it's social and so on, and that's fine, but that's to me separate from the two basic things. Am I there and is this happening? That's it. You're talking about another level or another layer.
[00:21:39.885] Kent Bye: Yeah, and just talking to Betty Moeller, one of the things that she said is that one of her students was doing this research into emotional presence to the point where she said that actually if you're scared, that'll change your perception of how you perceive the world. So my sense is that, you know, given the elemental theory, it's sort of like you can't separate earth, air, wind, fire. They're all happening all at the same time, all at once. And so I do feel like there's an emotional component to our perception, such that if you're angry, you may perceive the world in a way that's different.
[00:22:08.605] Mel Slater: What I hear you we're not talking about the same thing I'm talking about the pure illusion that I am in a place and the pure illusion that the events here are happening that as an analogy you could have Eight lines on a piece of paper and you know, they're eight lines, but actually what your brain tells you is a cube That's the level I'm talking about. So you could have you know, one of these illusions of this color seems to be different from that one, but when you look at it in a different context, because each color is surrounded by other colors, you actually realize the two colors are the same. How that affects you emotionally on someone is a completely different question. So I'm not talking about the same things that you are. I'm talking about the pure illusions. I'm here, and this is happening. That's it. I mean, everything else is I wouldn't call it presence. I really don't like it to be called presence because it's for 25 years, it's confused the issue. So some people are saying, yeah, yeah, you know, this doesn't engage me so much. But yes, it's true. In real life, I can be sitting in an orchestra and I'm totally uninterested in the music and it doesn't engage me. But am I there? Yeah, I'm there. It's a separate layer. It's a separate issue. So this is the kind of distinction I'm trying to make.
[00:23:24.814] Kent Bye: Yeah, and I talked to Dustin Chertoff, who back in 2010 wrote a paper having kind of a similar approach of having these different dimensions. And the thing he said is that he was looking to experiential design, which is kind of from a marketing side. So I guess I'm kind of coming up with an experiential design framework and you have a presence framework. Is that what you would say?
[00:23:44.529] Mel Slater: No, I'm just saying they're answering different questions. So I'm only answering the perceptual and cognitive questions. I'm here and this is happening. Now, given that I'm here and things are happening, how does it affect me on, say, the emotional level, on my sense of aesthetics of the thing? These are all different questions to the ones I'm talking about. And I think they're really important to understand. Well, like a movie maker or a theatre director, they know what kind of things the content of the environment, what has to go into the environment to make something appealing in an emotional sense or to make people happy or to make them scared. But these are all different questions. Really I'm looking at very fundamentally the relationship, the mapping from the objectively describable technological features that make up a particular system, what I call the configuration, and the effect it has on, I feel here, and I feel that this is happening. Everything else beyond that is like a different level, a different layer. So I think it's really important and valuable work, but it's not the questions I'm particularly answering in this particular research. In other stuff, yeah, I'm interested in how much the emotional impact is. And the thing about the body, the body to me is this is like a conjunction of where plausibility and place illusion meet. Because the place illusion aspect is from sensory motor contingencies. If I bend my head down, I see my body. So if this happens in virtual reality, it's like the sensory motor contingency component where I'm using my body to perceive something and I perceive it. And the plausibility stuff comes down. When I look down, I expect to see my body and I see it. And if I move, I expect to see my body move, and it moves. So this is on the plausibility side. So the body ownership and agency to me are like, if you think of plausibility and place illusion as orthogonal axes, it's the origin. It's where they both meet.
[00:25:46.051] Kent Bye: Do you have interactivity and agency within your framework at all?
[00:25:49.695] Mel Slater: Like how you think about that? Once you dig into the body, then body ownership and agency are now part of the study of that. And we try to understand what are the factors that influence your sense of agency and your sense of body ownership. That's like digging into a particular, as I said, if you think of the point where plausibility and place illusion meet, that's the body, and then you dive into that body and a whole number of other questions open up.
[00:26:21.284] Kent Bye: It does seem like the thing that is different to all virtual reality is the body, and I've also been hearing different people talk about embodied cognition. Is this embodied cognition something that you've looked at, or are you mostly just looking at the sense of self?
[00:26:34.507] Mel Slater: Embodied cognition is a particular paradigm that relates your body to how you obtain knowledge and so on. I mean it's a paradigm that's existed for quite a long time separate from this. I think our concern is more with, you know, I look at a virtual hand and I have the feeling it's my hand even though I know it's not. So this is like the body ownership. I move my hand and I see the virtual hand move. I attribute that action to me. So this is agency. So again, I am interested in the very, if you like, the pure aspect of that. Then another question that follows on from that is, so what happens when you have the feeling, the illusion that a particular body is yours? And this is another level of question.
[00:27:21.468] Kent Bye: It seems like with consumer VR right now we only have hand tracking and it feels like the research that you've been doing in your lab with doing full body tracking with invoking the full body ownership I think that there's a little bit of resistance within the consumer VR community for example of like what's the value of tracking the feet and I'm like If you just track the full body, it's going to give you this whole other dimension of sense of presence. How would you describe and make the argument as to why getting the full body in a VR experience, what difference does that make?
[00:27:51.696] Mel Slater: Well, it depends on the application. So if I'm walking and I look down and I don't see any feet, it breaks the illusion. It breaks the illusion of presence. It breaks the illusion of body ownership, agency, and everything. So it's basically, it depends what the application is. It's always strange if you're in virtual reality and you look down and you're invisible. This is, I think, one of the first experiments we ever did back in, like, 91 or 92, was the importance of the body. And in those days, the body may have been ten bolligans, but it was still better to have those ten bolligans than be invisible. So it's really fundamental to your experience of the grand category of presence to have a virtual body. Then if you're moving and the body doesn't move with you, then again it can break the illusion. So for me it's, I mean, let's go back again to what you said earlier. Having and changing the body is fundamental to virtual reality because in a way it's the only thing that, one of the only things that media can do that you can't do in any other way.
[00:28:57.518] Kent Bye: Great. So what are some of the biggest open questions that you're asking right now that's really driving your research forward?
[00:29:03.922] Mel Slater: Well, one of the interesting things is it comes back to what you said. So actually you can do full body illusions almost with consumer hardware. So at least for the upper body, you can use like a Vive with hand trackers and use inverse kinematics to get some level of full upper body ownership. And now Vive have come out with the extra trackers so you can put them on the feet. And it'd be interesting to know what the limits are, how much you can get away with just by having one tracker on each hand and one tracker on each foot using those kinematics. And can you still get a full sense of body ownership and agency from that? Yeah.
[00:29:42.618] Kent Bye: Great. And, uh, and finally, what do you think is kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:29:50.897] Mel Slater: The ultimate potential of virtual reality, that's a very big question. Actually, two of us have written an article where we do go into this in the final chapter. This article is about some frontiers in robotics and AI in the virtual environment section. It's a 60-something page article, but the last chapter we start to address some of these issues. So, one is that virtual reality will just become another place. So, okay, I'm now in my living room watching TV, but I decide I want to do something else, and I go in another room where I put on a head-mounted display or I put on spectacles, and that might bring up augmented aspects in my environment. And I press a button, and the real environment completely disappears, and I'm only in the virtual reality. and there I'm actually in a meeting with a friend who's a thousand miles away. So I just think it would become ubiquitous as part of everyday life and it would have its own norms and behaviors that, you know, just like, well, as I say in the article, when you're in the school corridor, you don't run. There's polite behaviors that you do in different circumstances. So in virtual reality, there'll be, again, a similar type of sets of norms that people will obey to be polite and to be social with other people and also to the environment. So I just think it will become ubiquitous and be another place where we can plan things, where we can experience things that are impossible in reality, where we can walk through the terrain of Mars without ever going there. I mean all these things have been said for 25 years, but now it's becoming a possibility. And the other side of it is, this is much more long term, is I think there'll be a convergence that virtual reality will essentially become a part of applied neuroscience. So eventually it will be figured out how to stimulate the brain so that you really don't distinguish between being in a real place and being in a virtual place where it's unencumbered. Now I have this particular thought and that thought takes me into a virtual reality environment because there's some brain stimulus that gives me images, sounds, touch and so on. So I think in that I'm talking about maybe there, over the course of the next 50 years, as neuroscience understands more about how the brain works and how it can produce images, how different stimulations can produce the sensation of images, which is just as real as real reality, that I think this will be the ultimate destiny of virtual reality.
[00:32:33.528] Kent Bye: Awesome. Well, thank you so much.
[00:32:35.395] Mel Slater: You're welcome. Thank you for talking with me.
[00:32:38.523] Kent Bye: So that was Mel Slater, who is one of the leading researchers of presence within the virtual reality community. So I have a number of different takeaways about this interview is that, first of all, this was a very difficult interview for me to, uh, unpack and to really understand. I mean, you can hear me kind of struggling and trying to compare my own conceptualizations with Mel's. And I think that it kind of comes down to different mental model and mapping around these different dimensions of virtual reality. And I think that I finally got it after listening to it multiple times and really churning over exactly what Mel was trying to say. So here's what I came away with. So it sounds like that there's three major components of a virtual reality experience. There's the objective qualities of VR, all the different hardware and field of view and graphics fidelity, all the things that you can kind of objectively tweak and measure. Then there's the subjective qualities of your feelings of presence that you're able to have. that is kind of based upon your own life experience, your own perceptual input that you're having when you're having a virtual reality experience. And then there's the content, the actual thing that you're looking at and going through within the virtual reality experience. And so when I think about presence, I'm really thinking about the content in terms of what within the content is really engaging these different dimensions of myself. And I've found that there's at least like these four different major types of presence that I've had from there, whether it's expressing agency with active presence, whether I feel like I have a real sense of embodied presence by having a virtual body or haptics or some of those dimensions, whether or not there's other people in the experience and I have this sense of social presence, or if it's like activating my mind in some way and just doing a great job of convincing me that this whole scene is believable with this sense of mental presence, or if it's some degree of emotional presence that's really engaging me on a deeper level. So those are all things that are related to the content that you're going through and those are all things that are very difficult to control experimentally. Then they're also not unique to virtual reality. You can have all these degrees of presence when you're in a movie theater or when you're on a date or when you're having any life experience at all. So I think that my elemental theory of experiential design, let's just call it for now, is really applying to all these different dimensions of life and living and specifically for VR content developers who are trying to figure out these different trade-offs in terms of creating a virtual reality experience. One of the really fascinating things that Mel said was that Stephen Ellis from NASA said that any good theory of presence has to have some sort of trade-off so that you can create these equivalence classes. And I do feel like there is a dimension of the experiential design framework of the elemental theory where you're able to kind of lower the level of agency and locomotion such that if you want to increase the level of social interaction then you have people sit on a couch and they can't just kind of teleport and blip away at any moment. You're going to get a better quality social interaction in that way. So there does start to be those trade-offs but it's more in terms of designing and creating the content of the experience rather than the overall objective characteristics of the VR that's creating the experience in the first place. So my sense is that Mel has just been more interested in studying those components that are completely unique to the virtual reality medium. And those are the two things of the place illusion and the plausibility illusion, which is having these virtual experiences that are tricking you into being into these other worlds. Am I there? And then believing that it's actually happening. So for Mel, there's not really a way to think about emotional presence or social presence or active presence. which to me was a little bit confusing because there are other researchers that talk about things like social presence or emotional engagement. And it's been a little bit confusing over the last couple of years of just trying to figure out how these different ideas and conceptualizations and models and mappings of presence kind of fit together. Now, the previous interview that I did with Richard Skarbez starts to get into some of those different dimensions where he is actually breaking down plausibility into these different component parts. Mel seems to be pretty certain that it's impossible to break down this feeling of the place illusion or the plausibility illusion into more fundamental component parts. And I'm not sure. I think that there are a lot of different components that are combining together and you can start to maybe separate them apart. But how Mel thinks about it is kind of like looking at the subjective quality of red. You think it's red and there's no way to kind of break that down into any other further dimensions. Mel also seemed to be hesitant or how people are using this word of presence and that he has a very specific and I'd say somewhat narrow definition of it. And, you know, I have my own life experience of what it means to be present in VR, what it means to be present in real life. And my conceptualization is that the more that you learn how to achieve these states of presence within virtual reality, you can start to feel that same degree of presence that you have in your day to day life and living. But I think that Mel wants to try to just kind of focus that on that narrow conceptualization of what presence means in VR. And to me, that cat is kind of out of the bag. People have been talking about presence for years because everybody has their own experience of what it means to be present. And people have different activities that make them feel present. So at the end of all of this interview, my takeaway was that my ideas about the elemental theory of presence are indeed just a quite different fork of how Mel talks about presence. And if he wants to just focus on the objective qualities and do his experimental process where he's kind of dialing up and down these different components to kind of figure out The trade-offs between the different dimensions of, say, should I invest more money in doing a full body representation of embodiment or look at the spatial audio? You know, those are the more technical aspects of the context of the experience. And then the content of the experience, that's where all the people who are developers, they're trying to figure out what the content is that is kind of invoking these other dimensions of immersion and presence. So again, I think that Mel's right into saying that our frameworks are different and that there are these like three different levels. There's the objective thing that's happening within the virtual reality experience. There's the subjective experience of whoever's going through it. And then there's the content of whatever is being shown that's doing these different levels of engagement and immersion. So that's all that I have for today. Just wanted to thank you for tuning in, listening to the Voices of VR podcast as we continue our exploration into presence within virtual reality. And I just wanted to make the announcement that, you know, I'm starting to change the metaphors that I think about Patreon. I've been talking about people becoming a donor and you supporting them financially and donating. And I'm moving into a mindset of like a subscription and you become a member. And with the membership, you get certain benefits of that membership. whether it's early access or live streams or Q&As or discussions or VR meetups. I think all those different dimensions of the membership is in the process I'm trying to figure it out. What are those different reward tiers? So you can become a member on my Patreon at patreon.com slash Voices of VR and I'm in the process of trying to figure that out. What's that look like? Who are you? Introduce yourself. What are your goals in virtual reality? What do you want to try to achieve? And how can the content that I'm already producing here start to feed into what you want to make to make your wireless VR dreams come true? So go to patreon.com slash Voices of VR and become a member today. Thanks for listening.
