Domna Banakou is a Ph.D. student studying with Mel Slater at the Event Lab in Barcelona, Spain. She’s been researching different aspects of the virtual body ownership illusion in order to investigate the short-term and long-term impacts of embodying virtual avatars which have different qualities than your physical body. Specifically, she found that embodying the avatar with childlike proportions can result in an overestimation of the sizes of virtual objects. She also has found that it’s possible to create the illusion of attributing things that appear to be coming from your avatar, but that you hadn’t actually said. There are a number of potential implications for how the virtual body ownership illusion through VR can alter our sense of self and identity, and Mel Slater’s Event Lab is on the forefront of investigating these questions.
LISTEN TO THE VOICES OF VR PODCAST
The virtual body ownership illusion occurs can be induced when the limb tracking within a virtual reality experience is accurate enough. Mel Slater has said that for most people who haven’t experience virtual reality, 100% of the time that when we’ve looked down to see our body, then it’s been our body. So it’s surprisingly easy to trick or fool the mind into believing that a virtual avatar is your body.
Domna Banakou has been researching some of the psychological implications of virtual body ownership, and found that embodying a child avatar changes our perception of sizes and can have an implicit impact on our attitudes. Here’s the abstract from here paper titled “Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes:”
An illusory sensation of ownership over a surrogate limb or whole body can be induced through specific forms of multisensory stimulation, such as synchronous visuotactile tapping on the hidden real and visible rubber hand in the rubber hand illusion. Such methods have been used to induce ownership over a manikin and a virtual body that substitute the real body, as seen from first-person perspective, through a head-mounted display. However, the perceptual and behavioral consequences of such transformed body ownership have hardly been explored. In experiment 1, immersive virtual reality was used to embody 30 adults as a 4-y-old child (condition C), and as an adult body scaled to the same height as the child (condition A), experienced from the first-person perspective, and with virtual and real body movements synchronized. The result was a strong body-ownership illusion equally for C and A. Moreover there was an overestimation of the sizes of objects compared with a nonembodied baseline, which was significantly greater for C compared with A. An implicit association test showed that C resulted in significantly faster reaction times for the classification of self with child-like compared with adult-like attributes. Experiment 2 with an additional 16 participants extinguished the ownership illusion by using visuomotor asynchrony, with all else equal. The size-estimation and implicit association test differences between C and A were also extinguished. We conclude that there are perceptual and probably behavioral correlates of body-ownership illusions that occur as a function of the type of body in which embodiment occurs.
Donna has also found that it’s possible to trick your mind into believing that you’re saying something, when it’s actually your avatar doing the speaking. After inducing the virtual body ownership illusion through accurate limb tracking, Domna describes an experiment where they had participants watch their movements in a mirror and provided haptic feedback called “synchronous vibrotactile stimulation on the thyroid cartilage” while asynchronously moving the avatar’s lips. It was possible to cause the participants to misattribute that they had actually said these things by both explicit surveys and implicit measurements that showed their fundamental speaking frequency had shifted.
Here’s the abstract from that paper titled “Body ownership causes illusory self-attribution of speaking and influences subsequent real speaking:”
When we carry out an act we typically attribute the action to ourselves, the sense of agency. Explanations for agency include conscious prior intention to act followed by observation of the sensory consequences, brain activity that involves feed-forward prediction of the consequences combined with rapid inverse motor prediction to fine-tune the action in realtime, priming where there is e.g. a prior command to perform the act, cause (the intention to act) preceding effect (the results of the action), and with the common-sense rules of attribution of physical causality satisfied. We describe an experiment where participants falsely attributed an act to themselves under conditions that apparently cannot be explained by these theories. A life-sized virtual body (VB), seen from first person perspective in 3D stereo, as if substituting the real body, was used to induce the illusion of ownership over the VB. Half of the 44 experimental participants experienced VB movements that were synchronous with their own movements (Sync), and the other half asynchronous. The VB, seen in a mirror, spoke with corresponding lip movements, and for half this was accompanied by synchronous vibrotactile stimulation on the thyroid cartilage (Von), and the other half not. Participants experiencing Sync misattributed the speaking to themselves and also shifted the fundamental frequency of their later utterances towards the stimulus voice. Von also contributed to these results. We show that these findings can be explained by current theories of agency provided that the critical role of ownership over the VB is taken into account.
Here’s a summary of other experiments into how VR can impact the implicit attitudes, behaviors, and brings up a lot of philosophical questions about the sense of self:
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:12.095] Domna Banakou: My name is Domna Banacu. I'm working at the University of Barcelona at the Event Lab with Professor Mel Slater. He's my supervisor. And our research in general focuses on embodiment, body ownership and agency in virtual environments. And this is what I'm also working on, maybe with more specific experiments, but the idea is to see how ownership over a different virtual body than your own that you see in a virtual world can affect your perception of the virtual world and can alter your behavior within this virtual world or eventually outside and transfer this behavior and attitudes into the real world which is what we're trying to investigate.
[00:00:56.262] Kent Bye: Great. And maybe you could start with maybe some of the first experiments that you did along that path in terms of what you were trying to evaluate and what you found.
[00:01:04.586] Domna Banakou: OK. So the very first experiment I did was to embody adults into the body of a child. And that was because there's been already research about ownership over different bodies, but there was not much research focused on a completely altered body in terms of shape, let's say. There are studies with different race, for example, etc., but owning a child body that is a completely different shape and form compared to an adult body hadn't been investigated, so we thought let's go ahead and test this out and see can we really embody people into a child body and how can we test this. So we did it, we had our participants, they were all adults, Students, however, as a first phase of the university, they came to our lab, they experienced ownership over a little child who was either a girl or a boy, depending on the gender. and they enter the virtual world and first phase they didn't really do much they just had to explore their real-time capabilities of their virtual body they had full body tracking, head tracking etc. We asked them to do some simple stretching exercises to see how they move their bodies and then we had to test whether they accept this body. So the most basic way to do it is to give them a questionnaire but this is a very subjective measurement so we had to find some other ways to test this. Therefore we first adapted a test that they use a lot in psychology it's called an implicit association test and we found that they matched items about themselves with children or adults and when they had a virtual child body they had better association of these items with children compared to adults and the result was compared to our control let's say group that was participants having not a child body but an adult body that was however scaled down to the height of a child. However, the proportions of the limbs, the legs, the face are completely different to a child. So you can see visually that is something different. And the other measurement we had was, we're trying to test perception of the world, was to ask them to measure with their hands some virtual objects they saw in front of them. And the idea was based on our concept from childhood that when we were all little we had perception of things being like really huge around us and we still look back at our school years and remember oh I remember this toy it was so big now it's small or it's normal etc and we found out that when people had the scaled down adult body of course they overestimated objects because they were small but when they had the child body they overestimated them even more compared to their baseline than the adult body And to further test these results, we run another control group where we extinguish the body ownership, let's say. It's a method that we use in the lab in general. And that is done by not giving them body tracking, like full body tracking. So while our participants are moving and they're trying to explore the body, they see either an animation or a completely asynchronous movement of their own body. And we have found that it extinguishes the illusion completely and therefore their results.
[00:04:24.104] Kent Bye: Interesting. So yeah, it sounds like, you know, when you're put into a child body, then you start to identify with it. And what else could you say in terms of what is it changing with the sense of self or with your sense of identity when you're going into a child avatar?
[00:04:38.093] Domna Banakou: Okay, at this point about the self, maybe I cannot say concretely as we just manipulated measurements in the world is object sizes. So it's like a perception of the size of the world. And we just did this association test, but does not really tell us a lot of well ourself it's supposed to be an implicit measurement but perhaps it's better to find some other measures just so hard is full to find implicit ways of measuring something however as a first step since we saw that you can kind of accept this body among other bodies that it can be your own The idea is to apply it in other research in psychology, like, for example, in cases where psychologists have to deal with intrafamiliar violence and so on and so on, and they adopt techniques of the empty chair where you have to, if you're familiar with that, a person has to sit in a chair and has to speak to, well, an imagined person, can be wife, a child, father, mother, whatever, and then they have to switch chairs and they have to reply back to themselves or this person. And in such cases virtual reality and this manipulation of the body will help us to really put someone in the shoes of someone else, let's say, and see how this can affect themselves and their attitudes towards others. So the idea is to take it further and really apply it in psychology and treatments that currently exist and see how that will help.
[00:06:08.160] Kent Bye: Great. And I know during the keynote yesterday by Mel Slater, he was showing videos from the event lab. And one of them was, I think, one of the experiments that you also worked on, which is looking into a mirror and then having a sense of self by, you know, having limb tracking with it. At some point you have the avatar start speaking and lips moving and the words are not coming from the person that's in the study. What was the reaction to people when you're in an avatar and it just starts to speak and say words?
[00:06:35.149] Domna Banakou: Yes, they basically confirmed our idea that since I see a body, virtual reality, and we know so far that after having synchronous body mapping, I accept this body as being my own, and this body moves as I do, this body does what I do, therefore it's my body, and after a point I accept that it's my body, then this body suddenly starts speaking, so you think it must be me who's doing the speaking eventually, and all the idea, what I was thinking before running this experiment is that Well, more or less all of us are playing video games, like first-person video games, and although so far they're not really immersive, it's just on the screen, if you play, say, many hours, after a while you really identify with your character, and although at some points you just have to follow the story and your character speaks by himself, you know, sometimes people get into this role and they really feel like they are their character. You know, we thought, does it really work? It's like if we make their virtual self speak, will they feel like it? And yeah, it seems that it really works. And that was the first time we'd like to see whether this extends to other body movements, because it's not the same like moving in hand and speaking are completely different circuits in the brain. And we're not sure whether that will work, but it's up to testing.
[00:07:55.022] Kent Bye: And so how did you measure that or test that in terms of, you know, if you have an avatar that you're embodied in virtual reality and just start speaking, how do you measure at the end of it whether or not you were thinking that that was actually you speaking?
[00:08:07.007] Domna Banakou: Yes, there's been other research and our measurements were based on previous research on speech that researchers have done. And what you basically do is measure their voice fundamental frequency or pitch if you want. So what we did was, as a first stage, when they first came to our lab, was to ask them to speak the words that they would hear later on. And we recorded their speech beforehand as a baseline that we use as a baseline. So later on, they did the experiment, they saw the avatar speaking, etc. And right after the experiment finished, they were asked to read out loud the words that were shown on the HMD, let's say, so they had to repeat the same words. And what we did was to test the difference in this fundamental frequency after the experiment minus the frequency before and do the appropriate analysis and see how that changes before and after. And we found that those who accepted the body and they really had ownership over their talking avatar, let's say, had a shifting of their fundamental frequency towards the frequency of the virtual body. And that was because we actually designed our virtual avatar to speak with a higher, let's say, average frequency compared to our participants, compared to Spanish speakers in that case. Maybe if you do the same with English speakers or German or French, it will be different, but each region has an average. fundamental frequency so we compared these and we chose a higher one and in the case where they accepted it they had a shifting of their frequency towards the voice of the virtual avatar and that can be accepted as a measurement based on other studies that there is a change.
[00:09:50.421] Kent Bye: Oh, wow. That's really involved. But when they're in the virtual body, you're not using their own pre-recorded voice. You're using sort of another voice that's just sort of shifted at a higher frequency. And then when they get out, when they say the words again, it actually shifts their fundamental frequency if they identify with it. And how do you measure whether or not they identify with it?
[00:10:10.129] Domna Banakou: Well, the actual shifting of frequency is an implicit measurement of that. And then we also have the questionnaires that we address. And there's a lot of questions about whether they accept the body, whether they accept that they are the agents of the body, whether they felt like the voice they heard was coming from their heads, let's say, or from somewhere in the room. And there were differences between the different groups.
[00:10:35.300] Kent Bye: I see. And so were there people that actually did not accept the body? And then if they did not accept it, then was their frequency at the same level?
[00:10:43.097] Domna Banakou: Yes, those who did not accept the virtual body was more or less at the same level and they were cases that they actually shifted the opposite direction. And we have proof again from other studies that shifting towards the opposite direction is an intention from the individual to actually bring their voice kind of towards their original voices. They're just trying to shift this error that they perceive towards their own voice so they tend to lower their voice, it's not correct, but to change their frequency the opposite direction just to compensate for this error that they perceive.
[00:11:18.781] Kent Bye: Yeah, and one of the things that Mel mentioned yesterday is that he found that dancers, people that are very familiar with their body and positions, that when they get into a virtual body are less likely to accept the virtual body because they're just more attuned to the differences in the virtual world versus their actual body. And so have you found other trends in terms of what may predict whether or not someone is going to accept a virtual body or not, or some of the other patterns that you've sort of seen when people are in a virtual body, but yet they don't accept it as their own?
[00:11:48.568] Domna Banakou: We haven't really run studies to test that. It's just what we've seen running pilots and running studies that we can reach this conclusion. But maybe it's a good idea to just run a study focused on that to see how the trend is changing. Of course there are individuals who join the participant and they report no ownership, no feeling at all, in which case we might have to run, for example, at the end a different kind of analysis and we have to separate our groups based on those who felt the illusion, those who didn't in case. we cannot really find anything interesting. But the majority of them seem to accept the virtual body when it's working properly. Well, the other group who doesn't accept but doesn't focus, I would say, that much is, again, what Mel said yesterday, computer scientists. They come to the lab and they focus more on the technical part of the whole experience and not really on the illusion itself. So, yeah, we're trying to avoid recruiting computer scientists for our experiments and focus on the general population.
[00:12:52.247] Kent Bye: Yeah, I think the quote that Mel said was that the further away that your test subjects of virtual reality experiments are from the computer science department, the better your results. Because they tend to focus more of the errors or, you know, the textures or something within virtual reality. Maybe you could talk a bit about some of your other experiments in terms of racial bias or embodying the avatar of someone who has a different race than yourself.
[00:13:16.369] Domna Banakou: Yes, we just finished running a second study that would be on racial bias in our lab. The first one we found there were differences when people were embodied in a dark-skinned avatar compared to a white-skinned avatar. My colleagues ran the study, not myself personally. But now what I did was to kind of do a similar study, again with a light-skinned avatar or a dark-skinned avatar, and see how long the effect will actually last when taking the participant out of the system. I really cannot say much about that because we just finished running the experiment so we haven't seen any results so far. So it's going to be very interesting to see whether there are long-term effects of these changes or it's just temporary as for the time that you spend in a virtual environment, perhaps like the next five minutes, one hour, one week. So we had a kind of long-term study so far. It's lasted not a year or years but a few months now. So our participants were exposed more than once in the virtual environment or different groups and like with difference of days or weeks to the illusion. So we have to see how these differences will affect their perception and behavior, if any.
[00:14:33.592] Kent Bye: And I think one of the concluding remarks that Mel said yesterday was that, you know, virtual reality not only changes your sense of place, but it can kind of change your sense of self. And I think, to me, that's one of the most exciting things about the potential for virtual reality. And I'm curious, how would you describe how what you've seen, at least in the short term, when people go through these different experiences and you do these scientific questionnaires and evaluations, how does virtual reality have the capability to change the sense of self or behaviors or some of these implicit measures that you've been able to do so far?
[00:15:06.144] Domna Banakou: Okay, again, it's just a belief that we have that it will in the long term change behaviors. It still has not been tested. We are running experiments as we speak to see the effects. But we do believe there's going to be a change because in the end, we kind of apply the same very similar psychology techniques now they're using psychology but we have the means of virtual reality and it's just easier to test things that otherwise individuals have to imagine and not everybody has the same imagination so now with virtual reality we just give them experience they do not have to imagine anything they just see everything in front of them and these psychological treatments really have these results in the long term we expect that virtual reality we have the same or even better results just because they see the experience and it's very vivid, it's right in front of you.
[00:15:58.058] Kent Bye: Yeah, and because you're doing this research, I'm curious what is motivating you to kind of investigate this and where you'd like to kind of see this go in terms of learning about VR and perhaps how it might be applied in the future.
[00:16:11.248] Domna Banakou: Well, regarding apply to the future, the studies that we are doing, like I said before, focus mostly on psychological, let's say, treatment right now, while in perception of ourselves. That's why we are collaborating with many neuroscientists that have a better background of perceptions of ourselves. I'm not an expert on that. So yes, they have more concrete ideas how this can help. And yeah, in the end, it's not only psychology and any kind of patients from schizophrenia like mental disorders everything covers a wide range in these fields and I see it's going to be very very powerful for the future in these fields and of course it's something that you kind of not know right now what the impacts will be in the future it's up to us to explore and investigate it's Well, it's relatively new and old field. It's an old field but it's quite still new because all these concepts of our self and perception and embodiment are quite new and we just started exploring them so we don't really know what's going to happen. It's just up to us to explore it more and test and do as many tests as we can, the whole community, and produce as much research as possible to see the effects in the future and how it can affect our fields and virtual reality in general.
[00:17:27.307] Kent Bye: And when you've observed from even yourself and other people, when they're able to kind of have this sense of presence in a virtual reality experience, how do you describe what is some of the key parts of what is allowing people to have that sense of presence?
[00:17:42.190] Domna Banakou: From what I've seen, regarding the technical part, if that is what you're asking, you don't really need to have many cues. As long as you give a person a body, the moment they look down and they see a body, they kind of accept that it's the body. I think Mel also mentioned yesterday that we are used our whole lives when looking down to seeing a body. It's the same in VR. I look down, I see a body. Well, I accept it if I have first-person perspective. From there on, if you start moving and you see your lips moving synchronously with this body, you turn your head and you perceive the world as you would in reality, then immediately you get this feeling of being present in the world, which is the meaning of presence as well. So I would say, yeah, the minimum, just having a body that is, well, Human-like or not, there is a discussion on that whether the body has to be human-like or can be any other object. So the community is kind of divided. Maybe we need more studies to test that. Yes, but as long as you see a humanoid body, let's say even if it is a robot, you kind of accept it and you feel present where you are. The moment you start moving, it gets even better. And when you can interact with, say, other virtual humans, then you get the concept of the plausibility illusion, no? You interact the world and you can get feedback, and you know that this feedback has to do with your actions, et cetera, then the better the system. And, of course, then technical aspects, the HMTs you use, et cetera, et cetera, you know, they add up into this feeling, but the basic one still works.
[00:19:24.327] Kent Bye: Are there any experiences in virtual reality that you really want to have?
[00:19:28.702] Domna Banakou: Yes, there are multiple studies on fear of heights and well I'm afraid of heights so I had this experience myself and it was quite nice. I think that if I was exposed to it more maybe it would kind of help me. I don't know, I can't really say. There are, yes, there are I guess multiple examples out there that I would be interested to try. I cannot think of anything concrete right now but Yeah, if I give it some more thought, maybe it's a new experiment I can run.
[00:19:59.035] Kent Bye: So it sounds like treating phobia for your own self, like the other experiences of being in really high places and to be able to actually perhaps rewire your brain so you're not afraid of heights anymore, it sounds like.
[00:20:11.430] Domna Banakou: Yes, exactly. And this treatment has been used for a while now, treatment of phobias in virtual reality. There's a bunch of studies of like fear of heights is the most usual, fear of spiders, public speaking, any kind. And yeah, it seems it has proved to work so far and it's quite powerful. So yes, it's very nice to see that VR system works and it's accepted by people.
[00:20:35.674] Kent Bye: And finally, what do you see as the ultimate potential for what virtual reality may be able to enable?
[00:20:42.426] Domna Banakou: The ultimate potential? Okay. I hope make us better human beings in a way. Hopefully we'll go to this direction and not the opposite, which is what we have to take into account when creating these systems. We just don't have to let it take a very bad turn to what we're doing because unfortunately people can exploit it in a very bad way. So, Yeah, for me it would be great if it just made us better human beings and we just empathized more with others and we cared more about things that we don't pay attention to in our everyday life, but when they are given to us in such a way we can really, you know, think about and see, oh yeah, I need to consider this in my everyday life.
[00:21:30.316] Kent Bye: Okay, great. Well, thank you.
[00:21:31.477] Domna Banakou: Okay, thank you very much.
[00:21:33.653] Kent Bye: And thank you for listening. If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash voicesofvr.
