Betty Mohler is a researcher at the Max Planck Institute for Biological Cybernetics, and she had some keen insights the nature of the uncanny valley being connected to expectations in my previous interview with her. I caught up with her again at the 2016 IEEE VR academic conference where she was talking about a recent paper about Appealing female avatars from 3D body scans: Perceptual effects of stylization. They used an automated way to stylize a 3D scan across a variety of different animated art styles, and found that most women preferred to have at least some stylization in their avatar. I talk to Betty about these findings, some of the associated ethical issues, and how this 3D avatar research could be applied to help treat people with eating disorders and body dimorphic disorder.
LISTEN TO THE VOICES OF VR PODCAST
For more discussion about ethical issues in VR, then be sure to check out this article that Betty mentions: Real Virtuality: A Code of Ethical Conduct. Recommendations for Good Scientific Practice and the Consumers of VR-Technology.
Here’s the linear avatar model by Michael Black called SMPL: A Skinned Multi-Person Linear Model (SIGGRAPH Asia 2015)
And finally, here’s the paper that Betty mentions about embodiment within child avatars: “Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes”, and here’s my interview with Mel Slater as well as the lead author on that paper Domna Banakou.
Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to the Voices of VR podcast. On today's episode, I have Betty Moeller from the Max Institute for Biological Cybernetics. When I last talked to Betty in episode 129 of the Voices of VR podcast, we talked about the Uncanny Valley and how the Uncanny Valley is really that feeling of disgust that you get when whatever you're expecting to see isn't matching what you're actually seeing in VR. And so what a lot of people have found in VR is that you really need to have some sort of art style that's stylizing something that moves away from this photorealistic look. In this podcast, I talked to Betty about some ways that one of her students, Ruben Fleming, has found to start to try to automatically give these stylized looks to digitized 3D body scans. So they have an art style from Marvel and Disney, Sony, Pixar, and Barbie. And they're able to allow people to have a slider to be able to determine how much stylization they want within their digital avatar of their 3D body scan. And what they found is that most people want some stylization because the least appealing is either the most stylized or the least stylized. In other words, having no stylized avatar at all. In this interview, we talk a bit about some of their findings in this, as well as how this is applied to stroke rehabilitation, as well as some of the ethical issues that come up when talking about body image and VR, and what the impact of that is when you come out of VR back into the real world. So, that's what we'll be covering on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. Today's episode is brought to you by the Virtual World Society. The Virtual World Society wants to use VR to change the world. So they are interested in bridging the gap between communities in need with researchers, with creative communities, as well with community of providers who could help deliver these VR experiences to the communities. If you're interested in getting more involved in virtual reality and want to help make a difference in the world, then sign up at virtualworldsociety.org and start to get more involved. Check out the Virtual World Society booth at the Augmented World Expo, June 1st and 2nd. So this interview happened in Greenville, South Carolina on March 22nd of 2016. And it was at the IEEE VR conference right after the banquet dinner, where there was a number of awards giving out, including a Lifetime Achievement Award given to Tom Furness, as well as a Technical Achievement Award given last year to Oculus, but there was no one there to accept it. So Michael Antonoff was there to accept the award this year. So with that, let's go ahead and dive right in.
[00:02:54.278] Betty Mohler: I'm Betty Moeller. I'm from the Max Planck Institute for Biological Cybernetics. And since the last time we talked, I also joined the Max Planck for Intelligent Systems. And I am studying primarily perception and action in virtual reality. And more specifically, lately, I'm studying how people perceive their own bodies and how that changes the way they act and see the world.
[00:03:15.783] Kent Bye: So what can you tell me about that then? What have you learned?
[00:03:18.460] Betty Mohler: What have I learned? Well, actually, we just finished hearing Fred Brooks talk about Ivan Sutherland's Ultimate Display, and that's a perfect thing. I mean, like he said, if you haven't read it and you love VR, read that Ultimate Display paper. And what he says, and I'm probably going to screw up the quote, but he says, you know, he talks about Alice in Wonderland and he's really saying, like, the ultimate display is a thing where, you know, a chair you can sit in and a bullet will kill you, right? And I love the Ultimate Display Paper because, especially about this Alice in Wonderland quote, right? And today, we haven't achieved something very simple, which is Alice in the Wonderland. right? Alice in Wonderland experiences this very magical thing. She becomes really small, you know, or she becomes really large. And that's something that in VR and as entertainers, we should be able to give people that experience. And so I pretty much study that. I study, why is it that our body is so important and so stable the way we feel about it and how that influences the way we experience the world around us. So can we make people feel like they have a small body? If we do, does it change the way they act? And I've found that it does, but I think we're far away from the true experiences people can have with the coupling between their body and the world around them.
[00:04:31.773] Kent Bye: And so we were talking earlier about looking specifically into the Uncanny Valley and giving people kind of like a slider where they can go somewhere between like photorealistic all the way to like a Marvel stylized avatar based upon kind of like a Marvel character. So what were you looking at, first of all, when you're looking at this in Kenny Valley, and then what were some big takeaways from that?
[00:04:53.797] Betty Mohler: Okay. So yeah, that's work that a PhD student of mine, Ruben Fleming, and my colleague that you've also spoken to, Martin Bright, and I, and the colleagues at the Max Planck Institute for Intelligent Systems have done. And it's so exciting because what we've been talking about also here at IEEE VR is what is our body going to look like? And so many times now the body is diminished. It's just your hands, maybe a little bit of a head, you know, but I don't believe we're going to stick with that for very long. Certainly not when we go to social interactions. We want to have a body. We want to represent ourselves some way. You know, we want to put clothes on and be a certain character when we go to the club or we're meeting with people. And the real question with our paper that we just presented at Grap in Rome is, what will that avatar look like, right? So at the Max Planck for Intelligent Systems, Michael Black leads a group that has this amazing scanning technology. So we can scan you, we can get your body perfectly. We can get exactly you, exactly Kent, you know, into VR. But one question is, do you really want that? Do you want to be Kent, right, in VR? And why would you want to be Kent in VR if you could be something else, right? Or if you could just snap your fingers and change your clothes get yourself a lot of biceps or whatever, right? So what we did was, not yet in HMDs, not yet really in immersive VR, but on the desktop, we asked people, which one is the most appealing character? Right. We actually in total, we looked at six original actors. We've only published the female side so far, but what we enabled them to do was to just either say, I find it appealing or I don't find it appealing or to use a slider, as you said, and make it appealing. So make the most appealing version of that actor. And what we had hypothesized and that turned out to be true was that people would want some stylization. that they wouldn't really want it to be the original actor, and that they also wouldn't want to take a human and turn it into Marvel, not completely. And that's what we found. We actually found that for Marvel, you could get away with more than Disney, and you could get away with more stylization of Disney than you could of Sony. And we think that was because Marvel is pretty human-like. It's superhuman, but it's pretty human-like. where Sony, you know, we just took one Sony character, it's not representative of all Sony, one Disney character, but the Sony character we chose was really more alien-like, right? So we found that people, and across all these different actors and across a lot of different people, they preferred a certain percentage stylized. And we also asked questions, other questions, like make this character repelling, for example, so that we could really get at do they really find just 30% the most appealing and the rest are also quite good? And what we found was that they really thought that most of them, like the highest repelling one, was the 100% stylized. So taking a human and turning them into Elsa, for example, it didn't work 100%. But the second most repelling character was the original Scan and I think the reason for that comes back to like our conversation last year at IEEE VR Which is that when you scan a person? into VR, you have a digital representation of them, unless you're doing that 90 times a second, you don't have a representation of them. You know, you're missing things and people don't know what to expect. And it's even heightened because they see that. Like I was telling you earlier, there's a digital Betty now that I have on my computer and that I've looked at, and I've also viewed in HMDs. And when I look at her, even though it's a moment of time representing me, I think, what can she do? What does she do and she doesn't her eyes don't blink, you know, and there's this Uncertainty and I think taking it into my colleague Martin I think would say too is taking it towards the stylized version makes me less weary of that It's like I'm learning now. What can that character do? And it's okay maybe if my virtual self now looks a bit more stylized that I move a bit differently as well, right? That I maybe would be willing to accept that. And so I think we're going to see more and more of this kind of thing. I hope we're going to see more of this in the future. And our next step will really be to do that in head-mounted displays. So getting two people in there, getting them to pick their avatars for and use that slider for a specific task like interacting or playing a game.
[00:08:53.858] Kent Bye: Yeah, to me it seems like, like you said, like a 90Hz update rate of all the different micro-expressions, social cues, eye gaze, different ways that the face moves and just body language and those social cues that happen. Do you think that when it goes to that 100% photorealism that we really kind of expect that to be that level of precision of all those other Subtle cues otherwise in order to get away from that we add some stylized kind of cartoony Which you know a lot of cartoons we've seen they don't have those nuanced expressions It is we are used to watching Pixar or cartoons where you know We kind of have a little abstract representation of that rather than sort of like the very nuanced like human like behavior So yeah, well, I guess what do you make of that?
[00:09:39.427] Betty Mohler: I mean, I think you're absolutely right and I think we can take advantage of techniques that animators have used, you know, and granted they have a lot of time to use them, you know, and to think about them and get them into their movies and get them into each and every frame, but we can think about those techniques and how they match up with the kind of tracking technology we're getting into the home, right? So I really think this is something that people will be using and will enjoy because A lot of my other research, I mentioned to you, involves scanning people. So since we spoke last year, we've scanned 100 people, most of them women, because we're researching body image, how people feel about their bodies, and disturbances in how they feel about their bodies. And it actually ties really well into the stylized characters, because you look at all the animated characters out there, and they're usually thin and tall, right? They're extremely, you know, or childlike. They have childlike faces, and they're thin and tall. And so, anyway, I mean, I think that there's a lot From the artist community. It's not just the gaming community. There's a lot from the artist community of how we can understand How do people want to represent themselves? But I think there's another side to that too just to kind of change topics a little bit is there is ethical questions there So I'm excited. We also are doing the studies of do certain groups of people like do teenagers Say have an unhealthy view of how much stylization they should have right? We've looked at a normal population people that are don't have eating disorders, for example, or don't have any body image disturbance, just the normal population. But we're going to look also at people, the same study where you look at how much stylization will you do with teens, say, which the research has shown that they are sort of biased. Women are dissatisfied with their body. We've published that as well. People, when they're given a slider on their own body, in other studies that I've done, women want to be sometimes 10% BMI thinner, you know, really just so much thinner. They want to lose five to 10 pounds. They're dissatisfied with their body. Fortunately, we found, and many other research has shown that too, that men, they're not dissatisfied with their body. But anyway, we really want to use this as sort of twofold a tool. to study how people perceive their bodies and how that changes the way they act with other people, but on the other side, to build avatars so that we can actually have a social experience in VR. So, and tomorrow in our panel, I think, I hope we have time to talk about that. Specifically, I'm working with Bernd Froehlich's team and Alex Kulik, and Alex will be in our panel. And also as our Oculus Award winner tonight talked about it, I think there is the real excitement in VR is when it can become social. Like right now, it's still clunky a little. It's still this sadness that it's an isolation experience. Maybe not for everyone. There's certainly things that are fun to enjoy alone. But, you know, you want to share it with other people. And we have to find a way to do that and a memorable way to go into VR with people we know.
[00:12:23.070] Kent Bye: So it sounds like you're using this really advanced kind of scanning technology to get very photorealistic scans of yourself. And then somehow you're morphing this into these different stylized art. So you've been able to do this to yourself and do the slider and be able to kind of judge what's appealing to you. So why don't you share a little bit of your own personal experience of what was that like to be able to kind of change your body representation virtually in that way and then your reaction to that?
[00:12:50.429] Betty Mohler: Yeah, I want to say two things about that. One is it was and is so much fun. I just had a student and my colleagues, the artist that I mentioned, Ruben Fleming, working on that as well as Martin Bright, getting it into a head-mounted display. And it's amazing because you look at yourself, this digital copy of yourself, and then you know, I have this, I can just press a button or ask my student to, oh, make me more Disney, make me more Marvel or combine Disney and Marvel. I want to see what that looks like, right? And then what's really interesting is, well, I was surprised by that I, because I think I have a very good image of my body. I mean, I don't have any, I'm not sad about how I feel, but I, I wanted a lot of Marvel and Disney, you know, I wanted equal or more to the participants that wanted it of these other actors that were, didn't mean anything to them, you know? So I was happy, I would say with 50% towards Marvel and even though she gets really big busted and you know she's much taller and the Disney one gets a little shorter but again really narrow, super super narrow waist but I think something that we also want to go after is that my experience after that and then I said okay now put it back or then I take it off and I look down and I really have this feeling like oh you know Well, I don't look like that, do I? Right? Like, I'm not like that. And so, and that's a pretty common phenomena known in the literature of face literature, I think being studied now in body literature, which is just called adaptation. You adapt to the visual stimuli that you see. And so, I'm really excited about, for example, everybody's always talking about the ethics of VR. Thomas Metzinger, a famous philosopher in minds in Germany, just wrote a code of conduct for VR. And that's in Frontiers, it's open source, everybody can read it. And I think it's important, actually, that we start having a code of conduct in VR, and this kind of falls into that. Like, I'm writing research grants, and I really want to do games for good, specifically with this eating disorder population. But what I just described, my experience, as someone who's healthy about their body, feels good about how I look, I have an obligation to understand that experience before giving this technology to people that, or even doing an experiment with people that don't feel good about their body. So but then I want to say on the second thing is that yes, we have this amazing technology that gives us the ability to do body scans. But what's so amazing is Michael Black's effort to make that open source. So if you just look up Michael Black on YouTube, and type in the words SMPL, simple, you'll find that he's made everything available. So yes, you can't, with that software, make a digital copy of yourself, but you can get pretty damn close. And it's really amazing. You can also port it into Unity. You can do a lot with it. And so I think, and he's already promoting it and making it open source available for academic purposes. I'm thrilled about that. So it's not going to be something that's just in our lab. And towards Alex's work as well, they use Kinects to do their work. And I think in combination, the kind of stuff that they're doing where they're getting point cloud data, they don't make a mesh, which would make it hard for us to manipulate the bodies, but they get just the raw connect data. I think we're going to be seeing more lifelike, more sort of accurate representations of people in the future. And you're going to have to do something to fix them. Like if you take the connect approach, you have holes, you have problems with it, right? So you're going to need to do something to make it sexier, so to say.
[00:16:03.705] Kent Bye: And so what are some of those ethical implications of this that you mentioned?
[00:16:08.153] Betty Mohler: Well, I was just reading it today. Metzinger has this first table is just like, it's great for people that want to do research. He talks about long term effects, sort of follow up effects. And the one that I was just, he actually mentions many, many things about our obligation. In fact, he has two kind of code of conducts, one for scientists, sort of people using VR for some kind of means, right? Which is pretty standard, just, you know, make sure you inform people what you're doing and what the possible consequences could be. Make sure you go to a group of people who are external to your scientific interest to determine what the risks are. But he has a code of conduct also for the public, which I think is great, because people are getting VR in the homes. And you may not think about the negative after effects of VR. And we've been talking at this conference about a lot of them. It's surprising. You just might not know how disturbing an experience might be for someone. So I talked about, I mean, I think I can only speak about what I'm doing right now. But for example, in our eating disorder research, we are asking people, Of course we screen the eating disorder people such that the therapists have to agree that they can do the experiment in the first place, but we also ask them how anxious they are before the study and before the scanning. So we take a person who has a problem with their body and put them into a body scanner where 22 cameras and depth cameras are taking every piece of information about their body. And so, you know, I think we're doing a good job because we did go to an ethical committee, a medical ethical committee, as well as we're trying to be responsible and gather the information as to how is that experience for those patients, both before and after the scans. And that's super important, I think. And also just super important because of the individual differences. You put an HMD on someone and if they don't know what they're going to see, then they could see things that really, really disturb them based on their prior experience and what they came to at this time. So, yeah.
[00:17:58.755] Kent Bye: What are you doing specifically with these subjects who have an eating disorder? How are you using VR technology to deal with their body image issues in this way? What questions are you asking and how are you approaching that?
[00:18:09.805] Betty Mohler: Well, with all of our research right now, I feel like we're asking the same very simple question, which is, do people that have been traditionally said have a disturbance in their body image or they, you know, everybody in the popular press, when you think of an anorexic person, you think of them looking in the mirror and seeing a normal size body or a bigger body, you know, the normal when they really are our skeleton and skin, right? And that's not really clear from the literature, not just from what we've been showing now, but there's a lot of literature on that. And it's not really clear that they do have a disturbance in visual perception of their own body. So in all of our research, also with stroke patients as well, we're asking the simple question, do they misperceive the dimensions of their body? Right? So back to the stylization, it's the same technology we're using with the eating disorder patients. We just give them a slider and they get to say how much weight they have on their body. And we have then full control over what visual cues they see. What I think is so amazing is in that scenario, and maybe also applies to the sort of entertainment side, is that we can scan them and then we can put them in front of a mirror, so to say, where they see themselves. It's not moving. The avatar, we don't have it moving, but it's in the same posture as them. And we can just say, make it yourself, right? That's great. That's what we do. They say, make it yourself. And then we ask them, make it your ideal self, right? That's two tasks we've done now many times. But then we can also just unknowingly to them swap out just the RGB. So they're seeing the same shape. Everything about shape is the same. but we just change their identity. They don't realize it's them. You know, afterwards we ask them, they don't realize it's them. It's familiar to them somehow, but everything about the shape is them and we ask them the question again. By doing that, we actually can, in a small way, get away from, is visual perception disturbed or are their attitudes about themselves disturbed. And that's work being done by a PhD student that I share with the clinics in tubing and Simona Mulbert and we're not quite finished yet so I can't say the final answer because we're 75 percent through the participant pool but what I can say is it's really exciting to have this technology to do that with those patients. I think this is really in the direction of VR for good, right? I mean, the technology that they use in the past with this eating disorder research is really, you know, many many studies have just taken a picture of the participants and done an aspect ratio distortion. And we all know when you get fat you don't just get wider. Right? So it doesn't make sense. And what we're doing with Michael Black's scanning and modeling technology is really the right thing to go after what the disturbance is. And in the future, we can ask more specific questions. But the main question will be the same, which is, do they have a disturbance of their body? And if so, where? And visually image, right? So because maybe they disturb the fact they think their torso area is disturbed, right? And so with this kind of modeling technology from Michael Black's team, we can go after that. Or maybe they have an attentional problem, that they just stare at the fattest part of their body and they sort of extrapolate that over the rest of their body. But without the tools, you can't answer those questions very satisfyingly.
[00:21:16.115] Kent Bye: So I guess it would be kind of in the realm of perhaps body dysmorphic disorder where people have a distorted view of their own body and I'm just wondering if what you're kind of saying here is that like if you take a scan of somebody's body and give them a perspective of their body that's kind of different than they've ever really been able to look at before other than through the lens of a mirror. that somehow they can have kind of a more non-morphed view of their body or like they have a more accurate or because it seems like it's perhaps an emotional or perhaps not a cognitive like rational thing you know it's like not a thing that they're you're kind of proving to them like ha ha look see we proved to you that your body is not as dis-morphed as you thought it was or I'm just trying to think of like what the mechanism here of what's actually happening
[00:22:01.860] Betty Mohler: Yeah, I don't know. I mean, that's hard to say. There are a lot of other populations of interest, not just eating disorders, but it's not about really revealing anything to the patients, although it's really more about understanding the underlying neural mechanisms, right? It's really understanding what's happening in this whole phenomena, right? And I would say, I mean, the part where it comes back in to helping the patients in any form or any way, is they do do mirror exposure therapy because your kind of discussion about mirrors is, well, if you're just confronting them with their body, then why not use a mirror and why not use like sort of tricky distortion mirrors, right? Of course that can be difficult to distort a mirror the right amount, but importantly, like we don't just do a mirror distortion. That would be this kind of aspect ratio thing. We actually model what their body would most probably look like on several thousand bodies. if they gain just one pound. So we have that precision to say, okay, what happens if this person who weighs just 70 pounds, what if they put on five pounds? What will they look like? And it's pretty accurate. Or, you know, it looks realistic. But I would say I haven't gotten the funding for that yet. And to be honest, I need to learn more about it. I'm trying to learn more about it. But therapists do use mirror exposure therapy. And that's somewhere where I think we need to understand the underlying mechanisms. But there are a lot of disturbances in their attitudes about their body and their language about their body and the way in which they view their body and experience their body. And for that, they do do mirror exposure therapy where they stand in front of a mirror and they talk through how they look at their body. And I think how great would that be if you could use a virtual mirror and you could, the therapist along with the patient could have control and say, you know, your target goal over the next four weeks is to gain five pounds. Let's talk about what your body will look like when you get there. let's confront you with that now in a realistic way and talk about how you're gonna feel and what you're gonna look at, where's your attention gonna be looking at, or where you're gonna be looking, what are your thoughts, right? So I think it could be a really useful tool. I'm not the first to propose that. There are others that have tried to do that and I just think the technology is really ready now to do that. But it's not, I don't think it's just for eating disorder patients. We also work with stroke patients. There it's not about how fat they are but how long they think their limbs are. But I also think for all of us, I mean, it's an interesting thing to think about. What if you go into VR and you, you know, you're given these little hands out there, right? What happens to the way you experience your body? And also afterwards, once you take off VR and you've been playing for a whole hour and you had these tiny little hands that could sure do something cool, but you had these tiny hands in the world. How does that change your perception of the world afterwards? These are really important questions. If we're going to start having people playing immersive games where we're altering their body.
[00:24:41.447] Kent Bye: Yeah, the thing that this makes me think of the stylized part, at least with the Uncanny Valley, is that whenever you sit down and get a caricature artist to draw you, they kind of exaggerate certain features in your face. And if you look through the comics, and you see the same thing with people who, you know, they just make this totally stylized rendition but they're totally identifiable as to what they actually look like and is there something about like going into VR and another dimension that I think has come up here a few times in talking to people is that there's this distance misestimation by up to 20 to 25 percent so your ability to be able to judge perhaps distances and size and proportion may be skewed just by the fact that you're into virtual reality and that there's Something else kind of going on there that when you're looking at yourself, maybe they you want this hyper stylized Version of yourself to make it more recognizable at a kind of a pattern recognition facial recognition way So yeah, I don't know some of those things that came up and I don't know if you have any comments about that I mean, I think it's I think of as two different things I mean, I just got done talking with Martin bright about doing a style.
[00:25:44.921] Betty Mohler: That's a caricature of a person, right? So there's ways to to do that computationally, you know, not with an artist in the space, right? So we're thinking about doing that. I don't think that it really lines up with the space perception research, I mean, or distance perception being underestimated. I've had really nice conversations here about distance underestimation, and most of my dissertation work was about distance underestimation. And I hope that that's a bit of a thing of the past, the distance underestimation. I mean, I found usually 30% underestimation. And it's important to know that while it's a very important problem that people misunderestimate distances, and I think as one of the panels this year that was really great on affordances, they misunderstand sort of what can they do. You know, they need to be told, you can do this. You know, where in the real world, if someone told you, you can touch the wall, you'd be like, yeah, of course I can. Right. But in VR, they need to learn these affordances. I think we also need people to learn these distance estimations. And that's what I found in about 50% of my research of my dissertation was that people, once they started walking around, they were very good at estimating space, but it lines up exactly what we're talking about with the body is that That doesn't mean that when they take off the HMD and they've been walking around, say one of our models we have, we make available virtual tubing into everyone. If you're walking around virtual tubing in and people that have programmed it screwed up the optics or screwed up any number of things, or it's just, you know, not perfect yet. So people misestimate the space that there aren't after effects. So after they take off the HMD, they might be walking a bit differently. They might actually be misperceiving distances in the real world. But what we found myself and a student from David Waller's group found that people just rapidly adapt, right? So what I was telling you earlier, for example, about the body, when I got out of the Marvel eyes, Betty experience, you know, I was disappointed in my body and I definitely was thinking, Oh wow, that can't really be me. It looks almost like I'm, you know, really weird proportionally. I'm weird proportionally instead of, you know, you have to see this Marvel Betty. She's really weird proportionally. But I, you know, have this after effect, but it goes away, you know, but, you know, I still think it's important to understand, but it's something that isn't lasting, but we need to, you know, back to Thomas Metzinger stuff, we need to understand the long-term implications and the short-term implications in order to really model good experiences, to sort of create good experiences that are good, not just in the 10 minutes or hour that you have the person in the VR, but also following.
[00:28:09.370] Kent Bye: And you mentioned the use of VR with people who just experienced a stroke, and how does that relate to body image?
[00:28:14.909] Betty Mohler: That very much relates to body image in that a lot of the literature, although there's, I think, no strong empirical evidence, has suggested that a person who has a stroke and has paralysis in their arm because of the stroke, that they misperceive their arm, that they think it's somehow shorter. And it makes sense in a way that, you know, they can't reach with that arm. So perhaps they have, you know, a distorted image of the body. Also, sometimes they can't have the same feeling sensation when they're touched on the arm. So it kind of makes sense from a neural basis that they may have some disturbance in their body image. And there's been tests in the past that were like holding up a stick figure and looking at the stick figure. And people have a disturbance often in how these images look, how bodies are supposed to look. But again there, what we're asking is, do people really misperceive how long their arm is? And now with VR, what's so great is, and I'm so thrilled about that collaboration because we take an Oculus DK2, into the patient room, the person who's most typically over 65, has had a stroke within the last 48 hours. We screen them, of course, that they fit our criteria and that they're willing to participate, they're given informed consent. But then they put on the HMD and they look down, they see a body underneath a sheet, underneath a blanket, just as their body's underneath a blanket, and then they look up in a mirror and they are asked to adjust an arm, they either see just their arm or one of their arms, or they see one of their feet, and they're asked to adjust them until they're the same size as their own. Right. And so it's not now about weight perception, but it's about arm length perception. And because of VR, it can be about your arm perception. Right. Most of the previous studies had to do with what does an arm look like. Right. And that's not what people claim is disturbed. And that's not what's problematic. That's disturbed. I mean, VR is, I think, really great for stroke rehabilitation. You probably have seen. And even here, there were some very nice research at Clemson about stroke rehabilitation. But the question is, is what are you rehabilitating, right? I mean, obviously you're rehabilitating paralysis, right? But in some forms of stroke, people deny that the hand is even their own. They look down, they see the hand, they say, why is my mom's, you know, my wife's hand here in the bed with me? And that's a true disturbance in body image. That's a problem with the way people perceive their body. But with the majority of the people asking the question, do they misperceive the dimensions of their hands? Because that would help you understand why they're not acting with it appropriately. And we just finished a non-VR study, because we don't always use VR, where we did that. We asked something like 40 patients, both two days after they had a stroke and six months after they had a stroke. how far they thought they could reach with their hand. And we're not ready to publish that so I can't say the final result but I think it's really important to understand what's happening with their perception of their own body because in so much of my other work I've shown that if you do perceive your arm as longer or shorter or bigger or smaller. And the normal population has some variation in that. It's not as if even in the real world we look at our body and we have some veridical perception. There's not accurate, there's just perception, right? So people that perceive their hands as bigger, they will perceive objects around them as smaller, right? So people that have big, you know, football players, they're going to see objects as smaller because they have a better potential to act on them. And so we know it's important And we're looking in the stroke population as well for arm dimensions. And I think this VR thing is so great because the study takes 10 minutes. And so far, we've only had a few participants, but they have overwhelmingly said, oh, that's really cool. Can I play with it now? Which I think is just great. This is over 65 individuals who just had a life-changing experience, not a positive life-changing experience. And with our experiments, they're positive, right? They know that it's for a good cause, that we're helping, we're trying to answer the right question. And they don't experience like, what is this awful VR thing? What is this awful junk you're putting on my head? They actually enjoy it. That's really great.
[00:32:01.966] Kent Bye: I'm just a little confused as to what is actually happening to the stroke patients when they go into the VR experience and then how they're changed. What are they getting out of it? And what are they seeing? And if they would have not had it, they would believe this. And after they do it, what do they believe?
[00:32:15.499] Betty Mohler: Well, they're told that we're doing a research study with the purpose of understanding how they perceive their bodies. They don't come out with any greater understanding. They just know that they get an experience where they actually get to look at an old man in the bed or an old woman in the bed, depending on their gender, and they look up. And what they get to do is they get to say, make my arm longer or shorter, right? So they, it's not a stylized version. It's not fun, you know, but they get to experience a body that's different than their own. in this perfect VR world. So maybe they enjoy actually this sort of cartoon, you know, stylization and maybe they even enjoy getting out of the hospital for a second and away from somebody coming in and poking them and saying, do you feel that? And can you move your arm? And you know, they get away from maybe the seriousness of the situation. I don't know why they would, you know, enjoy it or, you know, but maybe it's, I can say it's because it's a German population and they are enthusiastic about science. They really trust and I think they ought to trust that that we care and we want to understand the underlying cause, right? So, but we're not far enough. We really are just at the beginning of that research. So we're not far enough to really say what we found, but I'm, I'm just so excited. Like with that professor, his name's Hans-Odo Karnath, and he's just a really world renowned expert working with stroke patients and has many publications in that area. You know, I met him 10 years ago and I wanted to do VR with him. And it was just not possible. I mean, he's done a lot of high-tech stuff. He has an icon lab of his own. You know, motion capture tracking lab of his own. And he's definitely been open to all that stuff. But now, finally, it is with total ease we take that technology in there. And my husband is a software engineer and he put a lot of work into developing the graphics, and also the experiments, such that we don't put the burden on the patient. And now, finally, that's possible, right, with this great technology. It's finally possible that we just go in, we ask them, put this on, and they don't get sick, and they don't, they're not really burdened by the discomfort of the technology. And that was something we really couldn't have done five years ago.
[00:34:13.797] Kent Bye: So what are some of the biggest open questions that you see are kind of driving your research in the future?
[00:34:19.060] Betty Mohler: Hmm, good question. I mean, I think one of the biggest questions is back to the body is how do we perceive our body? How flexible is it? How consistent is it that we perceive our bodies in a constant form? And how plastic is that? Right? So, and I've loved that in VR. Like, that's not a new thing. I visited USCICT, who they're hosting VR next year. And I met a guy there who told me, this is now, I don't know, I think almost 10 years ago. He said, I spent a day in VR in an Imagine head-mounted display, and I became, I wanted to be a bear. And I just did it. I don't even know who that was, but I just loved it. I talked to him for a few minutes about it. What did you do? You know, it was just an experience. It wasn't published. It wasn't. But the point I want to make is that can we experience being a bear? You know, can we truly, really? And what is that like? And how plastic is it that we experience what we're like? You know, Mel Slater did such a great PNAS article on becoming a virtual child. And he did really, I think that's a great paper example, if people are going after altering your self character, because he let people be a child or an adult, you know, he had the two groups of people. And then he looked at how it changed your spatial perception of the world, as well as your attitudes about your body. And I think that's the biggest question that I want to tackle in the coming years is How plastic is our experience of our body? And I think the answer is going to be twofold. And important that we go after both things is that some things, they're just not adaptable, you know, or they take a long time to adapt if they are adaptable. And some things are just instant, right? If I truly tried to imagine myself looking in the mirror when I was moving myself to Marvel, that was really me. You know, and it was quick. It was really quick me. But, you know, other things like my eye height, you know, she was growing in height, but I didn't really feel like I was that Alice thing I said. You know, I didn't feel like I was really getting bigger. My body was telling me I'm normal Betty size, right? So I think this is really the main driving question I want to go after. And I guess one that I wouldn't mind going after too is the power of social. social experiences and social presence that can interact with all that stuff. I really am excited about the opportunities to experience VR with many people or you and a partner and how that plays out.
[00:36:36.498] Kent Bye: Great. And finally, what do you see as kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:36:43.862] Betty Mohler: I mean, I think the keynotes tonight hit on that, especially Thomas Furness. I mean, I would really love to see it as a, as an educational tool, as an education in so many ways, not just the sort of high school traditional education, you know, educating people about what it's like to be in a certain country, right? And educating people about certain perspectives, right? Right now in Europe, we have a lot of refugees, but I don't know what it's like for them. You know, and VR could be a powerful tool to convey that and to help people have empathy for others is another thing that Mel Slater, I think, is really pioneering and working on. But yeah, I think I would love to see it be an educational tool. I would love to see it be something that also is a creative tool to help people be creative. And, you know, just like games are doing today. I think games, a lot of people are scared of what games are doing to kids. A lot of people aren't, though, and they realize that it is something that helps make your kids smarter. if they choose the right games, right? They really have a great potential to make a childhood wonderful, as well as improve the skills that your children have and the experience that they have. So, yeah.
[00:37:49.392] Kent Bye: Awesome. Is there anything else that's left unsaid that you'd like to say?
[00:37:52.454] Betty Mohler: No, I just want to thank you. It's always so wonderful talking to you, and it's been wonderful here at the VR Conference seeing you talk to so many of the people that I love and think are great, and I'm really looking forward to hearing their podcasts, too.
[00:38:02.919] Kent Bye: OK, great. Well, thank you so much.
[00:38:04.480] Betty Mohler: Thank you.
[00:38:05.864] Kent Bye: So that was Betty Moeller of the Max Planck Institute for Biological Cybernetics, and she was talking about a paper that was just published called Appealing Female Avatars from 3D Body Scans, Perceptual Effects of Stylization. And so there's a number of different takeaways from this interview that, first of all, it's pretty interesting that they've come up with a semi-automated way to be able to morph somebody's avatar between an actual real life and a number of different art styles ranging from Marvel, Disney, Sony, Pixar, and Barbie. And in looking at the research paper that was published by Ruben Fleming and Betty Moeller, as well as a couple of other authors, It's interesting to see that most of the participants did want at least 33% stylization, and that in some of the styles, like in the Marvel and Disney, that the 66% actually went up, and in the other styles it actually started to decrease a little bit because it starts to get really extreme. I'd recommend taking a look at the actual design styles in the paper or on the post for this podcast to kind of get a sense of what the stylization looks like. And so it's not really all that surprising to me that people prefer to have some level of stylization because that kind of matches my own personal experience and what I've seen other people talk about in terms of needing to have at least some sort of stylized character within a VR experience. So I think this gets to a little bit of our expectations in that when we start to see someone that looks like a human, then we want to see everything that that person is doing behave in exactly the way that we would expect a normal human would behave, including different dynamic social interactions with that person if you're in the scene with them. And so I think there's just something about adding this level of stylized art to these avatars that just is a cue to our brain to be a little bit more forgiving in terms of what we're experiencing within the VR experience. And so I don't fully understand a lot of the math in terms of how they're able to do this, but in just reading through the paper and their conclusion, they claim that they've figured out some sort of automated way to be able to have an automated process to be able to add these different stylizations to your avatar. So I think it'd be really cool in the future to be able to start to use our actual body scans of ourselves and be able to start to adjust and tweak it based upon these different types of stylized art. At this point, the avatar selection within all of the different social VR applications is extremely limited. I think there's a huge space for a company to come up and to start to provide some sort of standardized solution for creating avatars that look great and stylized, but also are compatible amongst all the different platforms. And so it'll be interesting to see whether or not the different social VR applications are going to have a hold down in terms of that being a monetization process for being able to customize your avatar in different ways. It was also really interesting just to hear Betty talk about her own personal experience of having no problem with her body image, but yet going and seeing an augmented version of herself within VR and then coming out of VR and start to have a little bit of disappointment about her own sense of body image. And so, I think that's a little bit of a red flag in terms of just being careful in terms of what the implications of these stylized arts mean. And I think that taking a look and reading through the Code of Ethical Conduct for Virtual Reality, there's some recommendations for good scientific practice for the consumers of VR technology that Betty mentioned and is free online, it's linked in the podcast notes, and that's from Michael Madery and Thomas Metzinger. So just some things to look out for and you know on the positive side it sounds like Betty's pretty optimistic about the impact of how these virtual reality technologies could start to be used in a way to help people who have some sort of eating disorder to allow them to envision what their body might look like if they were to gain more weight, or perhaps move beyond the constraints of a 2D mirror and allow someone to really get a more accurate assessment of what their body actually looks like. And so it sounds like they're starting to perhaps start to answer some of the question whether or not it's a perceptual block that's happening when people have some sort of body dysmorphic disorder, or whether or not it's some other cause that is not perceptual. And perhaps there's additional insight that may be coming from stroke victims who don't have an accurate perception of their own body as well. And so that's it for this episode of the Voices of VR podcast, and if you do enjoy the podcast then please consider telling your friends, spread the word, and if you'd like to become a contributor to the podcast then please consider donating at patreon.com slash voicesofvr.
