On today’s episode, I talk with Dr. Benjamin Lok from the University of Florida about how they’re using Virtual Humans as patients to train medical students. He talks about the key components for creating a plausible training scenario which include both accurate medical symptom information, but also more importantly a robust personality and specific worldview. Humans hardly ever just transmit factual data, and so whether the patient says too much or not enough, the students have to be able to navigate a wide range of personalities in order to get the required information to help diagnose and treat the patient.
LISTEN TO THE VOICES OF VR PODCAST
Virtual humans help to embody symptoms that a human actor can’t display, assist in going through an extended interactive question and answer path, or they’re used within collaborative training scenarios where it becomes difficult to get all of the required expert collaborators into the same location at the same time.
Dr. Lok makes the point that creating virtual humans requires a vast amount of knowledge about the human condition and that it’s really a huge cross disciplinary effort, but one that is one of the most important fields of study since it has so much to teach us about what it means to be human.
Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to The Voices of VR Podcast. So on today's episode, I have Benjamin Locke. He's a professor of computer science at the University of Florida, and what he does is he works with virtual humans. So sometimes when you use training medical students, they need to be able to figure out whether or not they're able to ask all the right questions. And while training with other human actors to be able to stand in as a standardized patient is the optimal choice in most situations, Sometimes it's actually better to use virtual humans to both display symptoms that are impossible for an actor to play out, but also sometimes be able to dig into different nuances of a series of questions to really see whether or not the doctor is able to ask all the right questions. So we'll be talking about the use of virtual humans in training applications, as well as why it's important to come up with a character and backstory and personality for each of these characters. So, that's what we'll be covering on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. Today's episode is brought to you by my Patreon supporters. I just wanted to take a moment and just thank all of the supporters that I've had on Patreon over the last couple of years. The Patreon really is this huge thing that helps give me the psychological safety net to continue to do the Voices of VR podcast. And it enables me to really do the type of independent journalism, oral history, and education to really dive deep into what's happening with virtual reality and where it's all going. And so I just wanted to thank all of the contributors that I've had so far. And if you'd like to become a contributor, then please do go to patreon.com slash Voices of VR. So this interview with Benjamin happened at the IEEE VR academic conference that was happening in Greenville, South Carolina from March 19th to 23rd. So with that, let's go ahead and dive right in.
[00:02:10.683] Benjamin Lok: My name is Benjamin Locke. I'm a professor of computer science at the University of Florida. And the work we do is a lot in virtual humans. So in VR, you have virtual environments, but I'm very interested in the people that populate them and how they can affect you. So I've been in VR since 1997. So I've been doing it for about 19 years.
[00:02:27.708] Kent Bye: Great. And so what are some key components of creating a compelling or believable virtual human?
[00:02:33.513] Benjamin Lok: It really depends on who's going to see the virtual human. So we do, for example, a lot of work with medical students, nursing students. So we create virtual humans as like virtual patients for them to practice on. And so for them, the accuracy of the medical information is critical, right? So the believability of a character that, if they've interviewed patients before, is this character having a believable backstory. So almost as much a storytelling component as opposed to just the graphics. The graphics are important, don't get me wrong, but much more important is the medical accuracy, the personality of the character, the backstory. So it's more of a patient, because a patient is more than just somebody with a cold, right? They've got a history, they've got a family, a backstory, and it's all those things that need to be pulled together. It's a very storytelling approach to creating a virtual character.
[00:03:19.351] Kent Bye: That's interesting. Yeah, the thing that makes me think of is that you're trying to, in some ways, create this illusion of plausibility. In terms of telling those stories, I guess it seems like one of the goals would be that you're trying to create this sense of presence so that you are able to believe that this scene is real. And so, for you, what are some of the key components of presence in working with virtual humans?
[00:03:41.765] Benjamin Lok: So I'm going to answer that question, but I'll also kind of rewind it back. My goal isn't necessarily to create a sense of presence. My goal is that for nurses and medical students to see a learning potential. So they're going into a simulation, and they see a character that doesn't have a backstory, doesn't have a point of view of the world, doesn't have a belief system in whether Western medicine works, for example. That's not real for them. They don't see any value in this training simulation anymore. You see what I'm saying? If you're trying to say, hey, I want you to get better at interviewing patients by working with this virtual patient. And that virtual patient is nothing like a real patient, because real patients talk too much, or they don't talk enough, or they think you're an idiot, or they've got something in WebMD that they want to tell you all about, right? If they don't see those components in a system, they're like, well, what's this training me for? You see what I'm saying? So it's more than just create a sense of presence. It's so that they see validity to what they're going through, so they can say, OK, I can see, it's not perfect, of course, but the system has the components that are valuable enough so that I can get some learning out of this system. But what's important for, when we create a virtual character, what components are extremely important? Obviously, visuals are very important. And there, it's not to have, like, photorealistic visuals, but it's visuals of whatever symptoms you're trying to do. So if you're trying to show somebody with, let's say, cranial nerve palsy, which is where you have, let's say, a brain injury of some kind, it might manifest itself like one side of the face being unable to move. or an eye not being able to move. You have to get that right. It doesn't have to look photorealistic, but you better have a biological model behind your character that's showing you what it is. Voice is very important. Audio is very important. To have good quality talent that can record things is supposed to say, I'm taking two pills a day. It doesn't seem right. You have to have somebody who talks like how a patient would really speak. dialogue, the script behind it, so having somebody write something so that it doesn't sound like it's just reading from a script or an unrealistic story. So all those parts, it's hard to kind of put in place, but the whole point is that you need to think about when somebody sees these characters and they see they have a story, they have the right audio, they've got the right visuals, and they have a learning objective they're trying to get to, that's when they can buy in and go, okay, I understand this isn't visually the same as a real person, but I'm able to play along with that. But if you don't clear those bars, that can pull away from what they see the value of such a simulation.
[00:05:57.605] Kent Bye: So it sounds like you're trying to add a lot of the flaws or the kind of natural behaviors that humans have. Do you have like virtual humans that talk too much or not talk enough? Or what are some of these more human components that you're putting into virtual humans in order to make them feel more human?
[00:06:14.930] Benjamin Lok: You have to build it, and we're doing a lot of work in concept of personalities, right? So, verbosity, for example. How much does somebody, if you ask me, what medications are you on? I could either answer, I'm not taking anything. Or I could say, I'm not taking anything, because you know why I don't like to take anything? Because I hate how drugs make me feel. And you know what, it's just big pharma, just trying to get money out of my pocket. I'm not taking, I don't care what you send. You can imagine that there's, again, I gave that just as a wild example, but you need to have a wide range of responses. So your character must have a point of view, must have a perspective on life. It's more than just an audio file with some text associated. It needs to be written. So we often work with creative writers that will author a personality behind these characters. They could be scared. They could be nervous about what's coming up. They could use jokes to alleviate the concern. I mean, that's what real people do. Again, when you're trying to train somebody on how to, again, our major population are medical students and nursing students. We want to train them on how to talk to patients. These things are critical because they look at that and say, hey, I know that this is the way patients are, so if I can practice in this safe environment to fail, I can get better so that when I talk to real patients, they're there. So we have people that talk too much, don't reveal enough. Not revealing enough is actually a really important skill because you have to teach people how to dig deeper and get at more information. We can also take a look at things such as teaching people to be empathetic to concerns that somebody might have. If you suggest something and the virtual patient says, well, I don't know whether I can afford that. Okay, now how do you respond to that? That's not an easy situation to deal with. You'd love for people to practice those situations. So looking at socioeconomic status, looking at gender, ethnicity, weight, all these factors that can play into how medical students interpret and treat folks, we can build that into a virtual character and create really compelling simulations from that.
[00:07:56.091] Kent Bye: And so when you're having these interactions with virtual humans, then is there someone kind of behind the scenes that are kind of like dynamically feeding these scripted lines, or actors, or is it completely automated with sort of like a A, B, or C type of choice of dialogue that you can possibly say?
[00:08:13.668] Benjamin Lok: So with these virtual patient systems, people implement in different ways. Some very common ones you'd see is commercial systems or the ABC sort of multiple choice approach. There's a whole class of work that's the stuff that we do, which is using natural language processing, which means that people can type in questions and there's algorithms to match and database lookups that automatically return back to the user without any human in the background actually responding. And there are systems that use a human in the background, a lot of medical simulations They use mannequins, have an operator in the back. They call them Wizard of Oz systems. I'm sure you and your audience have heard that before. So all three of those things are very viable situations. I think it depends on how distributed you want these systems to be, right? So, for example, any of the first two systems, you can put it on the internet and hundreds if not thousands of people can use them. And that's definitely where we like to do a lot of my group's research is into that space of saying, we want the user to be able to type or speak naturally to the character and see whether we can figure out what they're trying to say and match what the character would say in response to that. But it's a very challenging problem and we don't get it right 100% of the time, but that's also somewhat like real people, right? Whenever you and I are talking, there's oftentimes where I think I got a point across, but because of a lot of different things, context, semantics, things like that, you might have a different understanding. They're actually very valuable for the learner to learn Using pronouns, for example, can be very confusing. Having compound sentences and trying to string multiple thoughts together can be oftentimes confusing, too. So helping to work on these communication skills, I think, are things that you can do in a simulation. That's what we try to work on.
[00:09:41.705] Kent Bye: And so someone going through one of these simulations, what would failure look like and what does success look like?
[00:09:48.166] Benjamin Lok: Excellent question. That is not so straightforward. So what does good and what does bad look like? That is a much deeper question than would appear at first surface. Because what does it mean to do a good interview of a patient? I mean, just think about that. So let's forget the virtual simulation completely. If you were a medical doctor and you had a patient come in there and you had asked them a set of questions, how do you know you did a good job? How do you know you got everything that you needed to get? I mean, there's no magic answer key out of them, right? So it's not like you, maybe if you ask that one extra question to unveil that one extra important bit of info, it's a very hard question and people don't really have a really great answer. They've got procedures that they ask you to follow, best practices, structures that they ask you to follow, but that's the best that they can do. And so what we try to do in simulation is to see whether they follow those structures as best as possible. So there are a lot of, common what they call mnemonics to help people remember what questions to ask. The extra cool thing about in VR and simulation is you actually do have the answer key because you programmed it, right? So you can say here is an optimal interview for a character that we created, patient X, wherever patient X is. You can actually create what an optimal interview created by subject matter experts saying if this person, let's say, has a headache, these are the questions I would expect people to ask. you can score them, this is what good looks like, this is what bad looks like. So simulation actually provides an environment that's better than reality because you actually have that quote-unquote the answer key per person and you can provide that back. And so that you're right in saying, providing the feedback, saying you interviewed somebody, here's all the opportunities that you missed out on. So like for example, if you ask somebody, do you have any allergies? And they say, Yeah, I'm allergic to cats. And then you go, okay, what medications are you taking? You've actually missed an opportunity there, which is what other allergies do you have? Or what medication allergies do you have? If you don't specifically ask those deeper questions, people might think, oh, you just have a cat allergy and you move on, but you might have a penicillin allergy or... or a latex allergy or some of these other, or drug allergies that you need to delve in deeper to get at. So with a simulation, you actually, since you know the answer key that this character might have those other allergies, you can actually score somebody and then give them that feedback saying, you missed out on this. This could be deadly if you get that wrong, right? And so it's a really great opportunity for people to make mistakes. I mean, it's very hard for, like if you and I are having a conversation and I make a mistake, I offend you in some way, It's hard to recover from that. In simulation, you actually get to practice that, avoid those loopholes, especially whenever you're talking about a patient and a doctor, and really improve communication skills.
[00:12:11.313] Kent Bye: Yeah, and it seems like, you know, a question would come up, it was like, well, why don't you just use a human to do that? And it seems like that these are so complicated backstories that it'd be kind of hard for an actor to really remember all those nuances of the symptoms and everything.
[00:12:26.487] Benjamin Lok: Well, so using actors, they call them standardized patients in the medical field, that's the gold standard today. And so let me be very clear, I think that they're fantastic. And they can do a lot of things. They can have a backstory. It takes them a little while. There's a lot of studies about how long it takes for them to create these backstories. And they're very good for a specific set of tasks. And we look at simulation to augment them. to do a lot of tasks that standardized patients and actors aren't good at. So I said earlier, let's say facial paralysis, to show that only one side of the face is paralyzed. I don't know of actors that can portray that very accurately, right? So that's where simulation come in and show abnormal findings, things you might not see every day. If you're trying to get somebody to demonstrate having a stomachache, I would rather you interview an actor. We can pay them. They do a really great job. That's fantastic. If you want, let's say, very challenging scenarios that are very hard to diagnose, or you want somebody to be able to practice at two in the morning, and they just want to get practice before they see the standardized patient, that's where simulation come in. So there's a place where simulation plays in. Just like I would make the analogy with, let's say, the aviation industry. The opportunity to go fly in an airplane with a trained pilot is just like practicing with a standardized patient. It's very, very valuable. But you still do time on a simulator on the ground before you spend time with him or her, right? And that's the same thing in communication. We can have you practice the simulation before you spend the valuable time with humans. Then when you go there, you will be better prepared and you can start tackling the higher order concepts as opposed to just the basic, hello, what brings you in today? What medications are you taking? The very rote stuff that you can practice on a computer. You don't need to practice with a human.
[00:14:02.998] Kent Bye: Now, one of the big issues that comes up with working with virtual humans is the uncanny valley and dealing with the uncanniness that happens with people that you kind of know are not embodied and giving all the body language or social cues. And so how do you address some of the issues of the uncanny valley?
[00:14:18.708] Benjamin Lok: I think that's an open area of research right now. Whether we're there, and if we are, is it better to maybe have a cartoonish character as opposed to a very realistic character? What I would say though, it all depends on your learning objectives. I always kind of go back to that because I'm a very application-based researcher. I really am very centered about what the user needs. What are you trying to teach them? If you're trying to teach medical students, for example, how to recognize a cranial nerve 3 palsy, you need all the fidelity around the facial eye movement, the facial paralysis, or the eye movement to follow the finger movement, the way the eyes dilate, pupils dilate, and everything. You need that level of fidelity. If you're doing something else, maybe that stuff doesn't matter. You might then want to do more, let's say, a cartoon render so that they're not too focused on, why isn't the character the same as a real human, right? So, I agree with you. On Candy Valley, especially whenever you're talking about humans and what people expect, that's really, really important in terms of knowing what their learning objectives are, but also setting the stage and letting people know. I think we're dealing with a lot of users that are very familiar with video games. They've seen high-quality characters. They understand a lot of how the technology works. So they're not necessarily fooled so much. And they just want to know, how is this going to help me learn something? So aligning their expectations with their learning objectives, I think, can help us avoid uncanny valleys. So don't pay attention to that stuff, because it's not really that critical for what you're trying to learn.
[00:15:37.077] Kent Bye: So there was a workshop on virtual humans and crowds leading up to the IEEE VR. And you had a chance to speak there. And I'm just curious what you were sharing in the context of that workshop.
[00:15:46.664] Benjamin Lok: So I think it's really interesting whenever people interact with virtual humans. I think it's interesting whenever we have one virtual human and now we're starting to think we've got the computing power to do small teams of virtual humans, large teams or thousands of people, and seeing how humans interact with other large groups of virtual humans. So I find that very, very interesting. So the work that I do a lot is in team training, which is where we have small teams, like in an operating room, where you might have a team of three or five people working on you as a patient. And I wanted to see whether this simulation can help folks learn team dynamics. Because a lot of the issues that happen in the operating room that cause, let's say, morbidity or mortality is because of communication skills. It isn't actually a knowledge problem. It isn't like your doctor doesn't know how to do a procedure and that's why bad things happen. It's because they said something, somebody else heard something else. And then there's a problem from that, because something wrong gets applied to, maybe a wrong drug gets administered, and things like that. And so, what I presented in that talk was saying, hey, what happens if we could use virtual reality to train teams of people to work better together? And I say, what does better mean? That means that they will have better communication skills, so they repeat back phrases to each other, so they make sure everybody hears what they're saying, but also standing up. If somebody's doing something that's potentially patient-dangerous, will you enable somebody, let's say, low on the authority totem pole to speak up and say, that's wrong? It sounds great to say everybody's in charge of patient safety, but the reality is it's hard for somebody that's not in a high position of authority to say, stop, we're doing the wrong thing. Can we practice that in VR? Can we say, hey, let's empower people by telling them how to say the right phrases, letting them see what happens if they do that, watch them push through a situation that would be very, very socially challenging and letting them get empowered. So we've done a lot of work in small teams of people and virtual humans to empower folks that are, again, low on the authority gradient to be able to speak up for patient safety. I think that that's a huge new area about how do we work on teamwork. And you have these virtual humans as our teammates, and that's going to allow us to really do a lot of training that we can't do right now. Because you can't train, if you needed eight people to do a training session, getting eight people together, with all their schedules, that's not going to happen, right? So you might know what to do, but in the big thing, there's all these social pressures, it's really hard to stand up to folks. And that's what we want to do.
[00:18:01.509] Kent Bye: And so in these virtual teams, how many of them are actual humans and how many are kind of virtual humans?
[00:18:06.993] Benjamin Lok: So we've done a variety. Most of the time we have one virtual patient and we can have two to three virtual teammates and one to two real humans. So typically, again, you might have one human and three virtual humans, two humans and two virtual humans, and we sort of see what the social dynamics are. At first we thought it'd be cool to have any sort of plug and play of virtual humans and humans. What we're starting to see is actually really nice to standardize everything and have one human and three virtual humans, for example, as part of a team. So you might have a a nurse, a surgeon, anesthesiologist, and a surgical technician, and a virtual patient. That would be a team, one group of folks. And actually having one human and a bunch of virtual humans that are standardized that can give the same learning experience for everybody is very, very powerful.
[00:18:46.805] Kent Bye: And then are you taking natural language input with these virtual humans in that context then?
[00:18:51.330] Benjamin Lok: Well in this context it's a very noisy environment because we actually do this in a real operating room at hospitals. So there we actually do a Wizard of Oz technique where we have somebody behind the curtain kind of controlling the characters. Though we do record voices and we're constantly trying out the latest in natural language processing and speech recognition and things like that. But right now it's such a noisy environment and there's so many people kind of talking over each other. I'm very excited when, and I know we will soon be able to support things like that, but right now we do a Wizard of Oz approach.
[00:19:19.403] Kent Bye: And so how do you measure the efficacy of this type of training then?
[00:19:23.384] Benjamin Lok: Well, I can tell you that what we can measure is in the short term, which means inside simulation. Can we see whether people will behave differently in simulation? Of course, the gold standard would be, if I can change your behavior in simulation, does that mean you change your behavior in the real world? And then even better, patients benefit from that, right? Now those last two stages, this is exactly what we're working on now. The first stage is what we would say is, well, we can try to train you on something at the beginning of the simulation. Then later on, we can have an evaluative stage where we see whether you apply what you've seen or learned before. So we can see the impact of simulation training within that simulation. But then, again, do you carry it outside the training? That we don't know. That's going to take a long time. But we've got commitment from, again, the University of Florida. And so you have health. The hospitals there have gotten the commitment to say, hey, we want to see whether this will go forward. So we're able to start seeing whether we can see behavior changes moving forward. But that's a hard, hard question.
[00:20:14.892] Kent Bye: And it seems like there's a certain level of a suspension of disbelief within the people who are in these situations with virtual humans. They're kind of just accepting that these are humans. And I would imagine that that might be measured through like social presence, what we could call social presence. And so I'm curious how you think about social presence, how you define it, and then how you measure it in these type of situations.
[00:20:36.294] Benjamin Lok: That's a good point. Yeah, we do use social presence questionnaires, for example. We've done physiological measures to try to see how much people rate these characters as believable. For us, what we've learned over time is we've taken these scenarios that we train people on from real-world scenarios. That's super critical. We have them in very compelling environments, because these are old operating rooms that they're familiar with. They have all the same equipment in there. All the sounds are right. And they've been told by their administration that this is very important. Take this training seriously. You get training credit for it. So the stage is set properly. Before they ever see any of these characters, irregardless of how realistic they look, things are set up right for them to come in there and suspend disbelief. So, most people come in and they're not fooled by what they're seeing. They get the point of what we're trying to do. But they understand that there is a learning benefit from this because the administration told them, hey, take this seriously. We see value in this. They're familiar with the scenarios because their colleagues get in trouble for not doing well in the situation. And the environment is set up so that they know how to use it because that's an operating room. That's where they work. So all these things play well into a place where whenever we do ask social presence questionnaires, they tend to do quite well, right? Because everybody says, yeah, these are very realistic characters. They're very realistic scenarios. The situation is believable that you're pushing me through. So it isn't a large leap for them to suspend, right? To say, hey, I want to play along. Because also, remember, they have identified a benefit for playing along. The benefit is they're going to get some important training. For a lot of these folks, we're trying to empower them. And that's a pretty easy sell for folks. It's, hey, you want to go through a scenario that's going to teach you how to be empowered, how to speak up at the right moments to authority? A lot of folks will say, yeah, yeah, I want to know how to do that. And so you've kind of primed everything to be in a good space.
[00:22:24.701] Kent Bye: And so what are some of the big open questions that you see around virtual humans that's really driving your research forward?
[00:22:32.486] Benjamin Lok: Well, we work a lot on how do you build virtual characters very, very quickly. So I think that building virtual humans right now takes months, if not years, just to build even one or two characters. It's very, very hard to do. And so we're trying to think about ways to try to make building them a lot quicker. It's something you can do maybe in a day. That would be fantastic. So how do you use different technologies such as machine learning or natural language processing and AI to really make building new virtual people a lot shorter? Interactions. Right now the modality you talk to these characters is mainly through speech. Nothing about gestures, nothing about eyebrow motions. Like right now I can make a goofy looking face and you'd... You'd be like, what's going on? Even though the content of my words could be something, even small changes in the face can have a different meaning. So we don't have that with virtual humans, right? Either the display or the tracking. So I know that the way somebody might be trying to talk to our characters. And then gestures. Most of our characters don't take in gestures. So really, the pipeline of communication with these characters is still so tiny compared to what it is between two real humans. And I think that that's really a lot of excitement is how can we do more research to expand the pipeline so characters can respond to somebody doing wild gestures or speaking in a certain way or using the full channels of communication that humans have with each other.
[00:23:46.198] Kent Bye: And finally, what do you see as kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:23:51.942] Benjamin Lok: So I'm going to answer this through the lens of virtual humans. And for me, virtual humans represent a new way to train each one of us to be better with each other. So right now, if I want to be good at flying an airplane, I get a simulator. I want to get good at Drive my car. I can get in a simulator before I get into a real car. Drive a tank. Lots of simulations out there. But one thing we don't have a simulation for, which is something that each one of us does almost every single day, is talk to somebody else. And that's a critical, critical thing, and we don't have no simulations for that. And I think that's where, for me, the ultimate virtual reality is a future where people are trained to be better with other people, whether we're talking about police officers, or the military, or educators, or business folks, or medical folks, because they got trained on best practices inside a simulation first. So there are so many high-stakes interactions between two people and there's no way to practice that. And I think that in the future we're going to look back and say, what? You didn't practice that? That's insane. We would say that about airplanes, right? You let somebody fly without practicing? That's silly. Why do we let high-stakes situations happen between two people or more people? without any sort of simulation, without any feedback, without a place for you to fail and get better at it. So that's to me what I think the future of virtual reality, especially virtual humans, is. It's going to be a place where in the future people are going to have that opportunity to explore, to fail, to learn new approaches, to be better at talking to somebody else.
[00:25:19.763] Kent Bye: Is there anything else that's left unsaid that you'd like to say?
[00:25:22.644] Benjamin Lok: I want to inspire folks to explore this area. I mean, the concept of virtual humans is very, very broad. We're just starting in that space. There's so many unknown questions. But I think that there's some of the most interesting questions in virtual reality because it speaks to the core of us. We are social creatures. Humans are born to interact with other humans. And that's what I think makes virtual humans the ultimate virtual reality. because that's what we're born to do. And so I would just kind of hopefully inspire people in communications, in psychology, computer science, anthropology. Like, this needs all your expertise to come work together in this field. So it's anything. It's one field that borrows. It needs so many people from all the other sciences to come in and help us really build these compelling systems.
[00:26:02.730] Kent Bye: Awesome. Well, thank you so much, Benjamin.
[00:26:04.052] Benjamin Lok: All right. I appreciate it.
[00:26:05.132] Kent Bye: Thank you very much. So that was Benjamin Locke. He's a professor of computer science at the University of Florida, and he's been working with virtual humans for training applications. So I have a number of different takeaways about this interview is that, first of all, one of the things that was the most striking for me in this interview was to really hear how plausibility is really quite context dependent, because we talk about video games and the Uncanny Valley and plausibility and what's it take to create these realistic different types of characters. And I think it really depends on the context to whether or not the person who's experiencing the VR experience thinks that the situation is plausible. In the case for these training applications, Benjamin was really making the point that the thing that's really the most critical for creating believable virtual patients is that they have both a realistic and reliable medical symptoms, but also have some sort of personality around it, that it's not just a transmission of facts when you're dealing with humans. there's different personality types that are either going to not give enough information or give too much information and share a lot about their worldview of how they see the world and so that'll change how a western doctor may interact with somebody who may be completely skeptical about the western medicine as well as the pharmaceutical companies and so they have to take that into account but Interestingly, the thing that Benjamin said was that the thing that is really interesting are the people who don't give enough information because it really creates this opportunity for the virtual humans to be able to provide some sort of standardized objective measure to be able to see whether or not a doctor is able to ask the right questions to evoke the information that they need. So they can start to have an experience where they go and interact with this virtual human AI slash set of dialogue scripts that have been kind of pre-recorded in a certain way, and they can start to see whether or not they were able to get the information or not. Now, Benjamin did say that the ultimate gold standard in training is going to be the human, because they're able to add a lot of nuance and personality and character that just makes it really believable. But in certain situations where that's not as helpful, for one, there's going to be physical symptoms that are on the face or body that is going to be impossible for an actor to really reproduce. Team training situations where it actually logistically may be very difficult to get everybody at the same place at the same time, and so starting to use virtual humans in that context But also some of these scenarios where they need to give a patient where it's not giving enough information, they really have to ask all the right questions at the right moment. And they'll be able to take that and then show them opportunities where they could really improve. So it's a little bit more standardized in that way. So from Benjamin's perspective, presence isn't necessarily the end goal for what they're doing, but they're just trying to create something that is plausible. And so one dimension of the uncanny valley that's important here is that if they're trying to diagnose something that is very physical and you actually do need to see a photorealistic type of rendering, then in that case, they actually do need to create virtual humans that are a lot more photorealistic and they're not necessarily going for a more cartoony stylized approach if it's actually not going to be serving the plausibility of the specific use case and training scenario that they're trying to create. I think that was an important distinction. Also very interesting to hear that they've been employing different creative writers to be able to really flesh out the personalities of these different characters and to give them a backstory and to try to model their personalities in different ways. And so to me, that's really interesting to see how these taxonomies of different personality types are expressed within these AI characters and in some ways, And if you look at these virtual humans as this very specific use case of training medical students, but some of the principles that Benjamin is talking about with virtual humans, I think are going to be applied to some of the new virtual assistants that are coming out with either Amazon Alexa or Google's Assistant and other big major companies that are going to be coming out with these different chatbots. Right now your primary interface for a lot of these different chatbots is through text and natural language input but eventually we are going to be able to interact with some of these chatbots in these virtual and augmented reality spaces. So one of the things that Benjamin was saying was how can you start to communicate non-verbally and you know, if a patient is giving you kind of a look like, hmm, like WTF, like, that's something that is going to change the entire interaction. So there's a whole layer of nonverbal communication that is going to be happening, and personalities as well, that is going to be kind of infused within these different characters. And so While right now it's a very kind of detached interactions that we're having with these different AI characters, eventually it is going to be moving more and more towards like natural language input. And we want to have these AI assistants be more human-like in different ways. And so I think that in the future, there is going to be a lot more embodiment of these AI assistants and they are going to be looking at ways to communicate information that goes beyond just that kind of content and the facts that's being transmitted. So that's all that I have for today. I wanted to just thank you for joining me here on the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends and become a donor at patreon.com slash Voices of VR.
