Nate Newsome is a Ph.D. student in Human-Centered Computing at Clemson University. He’s researching Interactive Virtual Agents that have some intelligence where you can interact it through voice interaction and some AI involved. It’s used for medical simulations and training. He had a poster at IEEE VR titled “Comparative Evaluation of Stylized versus Realistic Representation of Virtual Humans on Empathetic Responses in Simulated Interpersonal Experiences”
He talks about some of the rapid response systems used to train nurses to detect signs of rapid deterioration. It’s an interactive character, and they can respond to different questions that you ask it. It’s used to help train nurses catch signs of rapid deterioration faster.
Nate talks a bit about the split between academic and consumer VR. There’s a lot of things that are happening within the academic virtual reality community, and that a lot of that isn’t being reported on within most of the consumer VR news outlets. Nate was happy to see the Voices of VR covering at the IEEE VR conference.
He sees that the consumer VR community isn’t operating on full information when it comes to saying what is and is not possible in virtual reality. In particular, academic researchers more expensive VR equipment and tracking equipment that are able to create really immersive experiences that go beyond what consumer VR has been able to produce so far.
Nate talks about the future of designing 3D interfaces with immersive technologies and what will happen when you put the human at the center of the interactions. Finally, he’s the most interested in training applications where there are unsafe conditions. VR can replicate the feelings of being unsafe without actually putting you in danger, and being able to do that will help train people to actually perform better in those real conditions.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:12.215] Nate Newsome: I'm Nate Newsome, and I'm currently a first-year human-centered computing PhD student at Clemson University. I got interested in VR a little while back when I heard about this program. Some of the work I do is I work with virtual humans and working with interactive virtual agents.
[00:00:32.087] Kent Bye: What is an interactive virtual agent?
[00:00:34.089] Nate Newsome: Sort of like a video game character, I guess you could say, but one that has some intelligence so that it can answer questions about things and actually get more information from you rather than just clicking questions. It can be anything from voice interaction to as simple as just some character that has a presence in the room and can interact with people on a little bit more in-depth level than a normal video game character.
[00:01:04.369] Kent Bye: So I guess in the video game world, they would call that a non-player controlled character or an NPC.
[00:01:09.933] Nate Newsome: Yeah, it has more of an AI, I guess, to it. And can be used for multiple purposes. We use a lot of ours for some medical training simulations. And our poster here is actually using one of our medical simulations. And all the characters are based off of actual patients, so there's a lot more depth to them than a video game character.
[00:01:30.916] Kent Bye: I see, and so this is like a medical training simulation, and then is it within a virtual reality context? Okay, so maybe talk about how do you have real characters and virtual characters sort of intermixed within this.
[00:01:45.399] Nate Newsome: Well the point of that particular simulation, it's called a rapid response system and it's actually been, we've had a number of papers on it here at VR, but it was designed to help train nurses to detect signs of rapid deterioration in patients. So the current standard for that is using like a $100,000 dummy that you can actually take vital signs from and things like that. But our system allows you to kind of interact with them on a more personal level as well. So you can still go through all the steps of taking their vitals and things like that, but you can, because we have an interactive character, you can actually ask them questions, kind of get an idea of their mental state. And then they can tell you more about how they're feeling and what symptoms they're actually having. So it helps a lot to be able to pick up on some of those things, especially for picking up on signs of rapid deterioration, which, yes, can be picked up through vitals. But if nurses are trained to be able to pick up on signs of mental deterioration and things like that, it helps them a lot to be able to catch that faster.
[00:02:49.429] Kent Bye: Yeah, and so you were mentioning that sometimes there seems to be this gap between the academic world of virtual reality and the consumer VR. And in talking to Eric Hodgson, he's mentioned it as sort of calling it the old VR and the new VR, where they're really like these two separate communities that are kind of operating in parallel, but not always a lot of interaction between the two. So from your perspective of coming from the more academic side of VR, I'm just curious of your perspective of how you see these two ecosystems.
[00:03:19.528] Nate Newsome: Yeah, I mean, I tend to follow a lot of blogs and stuff just to keep up on what's going on with VR on the consumer level. Every time I do, I always feel like I'm missing a little bit more, especially since I come to these conferences and things and see a lot of what academia is doing, particularly their research. And a lot of that information isn't coming out in things like Game Developer Conference, things like that, from the consumer products. So a lot of what people are reporting about VR isn't necessarily the whole picture, I guess, of what's actually happening right now in the state of VR. So it's nice to see somebody finally actually coming to some of the academia side of things and trying to get more of the full picture for the public.
[00:04:04.948] Kent Bye: And so what are some of those things that you see that are missing that, you know, you think people should be learning about beyond just sort of the gaming or entertainment applications?
[00:04:13.379] Nate Newsome: Well, a lot of what kind of triggered me to it was that everyone's saying, well, you can do this, but you can't do this in VR yet. But they're not necessarily working on full information there, especially if you were to see what we're doing in research and the academic side of things. There's a lot of possibilities that we can do that they're not really showing. So VR isn't restricted to the Oculus with their tracking system or even Valve's new tracking system or anything. There are other possibilities out there and other things that are actively being worked on that people aren't hearing about.
[00:04:48.369] Kent Bye: Well, I think part of the gulf is that, you know, you look at a system like a cave and the price point of that, of a fully immersive room that has this projection of a VR environment where you're able to walk around in and have a sense of embodied presence because it's your body and you have glasses that are able to put you into this virtual space. But it's sort of beyond the price point of any reasonable consumer. And so we've, I guess, kind of just ignored it. What do you see as the value of some of these higher-end systems that may be out of the range of people actually having access to it, but what's possible with some of those?
[00:05:25.644] Nate Newsome: Yeah, I think particularly the bigger tracking systems and things that we have really allow for a lot more immersive experience, but being able to walk around in a big track space and things like that are really helpful. But I think there's still a lot of potential with the cheaper side of things, I guess, or with the more consumer version. We do a lot of work with strictly consumer products as well in our lab, and particularly a lot of training simulations that are geared towards that consumer, especially with a lot of tech schools and things like that, doing some training simulations for that.
[00:05:57.814] Kent Bye: And so the human control interface, HCI, or here at the IEEE, there's the whole 3D user interface, sort of the 3D UI section or separate conference that's happening concurrently with IEEE VR. But coming from that human control interaction and 3D UI, what type of problems are you looking at or insights do you have in terms of what the space is and what you're trying to do?
[00:06:21.089] Nate Newsome: You know, for a long time, software design hasn't been strictly focused on the human side of things. And that's not true in all cases, but generally, it hasn't been focused on making things intuitive and actually giving people what they really need. And so now VR is kind of allowing us to rethink how we're designing interfaces, particularly now that we have the whole three-dimensional space to work with. So I think there's going to be some interesting changes to the way user interfaces are designed. I think if the way we work is working from a human-centered perspective, so I think it can yield some pretty exciting results.
[00:06:59.131] Kent Bye: And finally, what do you see as sort of the ultimate potential for VR and what it might be able to enable?
[00:07:05.831] Nate Newsome: That's a big question. I think the most exciting thing for me so far has been training applications, particularly for things that could be dangerous, being able to put people in an actual immersive environment where there's simulated danger that still gives them those feelings of being in danger, but being able to train in a safe environment and still having that experience to work with.
[00:07:31.386] Kent Bye: Great. Well, thank you so much. Yeah, thank you. And thank you for listening! If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash voicesofvr.
