Trond Nilsen studied with Tom Furness at The University of Washington and currently works with him the RATLab LLC. He has a Ph.D. from University of Washington, which focused on building educational games that teach human anatomy. He moderated a couple of panels at the Seattle Virtual Reality Expo focusing on education in VR as well as medicine in VR. He talks about the experts featured on each panel as well as the some of the most memorable insights. There is a lot of huge potential for education and VR, as well as a lot of difficult challenges for navigating the privacy and regulatory controls that healthcare VR applications will be facing. Trond also talks about the VR Hackathons, the Virtual World Society, and his passion of world building in VR.
LISTEN TO THE VOICES OF VR PODCAST
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:12.034] Trond Nilsen: I'm Trond. I work on a variety of projects, so I find it very hard to categorize myself. I work with Tom Furness at The Rat Lab, which is a private technical services firm. We do a variety of software and hardware engineering work with various startups. We have a medical device firm, a scientific instrument firm, and now we're also working with Envelop VR on some projects with them. I have a PhD in Industrial Engineering from the University of Washington which I finished not too long ago which was focused on building educational games, intelligent educational games for teaching human anatomy on the web. So I've been doing a lot of work in the 3D visualisation space for human anatomy and then before that I was at the Human Interface Technology Lab in New Zealand from about 2002-2003 where I was working on tabletop strategy games and command and control systems. I like to tell people that my claim to fame is that I built the world's first tabletop real-time strategy game.
[00:01:04.199] Kent Bye: Nice, and so you got your fingers in a lot of different areas and we're helping lead a couple of different panels here at CVR. Maybe you could talk a bit about what those panels were and what type of things you were talking about.
[00:01:14.084] Trond Nilsen: Sure. Yeah, so I've hosted two panels today. One was on virtual reality and education, which was earlier this morning. We had three panelists there who occupied a fairly broad range of perspectives on that. We had one of the very first people who ever worked on virtual reality education back in the late 80s, early 90s, doing work where she was Putting people into a virtual world where they were shrunk down to the size of atoms and molecules and would, you know, work to create these things and part of a science curriculum. We had a woman, Lisa Castaneda, who's the founder of a company, Foundry 10, who are working with high schools to put VR systems into the hands of students, working on projects and getting into the the nitty gritty of how do you actually get these systems into classrooms and past all of those complicated practical boundaries that you have to do. And then finally, Erica Feldman, who's a, she's a freelance researcher working on the cognitive models of early childhood learning and how that ties into virtual reality. And then we just now finished the medical panel, which was a really fantastically interesting panel. I had Ari Hollander, who's the founder and CTO of DeepStream VR, who do what they call cyber therapy, which is using virtual reality environments to help people Adjust their mental state. So this is everything from treating pain so what the one of the things they do is they have the snow world environment and several other environments that will Distract the mind and reduce pain and it's not just sort of you know, you think you're feeling this pain It is fundamentally less pain and they've proven this by doing an fMRI studies of people in these things while being subjected to artificial pain like literally they stick a pain stimulator on the person's foot and and then they put them in the game environment, have them play the game and look at their brain, and their brain is simply firing less pain as they're doing this because they're in VR, which is just fantastic. The level of difference they're getting for this is like the same level as morphine. It's not just a slight effect, it's a massive effect. So we had Ari doing really cool stuff, we had Mike Arato who's the CEO of vRecover who's doing physical therapy using virtual reality, he's a long time collaborator in this space, he's the chair of the medical working group of the Web3D consortium, so he's been in this space for quite a while. And then Andy Wright, who's a doctor, he's a gastroenterologist here in Seattle at the Northwest Hospital. He works with the group, unfortunately called ISIS, which is the Institute for Simulation and Interprofessional Studies, and they do surgical simulations and other such things. He's worked in the space of how do you teach surgical skills, not just surgical skills, but teamwork and training and situational awareness and all of these complicated things that surgeons need to be able to do to work effectively in the hospital. they were talking about a whole raft of issues. Medicine's really challenging for VR, because we all sort of have this instinct that there must be an application for VR in this space, but we don't really know what that application is, partly because it's such a huge field, right? There's many, many activities going on, but partly because it's closed to most people, right? Doctors are like priests. They know the secret language of how this works, and the rest of us, we go to see the doctor, and we're a bit scared when we do so. So there's a lot of really interesting stuff in that space that needs to be talked about.
[00:04:10.725] Kent Bye: What were some of the more striking points that were made in, say, the education panel this morning?
[00:04:15.928] Trond Nilsen: One of the things I found really interesting was Lisa's group has been doing this thing where they have been taking VR systems and putting them out into classrooms and having students work on projects with them. And one of the sort of reactions that they get back from the students is, sure, VR is cool, and they're very excited about it when they first play with it, because who isn't? But these students clearly are more prescient, I think, than many, many people you talk to, but they play for a while in a classroom, but eventually they're like, well, this is cool and all, but how is this gonna look in a year? Am I gonna be just using this like the computer in the corner of the classroom, which I'm allowed to search things on the internet, but I can't really do anything fun on? Is VR gonna be sort of tainted in that same kind of way in schools, or is it gonna be something cool? So there's a lot of interesting enthusiasm in that space, but it's interesting that the students are the ones noticing that sort of thing, which I thought that was really, really quite cool. One thing that was interesting, I asked our panelists to make their predictions about where VR would be with respect to education over the next 10 years. Absolutely completely different answers. One person thought things wouldn't change much at all, other person thought everything would be fundamentally different. One point that I thought was really quite interesting is the idea that with virtual reality, not only do you take the students from the classroom to Rome or Verona or wherever you want to take them for your field trip, You also allow them to take the classroom home with them. So you can see how virtual reality would allow parents to be part of what their students are learning in a way that they can't do otherwise. So you can sort of imagine taking the classroom home, dad can look in on the world I'm building, you can see how because more education is happening in these virtual environments, if we're talking about VR not just as a go in a single application and I start up my application, I go to Rome, I come back and I close it down but more as a a platform or an environment, you can then start to think about how would you instrument this platform and environment to observe what the students are doing, what they're learning and how would you feed that back to adjust what they're doing next or to assess them, assessment in a way that doesn't involve exams, that sort of thing.
[00:06:10.562] Kent Bye: And in terms of the medical panel, what were some of the more striking points that were made there?
[00:06:14.928] Trond Nilsen: Yeah, so the medical panel, we had a really nice optimistic time because there's so much that can be done in this space. I have a list here that I'll try and pull out of the possible areas that VR can be used because Go back to the education part. If you think VR and education, education is, you know, it varies in terms of content, it varies in terms of the types of students, but fundamentally virtual reality and education, you're doing the same kind of thing. You're teaching in almost all of what you're doing. In medicine, there are many, many other things in addition to training. You have to teach knowledge, like anatomy, physiology, those sorts of things. You have to teach skills. You have to teach soft social things that are not, we're not talking about the practice of surgery, like how to sew something or anything like that, but the complicated things of how do you recover from errors? How do you communicate properly with nurses? How do you work in this environment? So that's one aspect. Surgical simulation, virtual reality treatments and diagnosis, so two entirely separate fields all there. VR for behavioral change, VR for visualization and diagnosis. So it's such a broad field. So when someone says, oh yeah, VR and medicine, that's gonna be a thing, that's like saying, VR is going to be a thing in everything because medicine is such a broad field. So that was really fascinating. A lot of sort of ideas for where it could go and what's going on. But then of course, reality came home to roost and we started talking about regulatory hurdles, which are complicated and fascinating. So one of the presenters was talking about a project they had done in the last couple of years. using the Google Glass as an information assistant for surgeons in the hospital. And he was relating that to get this thing, actually just even to get permission to have this thing in the hospital, took nine months of fighting because of HIPAA compliance. And even there, they couldn't do it. And the reason is that the service Google uses for Google Glass, they're not HIPAA compliant. And HIPAA is one of the many laws you have to pay attention to. It's the one that is, I think it's Health Information Privacy Act, something like that. And that's one. Another one is if you want to use these for anything diagnostic or anything treatment related, anything where screwing up could lead to someone getting hurt, you have to get FDA approval. And that's why the pharmaceutical industry is so huge, right? FDA approval is really complicated to get. The testing there has to go through that so hard. So there's a lot of complexities and hurdles there. There was one other thing. Oh, the tolerances. That if you're doing these sorts of things where you're teaching surgical skills, if I'm teaching you sort of general skills, like just, you know, good reaction time, hand-eye coordination, the sorts of things that you might learn from playing an educational game that's not targeted to a particular thing. you can get away with an awful lot of error margin so it doesn't have to be perfect, you don't have to have an exact replication of what's going on there. But if you imagine a flight simulator, you're using a flight simulator to teach someone what it feels like to fly say a 747 and just imagine you tweak the gravity on that to 95% of normal, well the instincts you're going to develop as learning in that flight simulator are going to be completely wrong and you're going to crash into a mountain or something. Same applies for surgery. If you simulate things wrong, or you set up the structures so that they're wrong, or the tools work differently, the speed and the precision with which surgeons have to do things, you get these skills wrong, and you teach them, you ingrain it into them wrong, and then they come to do this on a real person, that way leads to disaster. So yeah, there's a lot of interesting complexities in there.
[00:09:23.583] Kent Bye: And so for you, what's next in terms of what you're going to be personally working on in the field of virtual reality?
[00:09:28.710] Trond Nilsen: Oh well, so I'm very much interested in the virtual reality hackathon scene. That's kind of one of my pet projects as I've been involved in the two VR hackathons that we've run here in Seattle and I'll be involved in the next one that we're running come spring. I'm also working to start running what I'm calling mini hackathons which are basically just hacker weekend type things so it's not a big event with prizes and things like that, it's more just hey come on over, work on a project, use our gear type stuff. So that's one thing I'm heavily involved in, and we've now run virtual reality hackathons, the same sort of group, in San Francisco and Seattle, and we're wanting to sort of expand this so that when people want to run a hackathon somewhere else in the country, that we can work with them to help them do that. So, you know, they don't have to come up with their own collateral or website and things like that. We can just use ours, have a thing in the space, and we can guide them towards sponsors and just generally do this. And, you know, this is partly about building a community, not just locally, but also nationally. and not internationally, actually. We are currently working with two groups, one in Belgium and one in Cameroon in Africa, both of whom want to run virtual reality hackathons, and we'd love to see more of these happening all over the place. So that's one thing. I'm also heavily involved in the Virtual World Society, which is a non-profit organization that Tom Finesse is in the process of organizing and setting up. He'll probably talk about that on his own. The goal there is to foster community, foster collaboration and exchanges of information, and then co-opt students and people at home to build virtual world, not just for fun, but also to create solutions to world problems. So a lot of the sort of things that we talk about here are virtual worlds for messaging, so using a virtual world to communicate what it's like to be living in a refugee camp just outside the borders of Syria, or having people understand what health situations are like in Africa, or having people learn about something, or any number of these sorts of things. So it's using virtual reality as a means to address world problems. we want to not really be making these things ourselves and selling them. We would rather build a community of people contributing things of their own and then sharing them. So we sort of, to some extent, use the template of the National Geographic Society, which in its early days was literally a society of explorers who would go out and come back and they'd report on what they'd done and show the worlds that they had uncovered to their readers. We want to do the same thing with virtual worlds, where people who are making worlds or exploring worlds go out, bring those back and tell about them to the readers. So We're in our really early stages, we are setting up the non-profit at the moment and all of that. But we'll be doing more, we're hoping to announce the first round of awards, we're going to be doing sort of a prize-giving ceremony hopefully at the Augmented World Expo next year, and we'll be doing that every year after, it's going to be the next Interwards, and we're hoping they'll go somewhere and become sort of a Golden Globe-style level of prestige. What else am I working on? I'm also finally working with Envelop VR and Tom Finesse on some assorted projects demonstrating their platform. So we're fingers in many pies.
[00:12:22.862] Kent Bye: What type of experiences do you want to have in virtual reality?
[00:12:26.422] Trond Nilsen: I'm a world builder fundamentally, I like, and when I say a world builder I'm not saying I'm interested in the graphical side of things or anything, I like simulating complex systems, by which I mean social systems, or geographical systems, so I have always been fascinated with virtual worlds that are living, breathing places, not just places where I go and there's a whole part of static things that I can interact with and maybe another person who's in there, but everything else is just, you know, like a character in World of Warcraft standing there with an exclamation point above his head. I want to create living, breathing worlds where there's characters moving in those places who live there, and this is their reality. This is, of course, far-fetched and things, but I think working on those sorts of things and creating that is just absolutely fascinating. I think that's what virtual reality does for me, is it allows me to be a creator in a way that I couldn't otherwise. As far as just experiences, I think, obviously, there's the normal, I want to see places I can't see. I want to experience the universe as if I could fly around and through a nebula, that level of those sorts of things. But fundamentally, it's the creation part that really drives me, I think.
[00:13:29.655] Kent Bye: And finally, what do you see as the ultimate potential of virtual reality and what it might be able to enable?
[00:13:36.218] Trond Nilsen: The ultimate potential of virtual reality. I am going to take the line from what Philip was saying earlier in the keynote, which was this whole idea that it allows us to become the gods of our own little universes. And everyone who's ever read sci-fi or fantasy or any of those sorts of alternate world type things, they start imagining, oh my god, these worlds are so cool and there's so many of them. I want to go live in this one for an hour. And I feel like that ability to take the single reality that we live in, because we all live in the same world, even if we don't agree with each other or even if we are working on different things in different buildings, we're still in the same physical world. VR allows humanity to shift from being a species that lives in one world to being a species that lives in many, many, many worlds. And while we're all firmly anchored to the physical world, we come to inhabit in a really meaningful way all these different places. So I think that's the thing. It's like we're exploring new worlds, going to completely different planets, except we never leave Earth. And we can do that from the comfort of our own living room, which is kind of awesome. Great, well, thank you so much. Yeah, thanks for coming. This has been great.
[00:14:36.398] Kent Bye: And thank you for listening. If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash Voices of VR.
