One of the most intense interactive storytelling experiences I’ve ever had was the Accenture AVEnueS training application for child welfare caseworkers.
I was tasked with assessing whether or not I should remove a child from her parents as a representative for Child Protective Services, and so I was interrogating the parents through a series of prompts with different types of questions. I had to match the style of question with the temperament of the parent, and try to resolve contradictory stories and incomplete information.
At the end, I had to make a choice deliver the news to the parent. I opted to see how intense it would be to have to tell a mother I was going to take her child away and the resulting heart-wrenching pleading that followed was almost too much to bear. I learned that I’m probably not really cut out to be a child welfare caseworker.
It’s this exact type of screening and soft skills training that Accenture sees is a perfect fit for virtual reality. They collaborated with Courtney Harding’s Friends with Holograms in order to create a fusion of 2D billboarded video in 360-degree photospheres with a light branching narrative based upon what type of questions you ask.
I had a chance to catch up with Harding as well as Molly Tierney, the child welfare strategy lead for Accenture, while I was at SXSW. We talked about how they’re using VR to do screening, on-the-job training for a wide variety of human services fields. Tierney praised VR as being an inexpensive, easy to use, and deeply immersive, experiential learning environment for soft skills training.
Another big takeaway is the power of conversational interfaces to create a sense of social presence. I saw this demo on a 3 degree-of-freedom Oculus Go, and it used monoscopic photospheres and billboarded 2D video. All of these are the most minimal ways to get content into an spatial experience, and it was really impressive for immersive and effective it still was.
It looks like Oculus will be phasing out the Oculus Go from Enterprise offerings, and so it these experiences will likely eventually all be fully spatialized with volumetric capture. But as stopgap, the 2D and monoscopic versions are still cheaper, easier to produce, and the immersion benefit probably doesn’t outweigh the additional tech production and post production costs. All of these technologies and production pipelines will take time to get fleshed out, but the good news is that there are so many VR training scenarios that will help to fund a fully spatialized workflow.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. So continuing on on my series of looking at the VR for Good movement, this last section I'm looking at different experiences that have some sort of technological or architectural innovation that is going to change the future of immersive storytelling. So when I was doing an interview with Amy Zimmerman from Unity, she was talking about how she sees the future of VR for good, not just to be able to tell stories, but actually to become a part of the solution in which you start to see other either training or educational or medical applications, lots of different uses of virtual reality to actually bring about some sort of direct change. So I decided to pull this experience and it's called avenues. It's actually produced by Courtney Harding She's the founder and CEO of the friends with holograms and Molly Tierney's the child welfare strategy lead for Accenture who used to be a social worker who? was responsible for going into these different homes and to try to see whether or not the child protective services should take away a child. If the child is in danger or if they're not in a safe home, then these social workers actually have to go into these environments and make these assessments. And so this experience of avenues is actually you going into this person's home and you doing these interviews with the mother and father and the actual child. So the thing that's interesting is that this was done with like 360 photos. So you're walking into photo spheres but they have billboarded 2d videos of Actors that are being able to act out these different scripts. So you have a choice of asking different questions. And so I Do you ask the open-ended questions? Do you ask the pointed questions? Do you try to get specific numbers? Are you trying to have them talk about their qualitative experiences? There's all these different strategies you could do as an interviewer and there's different temperaments for the people that you're interviewing that are going to respond in different ways. And so part of this experience is for you to actually speak out the different questions and there's the different natural language processing that can detect what question you're asking and then it branches off into these different experiences and it allows you to kind of navigate and you're trying to investigate what the actual story is. What's the truth? Are they trustworthy? Are they consistent with all the different stories that they're telling? And you have to make a decision at the end whether or not to take this child from their parents or not. And I have to say, even though this wasn't necessarily like an explicit storytelling experience, it was one of the most intense narrative experiences that I've had, because you are engaging in this process of deciding whether or not to take away someone's child, which I can't think of a more big decision that you could make in somebody's life. And so this experience tries to simulate that in different ways. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Courtney and Molly happened on Saturday, March 9th, 2019 at the South by Southwest Festival in Austin, Texas. So with that, let's go ahead and dive right in.
[00:03:01.751] Courtney Harding: So my name is Courtney Harding. I'm the founder and CEO of Friends with Holograms. We're a full service virtual and augmented reality agency.
[00:03:07.714] Molly Tierney: And I'm Molly Tierney. I'm the child welfare strategy lead for Accenture.
[00:03:13.077] Kent Bye: Great. Yeah. So maybe tell me a bit about the project that you just worked on and how it came about.
[00:03:18.390] Courtney Harding: Yeah, so we worked on Avenues, which is a voice-activated VR training experience for social workers to help them get more comfortable in the field and learn about asking better questions. We made it as realistic as possible so that we could help train social workers the best that we could.
[00:03:35.272] Molly Tierney: Yeah, we really thought we could, by replicating in virtual reality the environments into which social workers step, like literally somebody's home, we thought if we got them an immersive experience, I mean like got their heart rate up, even caused anxiety or fear, all the things that get in people's way of decision making, that we could accelerate the pace with which they become seasoned decision makers. Typically that takes a really long time, years of practice, to, I don't know, get really high mastery of pattern recognition of human behavior and reactions to questions that you ask, how to de-escalate situations, how to phrase a question for a particular type of person so you're more likely to get more information from them, and how to filter a lot of distractions in the environment. We made this a very rich experience audiologically, visually, and all of the reviews that we're getting about it is, it feels like the real thing.
[00:04:36.903] Kent Bye: Yeah, well, I just went through the VR experience, and I've never been an employee of Child Protective Services, but this is probably the closest I've ever gotten, and it was really intense. This is like one of the more intense experiences that I've had just because You're navigating these moral dilemmas of trying to figure out what's happening, who's telling the truth, but you're basically, you're representing this entity that could be taking away the children from the parents. And you have to weigh the safety of the children versus the larger risks that are happening with the children, but also the connection that a parent will have with their child. So it sounds like that you've done this for a living. And so maybe you could talk about that process of trying to take what it took from your own experience and how to train people how to actually do this.
[00:05:25.388] Molly Tierney: Yeah, you're making a good observation. I was in child welfare as a practitioner for 25 years before I joined Accenture. And, you know, the fact is child welfare does often get it wrong, right? As a field, we have too many experiences where we should have removed a child and we didn't, and then perilous things happen to that kid. Those are the errors we can see. The errors we can't see are the enormous number of kids who come into foster care when we could have kept them safe at home. And so what we really hope to do with this is, yeah, it's giving people practice, but Really, it's not just about the part of the time where someone has the headset on. We imagine that there's also a learning opportunity when the headset is off in discussion, so that people can share, well, what did you see? Well, I didn't see that. Well, when did you say that? Did you hear him say that? No, I didn't ask that question. And how did you feel when you were sitting across the table from Lance, a character who is a great big bear of a man? And if you asked a particular type of question to Lance, he gets really angry. And I'm five, too, if it were Me, I might have been afraid of Lance. And if I were afraid of Lance, I might have said, oh, it's not safe for the kid. And see that error? Me feeling afraid of Lance is not the same as Sophia, the seven-year-old girl, being afraid of Lance. So giving caseworkers the opportunity to peel away at the things that help complicate how they inform themselves so that they could have a more laser-like focus on the singular question, is this safe enough for Sophia? And that's it. But there's so much in our way, all of us, me too. All of the bias, all of the life experiences we have, all of the ways that we react to people. that invariably get in everyone's way. Our hope is that we're peeling away at that and what we see is we're enabling child welfare agencies to have fewer errors so that they're able to more likely keep kids safe at home with their parents for the very reasons that you have stated and that we save the lives of children who need us. And let me ask you, Kent, did you take or leave Sophia?
[00:07:32.858] Kent Bye: Well, my heart was like, I want to leave her. But I also, from a narrative perspective, wanted to see what happened by taking her away. So it was a little bit of like, I wanted to take away to see that reaction. For me, that was a more interesting, from a narrative perspective, to put myself in the position of taking her away. Even though I didn't want to, I wanted to see what would happen and to have that experience. And it was super intense. I mean, it was like one of the most intense experiences that I've had in VR, having to say that. And so and I've done a lot of VR so there's a part of me that Was seeing it through many different eyes of like as a narrative experience, but also like as a training experience So yeah It was a long pause that I took to like decide what my decision was gonna be and sort of weighing what I wanted to experience that moment
[00:08:18.880] Molly Tierney: Remember when we fought with the developers about people have to have a much longer time before they decide because they wanted it make it really brief and I was like no people are really gonna struggle with this question so let it let a time I think it's a three or four maybe even five minutes we put in there of just you can sit and think about it.
[00:08:35.255] Courtney Harding: No, and I think our biggest thing was really let's make it as realistic as possible So that's why we use real people as opposed to like game engine characters That's why it's voice activated as opposed to a different type of interactivity because voice is realistic, right? It's realistic for you to sit down with someone look at your notes on a table and ask them a question and get their response. So we tried to make this as realistic as humanly possible so that people would feel as immersed as possible and not break the immersion whenever possible. And I think that's one of the reasons it's so powerful is because we really stuck to that realism. And I think once you start breaking it out into, you know, breaking up the experience or breaking the immersion, then you lose it.
[00:09:13.320] Kent Bye: Can you talk a bit about the process of crafting and writing what's essentially a branching narrative of a series of these different questions, but getting information for how to even construct an experience like this for training purposes?
[00:09:27.463] Courtney Harding: Yeah, so that was definitely something that Molly had a huge hand in, because she really provided the expertise. And we have a grid that we used, and each question has a different answer. So we had to make sure everything was open, closed, or somewhere in the middle, so the amount of information related to the question that you asked. And it was a very complicated process to make sure it all worked out. We also did this under a very tight deadline, so that made it really an extra level of fun. But everyone in the project really just like stepped up and worked incredibly hard and worked nights, weekends, middle of the night in my case, because I was in Asia when some of this was happening. So we just, I mean, I'm really happy though with how it turned out because we really crushed it. And I'm excited to do more of these going forward and to try to work new things in and new types of interactivity. But yeah, I mean, this was a, the script for this was complicated for sure.
[00:10:19.063] Kent Bye: Yeah, and to me, I ask questions for a living. So I was looking at all the different questions, and I was struck at how sometimes I would ask a question that I thought was maybe the best phrased question, but then it would get a yes or no answer that would kind of stop the conversation. And so what is the art of asking the right question?
[00:10:36.171] Molly Tierney: And so this is a really great window into how we designed Avenues. There actually is no correct answer. writ large for the experience about whether you should leave Sophia at home with her mom or whether she should be taken into foster care. And that's for the primary reason that caseworkers don't know when they go into a house what's the right answer. And to Courtney's point, we really wanted this to be as real as possible. So for our money, there's also no correct kind of answer. So Courtney described it. There are three different types of questions. in avenues, a sort of binary question that you get a yes or no answer, a quantitative question, it was like how many times, and a qualitative question, can you tell me more about that? And the trick of it is, there's not a better type of question. There is a need to assess which kind of question for which kind of person at which moment in time is going to get you the most information. And the characters are actually designed to have different reactions to questions. So if you ask the character Lance an open-ended question, hey, tell me more about that, he totally chills out. And he gets you good information because his character feels like you are now treating him with respect. If you ask Lance a yes or no question, you make him really mad. And in order to show you how mad he is, we actually took him off green screen and made him do 25 push-ups. Right? Remember that? Monica's the other way around. If you ask Monica a yes or no question, she calms down. But if you ask her an open-ended question, she starts babbling and she's off topic and you can't really, you can't even remember what question you asked. And so, like, that's just a window into there's no correct type of question. Any good caseworker needs a menu of them. Because there is a time to ask a yes or no question. Does he hit you? Is a yes or no question that you just, you, You've got to draw the line and know where am I on that data point. And there are other times when asking more open-ended questions are the right thing to do.
[00:12:29.895] Kent Bye: Yeah, there's so many different variables that you're trying to read the temperament of the character, you're trying to build trust and rapport, but you're also at the same time trying to gather information. So for me, that's a lot of things to juggle, especially as you're asking these sequences of questions. So I'm just curious how you were able to balance all those different things, and if trust and building rapport is something that you could start to model in these types of interactions.
[00:12:54.131] Courtney Harding: Yeah, I mean one of the things that we do with Monica early on is set her up as a potentially unreliable narrator, right? So I think the second question you ask, you get an answer where she starts explaining something and then she doubles back and then she sort of changes paths and then she is all over the place and we originally just thought that was a bad take. We just updated the lines, the actress did it the first time, she kind of flubbed it and we thought, all right, whatever, let's do it again and again. And then when we watched that take, we were like, oh, this works. Right? So I think a lot of it is just sort of like reading these people's personalities and figuring out, can you trust her? Like maybe she's just nervous and afraid, right? Your child welfare, you just showed up at her house. She's freaking out. Or maybe she's just bold-faced lying to you. And I mean, we've demoed this now for thousands of people, I think, at this point. And the number of responses to that, some people are like, oh, yeah, she's scared. And then other people are like, that woman's lying. I knew it immediately. I got two questions in, didn't need to hear any more. And the range in between, I mean, I've I would bet it's probably 50-50 in terms of who takes the children or who leaves the children, which is exactly what we were going for, because we try to make it as nuanced as possible, right? If there was an answer where it was like, Lance is an abusive monster, then fine. That's the end of the story. Take the kids away. But we had to make sure it was so many gray areas and so hard to make that final decision. I think you had the experience, and I've seen tons of people have this experience where they're like, I don't know. I don't know what to do. And that's true to life, right? If you're a social worker and you walk into a situation and it's very clear that something terrible is happening, your decision is kind of made for you. If you walk in and it's like, I don't know, are they? Aren't they? That I feel like is where it's a real struggle and people have to learn how to go in and make those decisions.
[00:14:32.418] Kent Bye: Yeah, I'm curious, how do you navigate interacting and asking questions with people where you don't know whether or not they're actually telling you the full truth, and you have to discern in that moment whether or not you're being lied to?
[00:14:45.262] Molly Tierney: I think you take the longer view, and over the course of the entire experience, how do you knit together information that was conflicting, information that was initially conflicting, and then you could feel like you really landed on, alright, well, I feel confident about some set of facts. You know, as we look to the future of avenues, when we made this last summer, The technology was at a particular place with voice activation where you have to script the questions the user is asking as well. We look forward to, you know, leaps forward in technology where we won't have to script the user's questions. And that would mean the user would be in a position to ask follow-up questions, to dig in more on a particular topic. And I think that will aid the how do I figure out what's really going on in this household phenomenon.
[00:15:34.960] Kent Bye: Yeah, maybe you could talk a bit about the larger context under which somebody would watch an experience like this in terms of you're designing a VR experience. And then how is that sort of fitting into a larger training?
[00:15:45.645] Molly Tierney: Yeah, sure. So, you know, we imagine that child welfare agencies can use experiences like this to augment their existing training because caseworkers, of course, get hired and then go through lots of onboarding training. And the idea that this could be used at the point of hiring so that people get a real sense of what they're getting into and don't learn that the hard way six months after they've been hired. and that before they're actually in the field, they can have experiences that replicate that. And of course, in sort of a more seminar format, have an opportunity with their colleagues to unpack their thinking and how they went about informing themselves when they did a home visit.
[00:16:24.484] Kent Bye: And what I noticed from a VR experiential perspective is that just by saying the words out loud and getting answers, it gives me this sense of deep social presence and immersion. And so I'm just curious from your perspective, what are the other design considerations that you're trying to do in order to increase that level of immersion in an experience like this?
[00:16:43.321] Courtney Harding: Well, so I think for us it's, like I said earlier, really keeping the user immersed as humanly possible. So not breaking at all. You know, obviously there have to be some transitions, but keeping those to an absolute minimum. And making sure it is as realistic as possible. So again, that's why we used real actors and actresses. We didn't use game engine characters. And make the interaction as realistic as possible. Is it unlikely that you'll have three separate note cards in front of you on a table at any given time when you're interviewing a family? Yes. But is it likely that you will have your notes on a table in front of you and look at your notes and ask questions based on that? Yeah, that's pretty true to life. So again, that was just a design consideration because people had to be able to read the questions. But that seemed as true to life as we could get. And then using voice rather than, say, a gaze or a clicker or something like that, that's realistic, right? I'm not going to gaze at a question, look up at you and expect an answer or take a clicker and point to a question and then expect an answer from you. I'm going to ask you a question and you're going to give me an answer with your voice. And, you know, again, anything that takes the user out of feeling like it's very realistic and very immersive is a negative. So that's what we try to do with all of our projects.
[00:17:53.532] Kent Bye: Can you talk a bit about some of the reactions that you've got from people who have social work experience, who may have been in these situations, and what their feedback was from actually going through this VR experience?
[00:18:03.106] Molly Tierney: Yeah, their general sense is that that's exactly what it feels like, that's exactly what it looks like, that's exactly how people behave, that we really got in the sweet spot of realism. And they're quick, once they have the headset off, to begin to imagine additional scenarios that they would really want to have access to, so that they could use it in their organizations.
[00:18:27.942] Kent Bye: And from the technology side, what's some of the hardest problems that you had to overcome in order to create this experience?
[00:18:34.626] Courtney Harding: I think for us, because we were doing it, again, on a pretty quick timeline, we had to pull everything together very quickly and get it done very quickly. But I think just making sure the voice works is hugely important. Luckily, we had a great technology partner in Conversive who did a fantastic job. You know just designing everything and making sure that it looked really good and that it looked really realistic So, you know as we look to do more of these the next question for us is not how do we replicate this? Because as great as this is it's been done But how do we bring in other elements and other narrative elements and other technology elements that are still very realistic and immersive? but also can make the experience even better and somewhat different so I You know, for us at Friends with Holograms, and I think for Accenture, too, we're always kind of looking forward to see what the next thing is so that we can, I don't know, keep coming back and winning more awards. I mean, if they're going to build a trophy room, then, you know, we got to fill it up with trophies.
[00:19:26.381] Kent Bye: So maybe you could talk about the award that you just got in the Mobile World Congress.
[00:19:30.315] Courtney Harding: Yeah, so we won a GLOMO award at Mobile World Congress. The other competitors were, I believe, China Mobile, LG, and other little indies like that. So really thrilling win. And yeah, I mean, we definitely are looking to do more of that and just really show more people and get more people excited about this. Because I think this has incredible applications, obviously, in social work. but in any space where you have to relate to people, empathize with people, and talk to people. And so figuring out how can we make this the gold standard for training in virtual reality.
[00:20:05.247] Kent Bye: Yeah, and I'm curious to hear a bit more about how this fits into Accenture's larger initiatives for training and what happens next from this point, because it's a bit of a proof of concept type of experience in that, is this something that's going to start to actually be deployed within different contexts, and then how to move forward from here in terms of the connection between virtual reality and training?
[00:20:27.053] Molly Tierney: Well, certainly we are on the cusp of deploying it in a variety of places around the country. What we see moving forward is more of a sort of library of the story of Lance and Monica and Sophia and other critical decision-making points that child welfare agencies get into with the families they work with. But we also imagine that if the task is give people practice at observing human dynamics, inquiring to get a rich set of information, and interpreting all of those things to make decisions about how you're going to take action, that the implications for that are not limited to child welfare. It's valuable in virtually every human services field, whether that's teachers, or cops, or Home visitors, we could make a very long list. Anytime someone's in a position of having to interact with people in this way, we look forward to cracking those eggs too.
[00:21:26.042] Kent Bye: Well, the thing that I'm really struck by, by going through this experience, was to see how it wasn't such a black or white, yes or no, right or wrong, linear type of experience, where it's clear what the answer is. It's a lot of more ethical and moral ambiguity, and it's an interactive dialectic, and so it's moving away from a linear way of thinking and more of a process way of having these conversations and asking questions and getting answers, but you're trying to navigate and map out this moral landscape. And it feels like VR as a medium is able to do that very well. I'm just curious to hear your perspective on that.
[00:22:00.556] Courtney Harding: Yeah, absolutely. I mean, the thing that's great about VR is it allows you to recreate situations you would be very hard to recreate in real life. So you can do this type of practice training with your teacher, or your classmates, or your colleagues, right? And people do that all the time. But there's a huge difference between doing that, because there's this uncanny valley of, like, you and I are role playing. OK, I can pretend that you're a potentially abusive parent, but really, I know who you are, and I know that you're not. Whereas with these actors and actresses, you don't really know that. And I think it's the type of thing where that can help, again, shorten the sort of training cycle. And one thing that we've had conversations with folks about is, how can we deploy this so that people can get to work more quickly and do it more safely? You can't send a social worker out into the field without extensive training, but maybe this can, again, help shorten that training cycle and get them out more quickly, you know, build up the pool of people who are doing this work. And the other nice thing about this is it can be done time and time again. So you could go through this experience 20 times if you wanted to, go down different question paths, really take a lot of learning from it. And that's why it scales really well in VR, because, you know, I'm not hiring actors and actresses or sitting in a classroom for hours at a time and dealing with that. I can just put on the headset, and the Oculus Go is an inexpensive headset that's easy to deploy. So, you know, I can do this again and again and again, and that makes the learning even more immersive.
[00:23:18.377] Kent Bye: How was this training done before? Did you just have people send out into the field and learn on the fly? Or did they shadow people? Or how did they actually learn these skills before?
[00:23:29.911] Molly Tierney: Most places use the Socratic method, like a lecturer in front of the room. There are a handful of places around the country that do simulations, kind of like role plays. You know, those are really heavy lifts and take a long time. So this would be a real accelerator for giving folks a real experience of being in the field.
[00:23:51.892] Kent Bye: Great. And finally, what do you each think the ultimate potential of virtual reality is and what it might be able to enable?
[00:24:03.597] Molly Tierney: Sky's the limit. I think it's limitless, the potential, particularly of this method, because it is, to Courtney's point, it's inexpensive, easy to use, deeply immersive. and gives the opportunity to learn in the only way humans actually learn, which is experientially.
[00:24:24.056] Kent Bye: Yeah, just a quick follow-up on that is, like, how do you measure success here? Or, like, how do you quantify or know that this is working?
[00:24:31.897] Molly Tierney: That's an excellent question. It's something we look forward to getting analytics about in terms of the impact on the learners, and we'll be back.
[00:24:44.757] Courtney Harding: Yeah, I know this is your signature question, and I've thought about this quite a bit at various points in my life. And I think at this point, it's the future of education. It's the future of how we train people to talk to one another and interact with one another. And I do think this can be really widely deployed. And it can be the type of thing where it shortens training cycles, it helps people get jobs. So one thing that I've thought about with this is, you know, the labor market's very tight. And there are a lot of people coming out of prison for nonviolent crimes or coming out of treatment for opioid addiction who are trying to get their lives back on track. How could we build something similar to this, where it's a screening tool, so you go to a store to apply for an entry-level job, rather than having to fill out a form and then someone looks at your form and say, okay, you have a felony on your record, we can't hire you. You can actually go through some situations and people can say, wow, you actually know what you're doing, let's hire you and give you a chance. So I think that's where some of this starts to end up, is really making a lot of impact on a broader social level.
[00:25:40.973] Kent Bye: Awesome. Great. Well, thank you so much.
[00:25:42.674] Molly Tierney: Thanks for the chance to talk about it.
[00:25:45.536] Kent Bye: So that was Courtney Harding. She's the founder and CEO of Friends with Holograms, which is a full-service VR and AR agency, as well as Molly Tierney. She's the child welfare strategy lead for Accenture. So I have a number of different takeaways about this interview is that, first of all, Well, being able to actually interact and speak out into an experience is highly effective. You know, this is a training exercise and you're trying to give you the sense of going into this environment and asking these different questions. And the fact that you actually have to speak out the questions, I think actually helps you to embody it, even though it's like written there on your cards. And maybe in the real life situation, you wouldn't be like reading it right off the cards, but it helps you just put it into your body with the different types of questions that you would ask. And a big part of this experience, I think, is trying to assess the different temperaments and trying to understand that either the, open closed or somewhere in the middle. So some people would have the binary responses. So they either give yes or no very short answers. Sometimes they would give very specifics in terms of quantitative, like how many times have you been using drugs or something like that? And then the qualitative, which is can you tell me a little bit more about that? And some of the different people that you're interviewing, they're more open to, you know, having those qualitative questions to be able to have open ended. And then if you try to get more specific, then they can get very tense and in their responses. And so you're not really a welcome to presence and this whole scenario, because you're representing in this experience, the Child Protective Services is this entity, this government organization that is potentially going to take their child away from them. And so The parents don't necessarily have all the incentive to be completely honest or truthful and so you're trying to match up the different answers from different people and trying to assess the whole situation and then make this moral judgment as to whether or not it'd be safer to take this child away from their parents or to leave them because they may have a harder time within the foster care system. It was fascinating to hear from Molly, you know, they're really trying to design this experience with a lot of ambiguity, like there's no clear answer as to what you should do. And I think that reflects real life in a lot of ways. And this experience was, even though it's not six degree of freedom, you're just using the Oculus Go, having a conversational interface actually has this huge amount of social presence and immersion within an experience like this, especially when they have. video captures of actors. I can't imagine that this experience would be quite the same impact if it was like a CGI created character because you would just lose a lot of the emotional expression and a lot of that realism that you get from dealing with these different issues and try to read the different subtle body language cues like VR and all the capture. It's just like way more expensive and I don't think actually it would be even as good as what they did with the 2D billboarded video and in the context of these photo spheres. So the photosphere is just trying to be in this environment where it's very stimulating. There's a lot that you can start to pay attention to. And as you go through and ask these different questions, you know, they try to design it so that you could go through it multiple times and go through different mini branches. And maybe that would give you a little bit more context or information, but. You know, this is something that they created for people that are just getting into the field to be able to give them some on the job experience. Maybe they go through this experience and afterwards they're like, you know what? I'm not really cut out to do this type of work. You know, it's just like way too intense. And I think this type of experience really starts to mimic that. Like I said at the top of this podcast, this is one of the most intense narrative experiences that I had in 2019. And it wasn't even like a narrative piece. It was a training video. but it just had this whole level of realism that was super impressed with the way that you were able to really feel like that you were there. You know, it would have been better to have photogrammetry, it would have been better to have like 60 degree freedom, it would have been better to maybe have volumetric capture in the context, but you know, actually, I don't know if it would make that significant of a difference, especially for people who are just brand new to VR and may not even be able to tell the difference. But, you know, this experience is a lot of trying to teach these soft skills, like these ability to be able to read situations that have these conversations, and you're really trying to get information out. So it's like having a conversational interface, you know, being able to actually use your voice as an expression of your agency actually gives you so much more opportunity to do a good enough job for these different types of enterprise training and soft skills training. So, uh, I guess, uh, to take a step back and to talk about why I think I wanted to include this into the VR for good, you know, I think this is a application of VR that is extremely useful. It's more of a professional enterprise, uh, I would say like training. And so, you know, I could very easily include a piece like this into something like a educational series, which I can dive into all sorts of other enterprise training applications that I've been covering over the last three or four years. But just the fact that you're using these different conversational interfaces and to start to think about teaching the complex nuances. I think this piece in particular, after getting through it, it's getting into these different moral dilemmas where you have incomplete information and you have a certain path that you can take. You have your own ability to assess people, whether or not you trust them or not, what questions you ask, your ability to kind of click and vibe with people and actually build up rapport and trust and be able to ask the right questions. That's one thing that I think that this experience doesn't mimic is this process of building and losing trust because If you are trying to build up rapport and trust with people, then you're able to potentially get even more rich and in-depth information from them if you're able to build that level of trust. But because it wasn't having sophisticated branching that was, you're on the path where you have more trust versus not as much trust. But I think that's like in the future where this can go is starting to model different aspects of trust and rapport and what information can you get out after you go through very specific branches. But this is something that is trying to teach people the complexity and the nuances of different dynamics. In this case, you're trying to learn what it takes to be able to get enough information as a child protective services, part of child welfare in general. And this is part of the larger initiative for Accenture, which is doing all sorts of other enterprise training. And they're really a big leader in this space, I'd say, in terms of figuring out what those use cases are and working with these really big organizations and uh, governments and entities to be able to bring virtual reality training to all these different industry verticals. I'm talking to Rory Dubuff at the XR for change. You know, there's just like this huge explosion because there's been like these early prototypes and proof of concepts that have been happening over the last couple of years. And a lot of those have been coming back and saying that this is like actually. Super effective and is way better than these existing methods which in this case Doing the Socratic method and being able to just kind of like have people imaginatively think about these different things It's a lot different when you're actually in the seat in somebody's home and you're across from what are essentially actors that are acting these out but you're able to actually have this interaction and you get so much more information as to whether or not you are in this classroom with the teacher trying to kind of walk you through these more disconnected abstracted ways of talking about things and That's what Molly said at the end, is that VR is inexpensive, it's easy to use, and it's deeply immersive. And it allows people to learn the way that they learn by actually having a direct experience, especially as you're able to be there and be able to actually mimic asking those questions, which is exactly what you do when you go into these different situations. So I think there's a huge possibility here. And for me, actually, this experience avenue is really showing the potential of voice activation. And there's just a lot more that could be done with other soft skills training here in the future. And also for a narrative perspective, um, Kevin Cornish is actually involved in this project as well. And he had a piece at Tribeca that is using this type of conversational interface as well. And so I had a chance to be able to see that experience and talk to him about that as well. I hope to air that at some point. Um, so I go back through a lot of different experiences that I've recorded over the last couple of years. That's part of the backlog to dive into, but. This whole concept of artificial intelligence and conversational interfaces, I think, has a huge future, especially looking at the previous conversation with Stephen Smith, looking at Storyfile, and looking at being able to combine all this different information in ways to have access to conversational interfaces. This is yet another way that you can be put into an environment and have this interaction and using the voice activation of these virtual reality headsets to be able to have these conversational interfaces. And I think there's just huge potential for where this could go to be able to either educate people or to bring about change around specific topics or issues. So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast, and if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a list of supporters podcast, and I rely upon donations from people like yourself in order to continue to bring you this coverage. So, you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
