VR is a new communications medium, and as such it’ll likely be applied to anywhere other existing communications mediums are already used — such as in courtrooms for presenting evidence. Kineticorp VR has already been presenting 3D forensic evidence in courtrooms, and I had a chance to talk with Tomas Owens about some of the unique challenges that VR faces in getting into the courtroom. They’ve created some prototype demos, and done some early prototypes that have yeilded some promising and surprising results. So I talk to Tomas about their work on that path to using immersive technologies within a legal context.
LISTEN TO THE VOICES OF VR PODCAST
Donate to the Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye and welcome to the Voices of VR Podcast. So virtual reality is a brand new communications medium. And with that, you could expect this new communications medium to be applied to all different dimensions of society. And at Oculus Connect 3, I had a chance to talk to somebody who's actually been working on creating 3D photogrammetry scenes to be used within the courtroom. And so, for Thomas Omans of Kineticorp, the next logical step for virtual reality is to have it actually be used within the courtroom for jurors. What are the implications of that? Is it going to be a technology to be able to give everybody a similar sense of spatial relationship and to help illustrate many different perspectives? Or is it going to be too powerful of a tool to be able to create a sense of what the reality was? So we'll be covering the implications and the future of VR in the courtroom on today's episode of the Voices of VR podcast. First, a quick word from our sponsor. This is a paid sponsored ad by the Intel Core i7 processor. If you're going to be playing the best VR experiences, then you're going to need a high-end PC. So Intel asked me to talk about my process for why I decided to go with the Intel Core i7 processor. I figured that the computational resources needed for VR are only going to get bigger. I researched online, compared CPU benchmark scores, and read reviews over at Amazon and Newegg. What I found is that the i7 is the best of what's out there today. So future proof your VR PC and go with the Intel Core i7 processor. So this interview with Thomas happened at Oculus Connect 3 happening from October 5th to 7th in San Jose, California. So with that, let's go ahead and dive right in.
[00:02:01.579] Tomas Owens: So my name is Thomas Owens. I work for a forensic animation company called Kineticorp. We're based in Denver, Colorado. And what we're trying to do is bring virtual reality to the courtroom.
[00:02:09.503] Kent Bye: Great. So it sounds like, is this something that would be, you imagine, kind of rendering out the crime scene and then show the witnesses? And then, yeah, walk me through what you would imagine when and how you would bring VR into the courtroom.
[00:02:23.510] Tomas Owens: Sure. So for right now, we kind of see your personal agency as a little bit of a limitation for getting it onto jurors' heads. You know, if I look left, you look right, we don't get the same experience necessarily. So in my opinion, I think augmented reality is kind of where it goes in the end, where us together are sitting either down as jurors, and there's a lawyer, you know, who can expand an environment around us, or we can pull up a vehicle, flip it to the underside, show you where, say, like a malfunction happened or something like that, to kind of give you a better, like, spatial recognition of what's kind of going on. As far as getting into the courtroom net right now, we sort of see it used for getting testifying experts on the same page. So whether or not someone who's, say, like a trajectories expert or a rollover expert, if they're not able to get to the scene or it's too dangerous or it's been completely constructed over, you know, we want to be able to recreate it at the time of the accident and then let everybody have the same kind of spatial awareness and understanding as if you'd went to the location yourself.
[00:03:13.148] Kent Bye: I could imagine a time when someone is able to construct a very compelling story in VR, but it may not be true. So then you have the battle of who can create the reality in VR the most compelling. Talk a bit about that, you know, because you're showing it as either the prosecution or perhaps it's the defense. But yet, you know, with the other side, how would they be able to counter that?
[00:03:37.502] Tomas Owens: Right, so one of my favorite anecdotes is when traditional 2D animations were first getting admitted into court, there was a big argument like, oh, these are cartoons, this is not factual, it's biased or whatever, trying to give you a visual sense of what happened, and so there was a whole process as far as getting just traditional animations involved into the court system. So for us, we just kind of see it as like the natural progression of where, you know, it's a communication medium. So the best way that we can communicate the physics of our case. So, you know, there's not a time where we're, you know, making up analysis or anything like that. That wouldn't benefit us or our client. So it's more, you know, all of our traditional analysis and creation techniques for the animations are still there. We're still 3D scanning environments. We're still using photogrammetry to place evidence down that doesn't exist anymore. And then we can kind of give that to the engineers. And then myself as a forensic animator, I would be building the environments and the vehicles and things like that. And then we pass it over to the engineers, and they do physics simulations for the vehicles. So it's all grounded in physics still, and it's all through traditional kind of analysis methods. It's just a new way to display or communicate those findings.
[00:04:38.593] Kent Bye: Yeah, so maybe you can walk through a little bit of what you're doing already in this field, because it sounds like you're doing a lot of 3D immersive photogrammetry stuff, but presenting it in a 2D medium.
[00:04:48.326] Tomas Owens: Right, so the issue that I find is, you know, we go out, we get these very high-fidelity 3D scans, we get really complex environments and stuff, and unless I'm viewing it in like a traditional 3D software, there's a bit of a loss of information. You know, when I take it, compress everything down to a graphic or into like an image that shows what happened, you know, if it's like a trajectory, so say we've got one person and a second person and there's, you know, a line between them showing how the bullet traveled, you know, all of that depth information is completely lost in one image. You know, I could do a sequence of images or an animation where it kind of rotates around, but again, just the sense of, like, scale and distance and speed is so amplified in VR that it's such a perfect medium for us to kind of relate, I guess, the emotional context, but therein lies a fine line as well. You know, we can't have the probative value, you know, outweigh the prejudicial. You know, it's always a matter of I can't be trying to sway the jury emotionally. That's going to get, you know, there's a lot of complications and people try and get that thrown out immediately. And once you have an animation thrown out, you're without your visual aid at that point, which is really, really powerful in today's court system. Just having something to show people to relate that information.
[00:05:51.989] Kent Bye: Can you talk about the boundaries in terms of being able to take a photogrammetry capture, it's a reality capture, versus creating kind of a virtualized theory of what you think might have happened? You know, there's a boundary there, I think, that you start to get into VR, where there's the evidence of what actually happened, and then there's the theory of what you think might have happened, and you kind of play it out. What are you able to do and not do in that case?
[00:06:15.688] Tomas Owens: So, I mean, that kind of stems to our traditional animations that we do now. You know, there's always going to be holes in people's stories. You know, people say that eyewitness accounts are usually some of the least reliable, which is not very fun for me when I'm reading through pages of depositions and people are kind of giving contradictory stories. So in that sense, our clients come to us as a visualization firm to kind of fill in the gaps. You know, we have a better understanding of the environment and the way these things laid out because I can see them in 3D. And so, you know, until we produce something in a three-dimensional form for the client, they don't really know the answers themselves. So they kind of come to us and say, is this argument valid? We'll go through all the analysis. We'll get the physics simulations. And then we can say, oh, this guy was going 70 miles an hour in the simulation when he said he was going 60. So kind of a conflict there. But virtual reality just kind of enhances that and the fact that we can get that information in a much more first-hand experience context for people.
[00:07:07.082] Kent Bye: And you mentioned that you're starting to do a pilot study. Maybe you could talk a bit about what that actually looks like and some of your early results.
[00:07:13.047] Tomas Owens: Sure, so we just recently had a focus group where we took 36 jurors, fake jurors, and basically had them sit through both the defense and the plaintiff side of things. And then afterwards, because they didn't want to contaminate their results, we sort of led them over and had them try the VR experience. So it was a reconstruction of the environment, again, done with 3D scans, so everything is accurate to scale. You know, my job in that is texturing, surfacing, that sort of thing. And so what we get out of that is the ability to gauge people's understanding of what they understood from the traditional speech of the lawyer versus what they can actually see with their own eyes. And so we had really favorable results for the first focus group. We had kind of across the board at least 60% favorable responses. So the majority of people either thought it was helpful, helped their spatial understanding of the environment. Some of them actually changed their opinion about the case once they had seen it in VR. They're like, oh, I didn't realize this was so close or that distance was this or whatever. And so we had very favorable results with kind of the first round of testing that we've done. And kind of going along with that, some of the other types of use cases that we have, we've got like police shootings, medical malpractice, any case that really benefits from being in different people's perspectives. So again, it's not really perfectly designed for every case. If it's a more kind of mundane case or doesn't involve people necessarily, you know, like a vehicle rollover or something, granted it would be incredibly compelling to say stand in the path of the vehicle as it's coming at you, but some cases are better suited for VR than others.
[00:08:38.183] Kent Bye: Can you talk a bit about some of the surprises that you had in terms of how people kind of react
[00:08:42.764] Tomas Owens: Absolutely. Definitely. One thing that was really surprising to me is when we did our focus group, we had one lady in particular was incredibly emotionally attached to either the case. I believe she might have been familiar with the place where this had happened. And so I put the headset on her and within about 15 seconds, she was in tears. And it took me by surprise the fact that, you know, we have to sterilize these environments. There's no, you know, gore or anything like that. Again, fighting that that fine line between being overly emotional But what really took me by surprise was the fact that kind of a sterilized, simplistic environment can still have that really emotional connection if someone has the personal connection to something like that that has happened.
[00:09:21.672] Kent Bye: Yeah, I mean, imagine that that could happen already with film or other evidence that's shown being able to invoke an emotional reaction. But I think with VR, it's particularly tricky in the sense that there is more likelihood of something being in a VR experience that could be triggering of PTSD of people if they already have had experienced trauma. I mean, I don't know if you can completely sterilize it or you know, there's no telling as to what might trigger someone in PTSD, you know, so putting someone into a VR environment may kind of have an adverse reaction to people. So I'm just curious if you've had any thoughts about that.
[00:09:57.341] Tomas Owens: So we're really interested in the research that academics are doing in VR as far as the psychological implications, you know, how it changes the way that either you feel about something, either directly after or sort of a continued effect afterwards. So we're definitely more interested in, I guess, how it affects people, like you said, whether or not it triggers PTSD. And again, traditional animations can definitely do that as well. And it's so hard to know whether or not someone is going to be triggered off by something, you know, unless you have intense questionnaires or... you're filtering out the people very specifically. So in kind of a general sense, we do the best that we can to, like you said, sterilize it and get it the most easily viewable and easily just communicated finding. Again, we're just trying to show physics, basically. So what I could show on a graphic or an Excel sheet, I want to show that in just a visually engaging way so that you understand simply the physics of what happened. And again, the emotional connection is almost like a good side effect of what happens, but it definitely can be detrimental. So we're looking into you know, just kind of the research and the implications as that continues to advance.
[00:10:57.404] Kent Bye: So what do you see as the path forward in terms of getting VR into the courtroom then?
[00:11:01.260] Tomas Owens: Right, so it's definitely going to need a few more advances. Definitely getting wireless is going to be a big benefit. Again, we don't really necessarily see 12 Gear VRs or 12 Riffs sitting on JIRS anytime soon. Could it get to that point? Potentially. But again, we kind of see the trajectory going into more of the augmented reality space where I'm not isolated in my experience and it's more of a kind of a collective experience that we can do and someone can kind of walk you through it or, you know, there's much less calibration for myself to do if I'm just watching someone doing an experience as opposed to being completely isolated in it myself, we find that we had to give a pretty hefty introduction to people who've completely fresh to VR. And so being able to limit that would be incredibly helpful, being able to get not only a judge, lawyers, everybody on the same page and experiencing the same thing. We kind of want to get that copy-paste experience for everybody.
[00:11:49.289] Kent Bye: Great. And finally, what do you think is kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:11:56.768] Tomas Owens: So, I see the ultimate potential of virtual reality just being the best communication medium possible. Oculus gave a great talk today about the social advancements and things like that, and from a personal standpoint, as a gamer, you know, I'm really excited about being able to hang out with friends, you know, around the world and whatnot, but I ultimately see it as being a big transformative shift in the way that we think, in the way that our consciousness changes, you know, being able to kind of strip away some of the boundaries that we manifest towards each other. And I think that as it develops and we start to, again, become less isolated in the experience, granted, there's still going to be a need for that and a want for that. But I think, ultimately, the end result of virtual reality is just changing us as a human species and being able to coexist without necessarily having these artificial barriers that we put up. And I think that the first experiments in social VR are a really good implication that those just completely melt away.
[00:12:46.751] Kent Bye: Awesome. Well, thank you so much. Thank you, Kent. So that was Thomas Owens of Kineticorp, and he's in the process of trying to bring virtual reality into the courtroom. So I have a number of different takeaways about this interview is that first of all, some of the demo simulations that I saw were pretty much computer generated, which really begs the question of, I think, what is the role of VR in the courtroom? Because whenever you're creating a synthetic simulation of something, it's very convincing as to like, this is what actually happened. And because it is a simulation, it's not like 360 video footage of the actual scene. So then it's going to be basically reconstructed and based upon a lot of different eyewitnesses. I think the danger is that whenever you start to have these different simulations, then it's very convincing for people to think that that is actually what happened. And so I think, you know, what Thomas is saying is that they're really trying to just go with the facts and the physics simulation of things. So that was interesting to see how much things were kind of synthetically created and to think about what is the role of this VR technology and would it actually be too powerful of a medium to be able to sway the jury one way or another? Because I could see a situation where a certain reality is created by either the prosecution or the defense that actually didn't happen. And so it would be slightly different than what was actually experienced. I think that has a lot of implications for truth and justice. On the other hand, I do think that it is a powerful medium, and there are situations where you can start to recreate different scenarios and see what was a reasonable type of reaction for people. One of the things that Thomas mentioned that I didn't quite get until I saw some of the demos was the importance of different perspectives, because there are perspectives of the victim and the perpetrator, you could say, someone who's been wronged and someone who is committed that wrong there is the perspective to see whether or not what they were doing was reasonable or not and whether there was intent and yeah i think there was a scenario where there was a truck that was changing lanes and then there was a motorcycle that was either speeding or going regular speed and there's a crash and so you kind of see both perspectives and you know from one perspective you're like oh my god that truck has just you know changed over three lanes and crashed into the motorcycle, it's crazy, but then when you're actually looking at the perspective of the truck driver, you're like, oh, wow, well, if that motorcycle was going super fast, then it's entirely possible that he couldn't see him in the rear view mirror. And so it just kind of, you know, depending on whether or not this was like the defense of the prosecution, you can kind of skew the VR experience to try to give more sympathy to one perspective or the other. So it's a very interesting area, and it's an area that I actually hadn't put a lot of thought into, but I think that it's actually going to have a potential role within the courtroom. I think that Thomas's intuition that augmented reality, I think, is probably going to be a little bit better situation, because it is a public forum, and you're basically telling people to go and cover up their eye contact and go into this other world. And so I think in the courtroom, having everybody be able to see a common augmented reality type of scene where the prosecutor could still be within reality and have eye contact with all the jurors, I think that's actually is going to be an important aspect of this. And maybe there will be certain scenarios where it makes sense to give the jurors a sense of the space so that they can put themselves into the crime scene and have a better idea of the different spatial relationships between different objects and the spatial relationship between the given scenario. They also in one of their demos had a bit of a gamification where you're driving a truck and you're making a left-hand turn and depending on how fast the motorcycle is going then that kind of dictates whether or not it's a reasonable amount of time that you think that you had to turn. And so it kind of gamifies that by randomizing the different speeds of the motorcycle so that you can start to see whether or not there was a reasonable amount of time or not. Again, this is based upon eyewitness footage. And then this simulation is kind of creating a sense of reality of like, well, is this actually what the spacing was, what the conditions were? So it starts to get really interesting, I think. And I think it's gonna be probably more of an uphill battle than than not just because there is such a Variance of eyewitness testimony, which I think a lot of this is what these simulations are based upon But in some cases it could actually make a huge difference so I think Thomas is right is that eventually VR is gonna be in the courtroom, but maybe a bit of an uphill battle, especially as people are It's just start to get more and more experience with VR. I have a hard time imagining that people be thrown into VR as the first VR experience while they're a juror because there's other issues that happen. I mean, some of these simulations that they're creating actually have local motion. There could be motion sickness issues. You know, you basically put someone into a VR experience, and if it's moving around and done poorly, then you're basically making the juror sick, and then that, you know, what the implications of that, where you're basically inducing motion sickness in someone who's supposed to be paying attention and making all these decisions. So, there's other considerations like that. So, having good VR design, but also just making sure that the people who are on the jury actually have some experience with VR before you start just throwing like, hey, surprise, we're gonna be throwing you into this VR experience. The other thing that I think might come up is just the emotional reaction for some people. It could actually have either a traumatizing or a very emotional reaction that could be surprising for people. I think just in watching some of these early prototypes of some of the types of cases that could be put into VR, it's tricky and complicated. And, you know, after seeing a lot of VR, I personally don't know what to think about it, like whether or not it's a good or bad idea. I guess it's dependent on whether or not the VR experience is used to defend me or to go against me, because I could actually see it going both ways of actually telling the truth of what is there, or anybody could create a simulation that can create whatever scenario they want. you know that's scary to be able to create simulations in that way that you know there's this whole thing of like oh this evidence is now struck from the record but you know after somebody that always like confused me because it's like oh well they've already heard it so they're just going to delete that from their brain well when you put somebody into a vr experience you're basically like giving them a memory as if they were there and then you're telling them to remove that memory from their brain, which, like, can you really do that? So it has the potential to be used in really sleazy ways, I think, and I think it just has to be really careful, because it's a powerful technology, and maybe there's a limitation that you're only using 360 videos, so you could only use it if there happened to be a 360 video there. You know, this is sometime in the far future, probably, where we have the ability to do that. or perhaps if the ballistics and the physics are so determined to be correct in terms of like recreating something from both audio sound as well as you know for the timing of things but also for the photos or if there's videos that have been recreated and you know you're kind of synthesizing all this different disparate points of eyewitness testimony maybe there's a lot of video camera footage and have some way of definitely combining the two like you could combine them but also have the original source material right there just to make sure that you could verify it so that you could recreate the scene and if somebody's standing in from multiple perspectives you actually have to the side you know the footage of what the camera is looking at but you're able to you know for sure recreate the positions and what was said and I think this could be really interesting and there could be ways to do it, but again, I'm cautious just because of how powerful the medium is and how the medium could potentially be abused to be able to say whatever you want. Or to be put into a perspective that occludes or shows the story that you're trying to tell. So, lots of really juicy questions and I'm curious to see how this evolves. So, that's all that I have for today. I want to just thank you for listening to the Voices of VR podcast. And if you like the podcast, then spread the word, tell your friends, sign up for my email list on voicesofvr.com, and become a donor at patreon.com slash voicesofvr.
