Gareth Henshall is from Bangor University in the United Kingdom, and was presenting a poster at IEEE VR titled “Towards a High Fidelity Simulation of the Kidney Biopsy Procedure.” Their goal was to create a low-cost training simulation that could allow doctors to train on having the experience of giving someone a kidney biopsy. They tried to do it without haptic feedback, and found that it was not effective at all.
They ended up using a haptic needle that was able to simulate a force profile for the different tissues of the kidney, liver, and spine. They captured these force profiles of the tissues by using a Force Sensitive Resistor Glove that they created that’s able to measure the pressure in Newtons measured over time for different substances.
They’re using a zSpace holographic imaging display to show a stereoscopic torso with the organs that are surrounding the kidney, and in combination with the haptic feedback then they’re able to recreate the feeling of doing this medical procedure in a safe and repeatable fashion.
The takeaway point for me is that to do haptics well, then you have to have a very specific use case. Here they’re recreating a specific medical procedure. And they plan on expanding this to other procedures with other force profiles so that this one system could simulate 30-40 different procedures, which is pretty much impossible to do right now since the physical models that exist today are created for each different procedure.
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast.
[00:00:11.958] Gareth Henshall: I'm Gareth Henshaw, I'm from Bangor University in North Wales. We're making a kidney biopsy simulator using the zSpace and a phantom haptic device. And the idea is to provide an affordable, off-the-shelf training simulator for doctors which can be used in hospitals. Because currently, having talked to doctors, the way they train this system is they watch their supervisor do it, they then research it, they then do it. which seems it's all quite rushed and it's quite a shock to the system. So having this innate ability to be able to use this, they can use it, just go into a lab, 24-hour access, load up the software and use it. With the haptic device, we've applied force profiles, which we've got through various sensors and then testing them out on current consultants in hospitals locally to get them as accurate as we can. So we've got different profiles Obviously if you hit the spine, which we hope you wouldn't, it's going to be much tougher to push through than if you hit the kidney, which is obviously just your organ and muscle. So we've applied multiple force profiles within here to be able to compensate for that.
[00:01:20.993] Kent Bye: So yeah, just to expand on that a little bit in terms of the technology. So it sounds like to give this haptic feedback, it's very specific to this task. And so you have a device and then you're lowering it down and imagine that once you puncture the skin, it's going to have different force. And so you're kind of recreating that spectrum as you're using this device and kind of looking in, in this Z space, 3D stereoscopic virtual reality projection, you're kind of getting your muscle memory trained to be able to do this. Is that kind of what you're doing then?
[00:01:49.722] Gareth Henshall: Yeah, basically it's meant to give as real a representation you can of actually injecting into human skin without physically touching a human. So the force profiles themselves will include at the start the actual force of entering the skin because that's going to happen. Wherever you are, you then have a little bit of force going from the skin to the organ. And obviously the force then increases as you go into the organ. But obviously, depending which organ you do go into, the forces are going to be slightly different. So we've had to compensate for that in different ways.
[00:02:21.103] Kent Bye: And so have you had like experienced doctors come in and kind of do this sort of virtual simulation and tell you how accurate it kind of feels from doing the real thing?
[00:02:28.878] Gareth Henshall: Yes, we've had constant feedback from doctors locally. We've had a consultant urologist who comes in every so often or we go to him, get his feedback on what we're doing and work together to create the most realistic scenario basically. We've had positive feedback from him generally overall. We started off without the haptic and just had a free floating pen, which was, it did the right thing, but obviously without the forces, it was so unrealistic, it was not even worth doing, continuing that end of it. So we added in the haptic device, which just turned the whole project around effectively, and he much preferred that. But overall, his feedback's been very positive, and hopefully we'll just keep working with him to keep progressing it to be more and more useful for him.
[00:03:10.897] Kent Bye: And so how do you measure the efficacy of something like this then?
[00:03:14.441] Gareth Henshall: Well, I guess, I don't know, I haven't done that really. I'm just behind the creation of the simulation. I've just been tasked to make it as realistic and then it's down to my supervisor and the doctors to decide if it is ethically right and we are doing the right thing. I mean obviously we had to use a generic torso shape and body shape so it's a very generic early entry training tool as opposed to one who's trying to refine their skills so I guess we could kind of skirt around it a little bit that way saying it's more of an entry level trainer as opposed to a okay you can do the procedure let's get you to a pinpoint accuracy more so because we've also included a transparency feature where you can turn the skin transparent on the patient So when you're first doing it you can enter the needle, turn it transparent and see exactly which part of the kidney you're hitting to get a more accurate feel and be able to see it better on the back just to help you practice throughout.
[00:04:14.513] Kent Bye: And so are there force profiles for, you know, going on the wrong path? You know, for example, if you're going down and not the kidney and, you know, hit a bone or something, it has a different forces. I'm just curious about the range of fidelity of where you're supposed to go. But if you miss it, can you tell just by the haptic feedback?
[00:04:30.386] Gareth Henshall: Yeah, yeah, you can tell completely because obviously, as we know, in reality, if you put a needle into bone, it's going to feel very different to put a needle into muscle. So we've replicated that as best we can with the force feedback. So there's different profiles for whether you hit different muscles, different organs, different bone structures. So if you hit the bone, the haptic will lock in place basically. You can't push through the bone. Obviously the haptic device has it. It can only take a certain amount of pressure before it will just give and you'll effectively fall through the bone. But we try to make it as real as possible with everything that's in that area. So we've only included the spine, the kidneys and the liver because we've kind of almost taken the assumption that the surgeons before us know roughly where the kidneys are in a torso, they won't be touching the heart and things like that. So we only needed three or four profiles to cover the area where the kidneys are.
[00:05:20.606] Kent Bye: And is this something that you had to generate yourself in terms of the force profiles, or is this something that other people have also been researching in other literature?
[00:05:27.307] Gareth Henshall: So the force profiles were already given to me. They were already done using a force-sensitive glove, which was made in the department. Using that and then in collaboration with the urologist to see if the realism, he was comparing it to what he would normally do when he did the procedures. But like I say, those profiles were already just given to me and I just had to implement them.
[00:05:47.757] Kent Bye: Oh wow, so this force-sensitive resistor glove, tell me a bit more about what is this able to do in terms of like, do you take an object and squeeze it and see how much force it takes until it breaks, or how do you use this?
[00:05:58.942] Gareth Henshall: So that's kind of what I know about it because it wasn't something I was involved in as such, but from what I know it was that basically what you described. You could take an object, you could squeeze it, you could feel the forces needed to compress it to a certain level. So they were using it on the back and just pressing on the back of the person. You could feel where the flesh was. You could feel where the bone was. And you could feel the different forces between the two. So they were starting to use that to do it. But like I said, I wasn't in the process of doing that. I kind of just saw it right at the end when I was given all the profiles, unfortunately.
[00:06:30.314] Kent Bye: Cool. And so what's next for you in terms of, you know, are there other different types of simulations that you want to use this technology to do other things? Or what's next for your research here?
[00:06:40.721] Gareth Henshall: The idea with this is we're going to keep developing this one. There's someone currently working on it back in Wales. We're going to try and expand it out because obviously it is a virtual environment. We can have multiple simulations in the same game as it were. Our next one we were thinking about afterwards is doing a kidney stone removal. Very similar procedure, just slightly different tweaks in there. So we're going to expand to that, but then we've also had someone doing like a lumbar puncture. So using the similar kind of systems and technology behind it to do multiple procedures. So it just means they're not restricted like they are with the more mannequin-based simulators where they can only do one or two procedures. They're able to do 20, 30, 40 different ones all on the same piece of kit, sat in the same chair. So that's kind of where we're hoping to go with this anyway. Okay, great. Well, thank you. Thank you very much. Cheers.
