Luca Marchetti is the CEO of Studio Evil and he talks about developing a serious game called Relive. Relive is a sci-fi adventure game set on Mars that is designed to teach people CPR. There were demos taking place in the hallways at Oculus Connect with people wearing an Oculus Rift while pressing on a Mini-Virtual Reality Enhanced Mannequin that was being tracked by a Kinect.
Here’s a video of the Mini-Virtual Reality Enhanced Mannequin and Kinect integration, which was being worked on before Palmer launched the Oculus Rift Kickstarter in 2012.
Here’s a description of the mannequin:
The project involves the development of a serious game and a self-learning software specifically dedicated to quality cardiopulmonary resuscitation.
We developed Mini-VREM (Mini-Virtual Reality Enhanced Mannequin), a CPR feedback device with a new motion detection technology including a Kinect sensor and software specifically designed to analyse chest compression performance and provides real-time feedback in a cardiac arrest simulation training setting.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast.
[00:00:12.022] Luca Marchetti: My name is Luca. I'm a game developer from Italy. We are working on a strange kind of game. The game is a game about CPR, Cardiopulmonary Resuscitation, and it's a game which uses virtual reality in the form of Oculus Rift headset and Microsoft Kinect and Intel Perceptual Computing Camera to track your movement during the CPR and tell you if you are doing it right or wrong. We built a whole game around this concept, and it's a serious game, but it's a game for normal people, it's not for healthcare professionals. So we are aiming to raise the awareness in young people about CPR. And the game, in fact, is a first-person sci-fi adventure on Mars, and that's the fun part of the game. I mean, we want the game to be fun and not boring.
[00:01:08.357] Kent Bye: I see and so in virtual reality you're on Mars and you were showing it here at the Oculus Connect conference and you actually had like a little CPR dummy that people were actually using as a little bit of haptic feedback in the virtual reality experience and so tell me about how you were able to tie together being on Mars and then doing CPR training games.
[00:01:32.040] Luca Marchetti: Well, you know, you can have a cardiac arrest on Mars. It's quite normal. Have you ever been to Mars? A lot of people are having a cardiac arrest there. No, seriously, the game needs to let people, young people, understand that CPR is important and they can save lives. And the setup is made like this because we need to interest people. So being on Mars or on a factory or in another place, it's quite the same. It's more interesting being on Mars. And it is an adventure. You need to discover things and use objects. doing stuff and sometimes during the adventure you have to learn how to perform CPR and perform CPR if you want to save people. That's the adventurous part of the game.
[00:02:21.595] Kent Bye: And so tell me a bit about the Kinect integration. I'm assuming that you're using the Kinect V2 and then using maybe the skeletal modeling to be able to track people's arm movements. Maybe you could just talk a bit about, you know, what are you actually doing to track what people are doing in physical space to mirror that within virtual reality?
[00:02:40.573] Luca Marchetti: Well, we use some kind of mixed technology to track the CPR movements. We use the Kinect skeleton, of course, and we use skin detection and image recognition to make some kind of sensor fusion to be more accurate. Then you play in first person, so you cannot see yourself, but you can see all the world around you and making CPR on people. You see your hands doing the chest compressions. I'm a game developer but we work together with medical doctors and people from Pisa University in Italy. They are specialized in computer vision and robotics and they are the authors of all the algorithms of tracking. We mix the technology, we mix the teams to achieve our results.
[00:03:28.498] Kent Bye: And so maybe you could tell me a bit about a failure state within this game in terms of if you're doing CPR wrong, what is an example of doing it wrong and then what would it show you within the game context?
[00:03:40.749] Luca Marchetti: Well, actually, in the game you could not do it wrong. I mean, no one is going to die in the game, actually. You work with a team, so if you can't perform CPR, maybe some other people from the team takes your place and, I mean, fill in the gaps and tells you how to do CPR. So the game is about learning how to do this kind of maneuver. So we don't want people dying in this game. That was a design choice.
[00:04:13.434] Kent Bye: Yeah, I guess what I was wondering is that if it is a game, then I'm wondering if you could just do it completely wrong and it wouldn't be able to tell whether you were doing it correctly or incorrectly. If you're trying to teach CPR, you know, how are you trying to determine whether or not they're doing it right?
[00:04:29.262] Luca Marchetti: Yeah, well, the game can tell you if you're doing it right or wrong. Speaking of depth of compressions and frequency of compression, the game is absolutely accurate compared to clinical tools. We wrote some paper about it, we made a couple of clinical studies about it, so we know it's accurate. And the game could tell you if you are doing it the right way. Of course, the game is not exactly made for training. It's made to let people know they can do CPR in the right way and take a CPR course to learn how to do it from professionals, of course.
[00:05:09.852] Kent Bye: And so what were you doing to be able to bring professionals into this VR experience? Were you doing motion capture with them as they were going through the process then?
[00:05:17.055] Luca Marchetti: Well, we work together with medical doctors about this. The scientific part is made by medical doctors themselves. I mean, we have a motion capture studio that we use to track movements and make all the animation about the game. And we, of course, track professionals doing CPR. And that's the way we work.
[00:05:39.350] Kent Bye: I see. And so here at the Oculus Connect conference, you had an actual kind of body dummy that people were pressing on. Do people need something like that? Or could they just use a pillow? Or what would they need to be able to actually do this experience at home?
[00:05:53.856] Luca Marchetti: Well, actually you don't need the mannequin to play the game. The mannequin, you can use a pillow, you can use a teddy bear, you can use everything you want. And you use Kinect and all these tracking technologies only for the tournament mode, so the training part. You can play the whole game with your keyboard and mouse if you want. We will teach you how to make CPR without the need of having a Microsoft Kinect connected to your computer.
[00:06:24.305] Kent Bye: And so what's next for this program? Where does it go from here?
[00:06:28.528] Luca Marchetti: Well, we need to finish the game before. We have two months of work ahead. We are launching the game in one or two months. And then we, I hope, we can move to the more serious training part. So use the game for real training. We are working with the Italian Resuscitation Council, which is the authority in Italy about CPR, to bring the game to a new level.
[00:06:53.188] Kent Bye: Yeah, so what is the target market and what kind of distribution channels would you expect? How is this going to get out?
[00:06:59.029] Luca Marchetti: Well, the game will be absolutely free of charge, so it's a free game and we will distribute the game on all digital distribution channels we can reach. We are trying to speak with Steam. We already have games on Steam, so we are trying to speak with Steam and other channels. Of course, it will be downloadable from our website.
[00:07:19.972] Kent Bye: And so are there other serious games or training applications that you're looking into doing within virtual reality then?
[00:07:25.035] Luca Marchetti: Actually not, we are concentrating on this game and we think this game will evolve for the next maybe year, I don't know, but we will work on this before concentrating on other games.
[00:07:38.922] Kent Bye: And finally, what do you see as the ultimate potential for what virtual reality could bring?
[00:07:44.632] Luca Marchetti: The potential for virtual reality, I mean, it's huge. I learned a lot of things about CPR with virtual reality now. I think there's a huge potential on this kind of application because it's less boring, more interesting for people, so you can bring more people trying to make these kind of things because it's a totally different approach to these kind of serious games. Great. Thank you. Thank you for asking me those questions.
[00:08:13.643] Kent Bye: And thank you for listening. If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash Voices of VR.
