Fight Back is an interactive narrative that is aiming to help teach the broad movements of self-defense to women. There were five different levels showing at Venice Immersive where you learn a new hand motion that’s detected via hand tracking. The experience had a wide variety of different problems with hand tracking properly detecting ranging from environmental conditions to having too many nested if-statement conditionals within their code that made it more difficult to trigger the hand gesture.
I spoke with director Céline Tricart about her underlying motivation in creating this experience, and the broader context of how learning self-defence impacts various statistics around domestic violence and sexual assault. Tricart’s The Key experience has also allowed her to keep close tabs on the shifting demographics of VR to having more registered women using VR, which encouraged her to push forward on developing a hand-tracking driven narrative even if there are still a number of bugs and glitches that still need to be worked out.
The Fight Back experience was developed with the interactions in mind first, but it also feels like the narrative in this experience has superseded some of the refinement of the interaction design. There’s still a lot of work that needs to be done to clean up the experience and prepare it for launch, but Tricart also mentioned a number of times the challenges in fundraising for projects like this. Hopefully they’ll be able to find some more funders who can see their vision for where they want to take this as there is so much potential in combining these embodied hand-tracked interactions within a narrative to tell a deeper story about what’s behind the shadow characters and providing a catalyst for women to want to learn more about the techniques of self-defense.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that's looking at the structures and forms of immersive storytelling and the future of spatial computing. You can support me on Patreon at patreon.com slash Voices of VR. So today's episodes with Celine Tricar who has a piece at Venice immersive called fight back so this is trying to teach different elements of self-defense to women and the piece is using a lot of hand tracking technology and having you go through these different motions that are trying to teach the essence of different self-defense techniques and in different forms, but also telling a story. And as you go through these five different phases, you're learning different techniques as you move along, but also trying to cast it within a context of this story where you're a star and you're fighting these different shadows and you learn more about the nature of these shadows as you go through this process. And so there's actually a number of different technical difficulties that I had as I was doing this piece. I did it early on. I think later on in the week, they had resolved a number of those different pieces that we're having with the hand tracking, but also just generally hand tracking within the quest. There's still, from Celine's perspective, some more work in order to really get it super solid, to be more fully integrated into different narrative pieces like this. And so, Celine in the past has worked on pieces like Sunlady's VR and also The Key, which actually won at Venice back in 2019. So yeah, just have a chance to be able to sit down with Celine to talk about this process of working with hand tracking But also these other dimensions of self-defense and trying to use these immersive technologies to reach out to other Demographics that are normally focused in on so lots of really good insights on that as well So that's what we're coming on today's episode of a severe podcast. So this interview with Celine happened on Monday September 5th 2022 so with that let's go ahead and dive right in
[00:02:05.088] Celine Tricart: My name is Celine Tricart. I am a storyteller, and I have been crafting and creating virtual reality experience for the past seven years. Before that, I was working in the film industry, and now I'm also involved making video games.
[00:02:23.346] Kent Bye: Great. So maybe you can give a bit more context as to your background and your journey into VR.
[00:02:28.521] Celine Tricart: Sure. So I started, I'm originally from France and I started working in the film industry and from the get-go I was passionate about new technologies and more specifically new technologies that apply to storytelling. So back in the days it was stereoscopic 3D. So I specialized in 3D back when I was in film school. That was way before Avatar and all of this like big wave of 3D movies. So it was kind of an odd thing for someone to be interested in 3D. And then I got lucky because I was one of the few people in France who knew how to shoot 3D movies. And then it exploded, you know, we had Avatar, of course, Gravity, and How to Train Your Dragon. And I started working on big movies as a 3D expert, so it's called a stereographer. And then in 2014, I believe, everybody involved in 3D at the time, we knew VR was right around the corner. We had heard about it. We were testing things with taping GoPros together and trying to do tests. There was no stitching software at the time. There was nothing like that. So it was all like using advanced, you know, compositing softwares and trying to figure things out. And in 2014 is when I kind of like decided to take the plunge and really, really trying to understand what was happening in that space. And I started by being a cinematographer on 360 movies and then slowly starting to make my own. In 2017 I went to Iraq during the war and I filmed a project called The Sand Ladies. That's a 360-degree 3D documentary about women fighting against ISIS. It was seven minutes long. And after that, transition to more interactive storytelling with the key that premiered here in Venice, because right now we are at the Venice Film Festival. in 2019 and now I'm here with an even more interactive project called Fight Back and on this side I'm also working on my first video game so it looks like I went from really passive third-person experience all the way to really crazy interactive first-person experiences.
[00:04:41.583] Kent Bye: Yeah, this dialectic between the story and then the interactivity, I think, is something that is kind of the heart of how to combine those things and a variety of different approaches for which one you are more centered in and what you start first with, with the story, or if you start with the game mechanics. So maybe you could take a step back and talk about Fight Back, which is what's premiering here at Venice 2022. And tell me a bit about, like, where did this project begin?
[00:05:09.022] Celine Tricart: Fight Back began a long time ago. It was one of those projects that took years to put together and finance. Actually, the idea originally came from when I was in Iraq in 2017. So when I was filming The Sand Ladies, I was so impressed meeting and interacting with those incredible women. Then when I came back to LA, I had this, originally it was called Wonder Woman, I had this idea of making a doc series in VR about women learning how to fight and defend themselves and taking ownership of their bodies and their strength. And I wanted to make a series about that with The Sand Ladies could have been an episode and then going and finding other stories like that. And over the years that project transformed into a feature film, documentary feature film, and then eventually reverted back to VR two years ago when I had the idea of making it an interactive experience where you get to feel that empowerment and feel that physicality by using the hand tracking technology. So that means that that VR experience is truly trying to connect you to your physical strength and your body strength and making you feel like you have superpower over the course of that. It's about 40 minutes long. So that's just for the general idea. Obviously, this is not the story that's being told, but that was how it all started.
[00:06:36.573] Kent Bye: I think I saw a pitch at some point where it was more about you were going to maybe do interviews with individual women from around the world. Is that one phase? And then what happened to go from that phase to make it more of a virtual experience that is focused on the interactivity and more of a voiceover but also embodied characters and a story arc in that sense?
[00:06:54.991] Celine Tricart: Yeah, no, actually, we have started filming some footage for this. We went to Kenya right before the pandemic because there is a group of women there called the Karate Grannies. They're basically women living in the shantytown of Nairobi. There is this horrible thing that happens where old women are being targeted by rapists because they think they don't have And so when I say old woman, we're talking 80 years old, you know, women that are living in the slums. So they decided to get together and they learned karate. And can you imagine 80 years old women in the slums and just learning karate? And I'm telling you, I've met those women. They are fierce as hell. So we went to Kenya, we filmed them. We started like getting that project going. But then, you know, it's just been extremely, extremely challenging to get anything financed. We keep talking about the difficulties of distributing our work and making revenue for the creators. I am very privileged in the sense that I come from France, and so I have access to government money in France. But even though it has been impossible to find money for that VR series, because everybody at the time, and I'm talking 2018-2019, just wanted to do games, so just wanted to do interactive work. And so we were very close to even letting go of that project. accepting the fact that he was not going to ever have so many projects collecting dust on shelves. You know, this is just what happens. And I was like, OK, this is going to be one of them. And then I don't exactly remember how it happens. But one day I had this idea. I came back to Marie, who is my producer on Fight Back. And I told Marie, I was like, listen, we're just going to have to do the Beat Saber of self-defense. And that's how I described it to her originally. It was like, imagine you playing Beat Saber, but instead of slicing cubes, we are teaching people, we're actually using gesture of self-defense. And so you play a game and you have fun. And at the end, we tell you, by the way, those gesture are just extremely basic. Yes, but they are actual gesture of self-defense. And, you know, if you've been able to play that game, then you can go and continue your training. You have all it takes. And that was kind of the starting point. And that idea actually got traction and we were able to finance it.
[00:09:19.056] Kent Bye: Can you talk a bit about your own experience in self-defense and martial arts?
[00:09:24.375] Celine Tricart: Yeah, so I started martial arts at 16 years old and it transformed me. I swear to God you would not recognize me as a young teenager. I was a very different person. I was very, very shy. I don't even know how to describe how I was, but through martial arts. And you know, it all started because I love The Matrix and I was obsessed with that movie. And so I decided to go and try to learn Kung Fu, literally because I love The Matrix. And I fell in love with Kung Fu. And so since then, martial arts has been a huge part of my life. It changed the way I behave. It changed the way I see the world. It changed the way I walk down the street. There's this whole thing like, I'm able to look at people and I can usually tell if they had not necessarily martial arts but self-defense training, especially women, it makes a big difference to them. So on one end this has been a big part of my life and on the other hand, as we know, there is still a lot of women out there being victims and survivors of domestic violence and sexual violence. And when I say women, Of course there is also men, it's just that the numbers are just, it's like 99% women who are survivors of domestic violence versus maybe not 99 but like 95. So of course I acknowledge the fact that there is also men going through this horrifying situation. But I've myself been a victim of domestic violence and I have had people close to me actually killed by their partners or their ex-partners. And so I was very upset, you know, in the years following the election of Trump, of course. Sorry, I'm getting emotional. And it felt like the whole world that turns against women Sorry. And I was just very upset. I was extremely upset against everyone. Men, women, doesn't matter. The whole situation was very hard for me. And every day you see on the news women being killed, etc. And so that whole project for Fight Back was just a reaction of anger, first and foremost. And then that anger, oh my God, I'm so sorry, Kent. So sorry.
[00:11:49.274] Kent Bye: That's fine, yeah.
[00:11:52.757] Celine Tricart: Very quickly I realized that it could... First of all, you don't want to start a gender war or anything like that. And again, it's not a question of men versus women, but it's just a question about how women are raised, how women are seen, and it's just so unfair. And it's been unfair for so many centuries. So originally it was just a project that was full of anger, I must admit, but very quickly, with the help of Marie, transformed into something that we wanted to be very positive. Like the feeling that we want people to feel once we finish that piece, because it still needs a lot of polishing, When I want to see it happened a lot today and actually was very pleased I want to see people with big smiles on their face. I want people laughing I want people feeling thrill and feeling the strength of their bodies and feeling empowered and for women it's very important that we remember that we are physical being and we are embodied and we are physically present and we deserve our space, you know and We are not some ethereal presence floating through this existence. We are here. We are beings of flesh and bone. And so there is a lot of rewiring that we have to do in our own brains and through self-defense. This is something that works really well for rewiring our brain into this like new version of ourself. And fight back is kind of like that little nudge of hopefully encouraging people to get trained. And actually we had a couple of women here at Venice. Marie was my producer. She was almost crying because we had a couple of women say, oh my God, this is what I need. I want to know more, but I want this. And it looks like it was about somewhere between 5 and 10% of the people here that had that reaction. And I'm like doing the math in my head because I actually have the data. We did a lot of study. I read a lot of study of the effect of self-defense on gender-based violence, et cetera, et cetera. And I know how self-defense training translates into that many less attempted rape, that many less successful rapes, and that many less murders. So if I can have even like let's say five percent of the people saying fight back being intrigued and getting some kind of training and it can be a day or two introduction to self-defense, well this translates into life that will be safe down the line because those never exist. And the goal with Fight Back, we have a whole impact campaign that we're going to take it to women's shelters, et cetera. The goal is to get half a million women through the peace, minimum 100,000 women trained in self-defense. We're going to do partnership with local association of self-defense and dojo, et cetera. And then we know exactly how many lives will be saved down the line by this project. And this is just what I want. I just want to stop killing women, and I want women to feel better in their own bodies.
[00:14:57.235] Kent Bye: Yeah, that's really powerful, especially with the VR piece where you're actually using your hands, and you can see your hands within the piece. And so maybe you could elaborate a little bit on how these three gestures that you're doing are connected to actual self-defense techniques.
[00:15:12.942] Celine Tricart: yeah first a little bit about hand tracking so hand tracking has been a blessing and a curse because right now we're September 2022 and I can say that I think hand tracking is probably one year away. I mean it's still they're improving a lot and it's getting better every time there is an upgrade but oh my god it's been so hard to develop that experience for hand tracking and like keep the faith. Everybody on the whole production people didn't believe it is like Celine don't do hand tracking it's a disaster go controllers and it's like no no no I understand it's not ready, but it will be down the line. I swear to God, it will be. And when it will be, we will be ready. We will have an experience ready. The reason why I wanted hand tracking also is because our target audience being VR beginners, it's very intimidating to have that controller. And I see I've embodied so many people in VR, and you have to explain the button and the triggers and the things. And there is something really great about seeing your own hands and interacting in an experience with your own hands that I believe make it more intimate and also more easy for beginners, when hand tracking will be working. Right now it just makes everything more challenging, but it's okay. Yeah, so let me talk a little bit about the gesture that we found. So we studied a lot, obviously we talked with a lot of experts. Myself, for the project, for PHYBAC, I became a certified instructor in empowerment self-defense. And so we picked a few gestures that we thought were going to be interesting for the game and also relevant for what Empowerment Self-Defense is. Empowerment Self-Defense is not martial arts. We're not gonna do high kicks to the head or anything like that. It's method that works for anybody type, any age, anybody really. Even people in situation of handicap or young kids. So those gestures and those strategies have to be just accessible to anyone. The first gesture that you learn, which is my personal favorite, we call it the circle of protection. And as you basically, you put your hands in front of you in the shape of a triangle. And when you open your arms, it kind of like launches that shockwave that pushes your enemy away. And this is teaching personal space and personal boundaries. The number one thing in self-defense is to identify threat, of course, learning red flags, and also learning to understand what are your personal boundaries. We have all very different boundaries. Like some people you can get right in their face, they're fine. Some other people, they need two meters, you know, to be okay. And so it's all about to find that personal space and establishing those boundaries and being able to communicate those boundaries to others. And that's consent and that's all of those things are very, very important. A lot of people are talking about that today.
[00:18:13.072] Kent Bye: Just to clarify the personal boundaries, is that just when you put your hands out in a triangle and move them out, is that meant to be a contact where you're pushing someone away or is that just to establish like, this is my personal boundary, don't come into this space?
[00:18:27.075] Celine Tricart: It's to establish that personal boundaries. I just want to say a quick note about that. In Fight Back, it's a metaphorical world and you're a star and you're fighting shadows. Shadows don't represent aggressors or they don't represent men or they don't represent anything like that. The shadows in the piece represent mostly your own fears. and over the course of the experience you have that little voice that talk to your head is your friend and it's helping you but very quickly it starts hindering you and basically what you're fighting in the experience is your own inner voice holding you back and telling you that you're not good enough you're not strong enough etc etc which is the number one thing that's limiting well women and so now establishing that boundaries is to understand that notion of personal space and have it respected and communicated
[00:19:15.262] Kent Bye: One thing I just want to add in there is that with these shadow characters, they're coming up into your personal space in a way that does feel fairly intense. You know, as I was going through the experience, it was sort of like a sense of urgency of wanting to get the gestures to work so that they take a step back. But I kind of want to step back when that happens because they are getting really into your personal boundaries. And so there is a way that even though it's a virtual experience and I've done a lot of VR, I still felt like as this entity is coming towards me, It created this sense of urgency or fear that was surprising in the sense of that it's kind of like a thriller in the sense of like most VR games don't kind of play with that transgression slowly into your personal space. It's more about a zombie killer or something where you're at distance. I haven't played all of the VR games that are out there, but this was an experience where I felt this kind of unique slowly approaching terror and you just want to have it get away, but it felt like that increasing tension that I actually experienced in this piece and wanting to get the gestures to work so that they take a step back and establish that personal boundary again.
[00:20:19.479] Celine Tricart: Yeah, and VR is a perfect medium for that because you really feel it. We were actually a little bit afraid that it would be maybe too much for some people. That's what we've been trying to do here in Venice, is we've been trying to transform the experience as a positive experience as much as possible. So we are usually staying next to the person playing the experience and we are cheering and we are clapping and we're laughing and reminding people that it's obviously all just virtual. But the interesting thing with VR is that you quickly, you learn so fast, you know. I've noticed that in my very early days in VR when I went to meet Noni de la Pena and she showed me Hunger in LA and in Hunger in LA, and she had her own 6DoF headset at the time. There was no Vive or anything like that. And I remember I was playing Hunger in LA and in the piece there is walls and of course they don't exist and I was just fascinated by the idea of like going through walls and I was jumping through walls during the piece and it was just like, oh my god, I'm going through walls. And I don't remember how long is Hunger in LA, 15 minutes, 10 minutes? I don't remember, but anyway, remove the headset, talk to Noni for like a few minutes and be like, great, okay, thank you. And I turned and I literally walked into a wall and I swear to God, I smashed my face on a wall. Because in those 15 minutes of VR, my brain has learned that it was okay to walk through a wall. And when I turned, I wasn't consciously thinking about walking. I didn't think, I was just, okay, I'm going back to my car. And my brain took me to a wall and I smashed my face. So that's also when I was like, ooh, wow, VR can be so dangerous. People will jump through windows and stuff. But also this is when I learned that we can learn quickly. And so the idea behind Fightback is to build that muscle memory, but it's not an actual muscle memory because you need a bit more than 40 minutes for that. But to bring those, wire those brain, wire those neuron in your brain of like making a gesture enough So you're supposed to repeat the gesture many times in the experience that your brain will know it. It's there somewhere. And if you continue your training, it will be reactivated. And those are all skills that will come back very quickly to you. I mean, that's the hope, at least. So establish boundaries. Another gesture we have is a very traditional crossing your hands in front of you, like crossing your arms, that creates a shield of light in the experience. That's just to learn to protect yourself. If you're under attack, just number one thing is to protect your head. This is more like a training into reflexes because we need to do in the experience, you need to do the shield at a specific moment in time before an attack and like time it right.
[00:23:06.595] Kent Bye: Just to follow on, because when I think about protecting your head in the boxing stance, you kind of put your hands and your elbows up to protect your head like that. But in this, it's more of a cross in front of your body. So you're just saying that you're not actually protecting your head. You're just trying to train reflexes.
[00:23:21.220] Celine Tricart: Yeah, you're training reflexes and you're training to block and understanding. It's all about that feedback loop of when it works and you're actually blocking an attack, you have that moment of, oh, I can do it. Versus when, you know, you are short or you're skinny or you're, you know, weak or you're young. and somebody comes and impresses you, but because it's a different body type, then it's just about learning that idea that you can block, you can do it. When you train in martial arts or self-defense, I've seen women in tears, you know, after this sparring match, and realizing that, yeah, they can take a hit, and they can also give hits, and they can defend them, and just having that thing of like, oh, It's really hard to describe how somebody who's never experienced it and who's never been raised in the idea that they can fight, in a sense that they can defend themselves, when they realize they actually can, there is a switch that goes on your brain. it's quite impressive to see. It's very impressive to see. We had that a couple of times here in Venice and when you see it happen, just like the way they place their body and they handle their body, it transforms them, you know. Hopefully it has a lasting effect. That's the only thing I don't know at this moment in time is when they experience fight back, if it works on them, will it continue working in three months, five months, six months? I don't know, but that's the hope.
[00:24:54.066] Kent Bye: And the third gesture is the actual punching gesture.
[00:24:57.252] Celine Tricart: Yeah, it's punching gesture and in Empowerment Self-Defense, we learn to identify weapons on our bodies because we are full of weapons, you know, like the knuckles, the elbow, the knee, etc, etc. And we learn to identify targets, the aggressor's bodies, and usually we say hard on soft. So, for example, obviously a knee on a groin, that works really well, or elbow on the jaw, for example, that works very well. Obviously, we can't teach all of that stuff in the VR experience, so we pick something very simple, which is a punch. The only thing that we teach in the VR experience is that there is weak points on enemies, and then you just need to be targeting weak points. Or enemies have two weak points, there is one on the head, one on the belly, and you just alternate between the two weak points. But there is an actual fourth gesture hidden in the game and that fourth gesture is what we call the power stance. The power stance is kind of like when you put your body in the shape of a star so you know you raise your arms above your head and you kind of like take a big lot of air in and you have that like position that if you hold it for like a few dozen seconds or a minute it really makes you feel great and makes you feel strong that sometimes people do that before important meetings or pitches just to like give a little boost so if you do that gesture in the experience you actually grow into a giant, so it makes you change your scale in the VR experience. So that effect that this position has on the mental states, we actually reflect it into the physical state in the VR experience.
[00:26:43.132] Kent Bye: Yeah, yeah, it's a nice feeling to be able to change scale like that and Yeah, as I think about this piece I think about there's the game mechanics and then there's also the story where you kind of start off with Setting the context in this particular piece of being with the stars and having this guide Telling you each of these different steps and then you're learning the mechanics as you progress through and the mechanics get a little bit more complicated as you combine them or more of them that are available. And so on a piece like this, do you start with the story that you want to tell and then add on the game mechanics? Or did you start with the game mechanics first and then figure out how to sort of build the story around that?
[00:27:24.252] Celine Tricart: In this case, the general game mechanics were first because it was very important for us to have those gestures, et cetera, and this like physicality and the hand tracking. But the story was kind of like right there. I mean, when the idea came, it was our materialized idea of like, OK, we're going to do a metaphor of light versus shadows, and you are a star, and you're fighting shadows, and you liberate the sisters from your constellation. So there is five sisters, so there's five levels in the game. At the very end of the game you discover the true identity of your sisters and the sisters are actual women that existed in history and one of them actually is still alive and I've met her and she's awesome and those are all women who did incredible things but mostly by using their own physical self and their own bodies to fight back. for various things. They all fight back for different reasons and in different contexts. And kind of like the message at the end is that, you know, there is that many stars in the sky. There's a lot of them that you don't see. Well, it's the same for exceptional women. There is some you see and there is a lot of them you don't see and they're out there and they've been erased from history, sometimes on purpose, sometimes they've just been forgotten, but there's lots of them. And the reason why there is that little thing at the end about the reveal is because I think there is also a lack of role models for women, not for women, actually for women and men, but there's a lack of female role models when it comes to physical empowerment. Usually people always think about the superheroes, okay, Wonder Woman, everybody thinks about that one. And it's always a character that has some kind of superpower that makes her different than us. So there is always that like, oh, she can do it because she's like an alien on a different planet. Oh, she can do it because blah, blah, blah. And so we wanted to share some stories of actual human beings. who, you know, were alive on this planet and were awesome and had also, for example, I'm going to talk about the Karate Grannies, which is those women in Kenya I mentioned earlier. Well, you know, if elderly women living in extreme poverty in Nairobi can do it, then maybe we can do it too, you know, like, They don't take the excuse of age or means or where they were born and where they were raised. They decided to do it and they did it. So those stories are just to also empower the audience into saying, all right, all right, I can do it. Yes. And they are the new role models that I'm offering.
[00:30:06.419] Kent Bye: Yeah, I really enjoyed the ending of this. And I think it made the whole journey through all those mechanics in each of the stages really have a really nice payoff. And I was surprised to hear of a suffragette who had jujitsu. I didn't even know jujitsu was around that far back. So yeah, that was a fun story. And yeah, here at the festival you have a couple of off-boarding processes where you have cards and you have things you can put like a little badge on your tag once you go through the whole experience. And so yeah, you have these icons or women that you can look up to that have, to a certain degree, fought back. And I don't know if there's any divinatory aspect of what it means as a symbol for if you pick a certain woman. or a certain one of the character icons that you had selected. But you have a little bit like a Tarot-esque aspect there, where you have these different characters and you're drawing one of the cards. But yeah, maybe you could talk a little bit about the off-boarding process of this experience, because in the festival context, you have the ability to introduce people into the experience and off-board them. So yeah, maybe both the on-boarding and the off-boarding of what you're trying to prepare them as they're about to go into this, and then help unpack it as they come out.
[00:31:14.462] Celine Tricart: Yeah, that process of in-boarding, out-boarding is very important to me. That's why it's such a luxury to do it in a festival setting like this one. That's actually something that I started with The Key in 2019, where there is that slow transition from the real world to the virtual world and then back from virtual to real. Because I always thought it was such a violent act to welcome someone and throw a headset on their face and be like, all right, go. And you're like, where do I go? So just having a little bit of it's always I defined as a kind moment of like taking somebody by the hand like okay This is what's gonna happen. I'm gonna try to make that transition as Respectful as I can so here the inboarding is actually pretty much focused on the hand tracking because that's the most challenging part of our piece and Where we explain the gesture to the people before they go in just to rehearse the gesture a little bit So then they have the correct motion when they start the experience Actually, it's been we're on like day five or day six of the festival and we've changed that in boarding pretty much every day today it feels like it's refined enough that Nobody today had any big issues with the hand tracking and the piece, which is the first day it's happening. So I think we've gotten to that point where the inboarding is set. And the outboarding is very important because we need people to feel rewarded because it's a challenging piece. I mean, it's pretty physical and it's very stressful. We try to encourage people to keep breathing and slow down and smile and be OK during the piece. But at the end, we still want to be OK. How do you feel? And then we gave a little pendant for the key. We gave a little key at the end of the experience, now we're giving a little star. I'm hoping to continue doing this and maybe in 10 years people will have 10 pendants from all my VR pieces and I'll give them a trip to Hawaii if they have all 10 or something like that. It's not a collector's edition, but anyway. And then, yeah, you have those little postcards with just the name and the image and a little quote from one of those historical characters that you meet. And the idea is that you don't keep that card. The idea is that, all right, this is a card of a character that is important to you or that you like. What about you find somebody in your life that need a boost or need a little word of encouragement or need a role model? And what about you pass that card along to that person? And just continue spreading the word about the new role models and the benefit of self-defense.
[00:33:54.572] Kent Bye: Yeah, and coming into Venice, I flew in on a red eye from Portland, Oregon, and I arrived here. And I had the press preview starting at noon. I landed in Venice at 9, so I had three hours to get here. And so I come in and fight back is the very first experience that I see. And after a few days, there was some fixes to make the hand tracking work a little bit better. But I had a little bit of a nightmare scenario with the hand tracking. I don't know, it's probably just environmental conditions and other things and maybe some bugs, but there's a number of things that, like, and also I may be like Wolfgang Pauly and there may be a Pauly effect where I just, something about me that is instigating bugs, because there was another experience here at Venice where some game-breaking bugs happened twice in a, like, 40-minute experience. at the end, so I had to do it again. But it sounded like that there was noticing that there needs to be high contrast or putting a black carpet on the ground and an infrared light. And maybe you could just talk a little bit about the temperamental aspects of hand tracking and lighting. And I guess as a challenge, when you're delivering a piece like this, it may work in your office. But if it's in a different environment, there may be environmental issues that are coming up. And you seem to be learning about some of those while we're here at Venice.
[00:35:14.138] Celine Tricart: Yeah, I mean, honestly, I'm going to try to describe some of the findings that we have, but there's still a lot of unsolved mysteries. I think hand tracking is kind of like this thing where sometimes it works great and you have no idea why. And sometimes it just doesn't want to work. And we had that person last night, and one of the gestures would just not work. There were three that were great, and one of them, impossible to just use it. And then I grabbed the headset. I was like, let me check. and I put the headset on my face and I could do all the gestures perfectly. And I was like, okay, what's going on here? It's a complete mystery. But here are the findings. Hand tracking uses those camera on the front of the Quest, especially the two camera facing down. So the number one rule is to keep your hands visible from those cameras. And the other thing is that it's very good at understanding position of the fingers, but it is not very good at understanding if you do any fast move, from close to far so for example or punching you know we go from close to the headset to far well it doesn't understand that depth so if you go really fast when you try to punch which is what everybody wants to do because you're attacked by enemies and you want to punch fast well the hand trackings go completely wacky so we have to explain people you have to hit slowly which is counterintuitive But you know what, we transformed this into a training in patience, a training in self-control, and like breathing exercise, go slow, think. That's how we kind of like change our embodying in that way a little bit, make it more of a like mind control the body kind of thing.
[00:36:57.446] Kent Bye: It's definitely counterintuitive when there's a big shadow creature getting into your personal space and you want this creature to get away from you and you try to do it as quickly as possible and it is counterintuitive to be like try to make this gesture and to get it to register which having the triangle you have a little star that says okay you at least registered it. And then the second part is you have to put your hands away from yourself. And so it's sort of like kind of a swipe gesture with both hands in opposite directions. And so you kind of have to maintain this tracking of that star. And what has happened for me is that star was disappearing. And so I couldn't actually complete the gesture. So I was trying to do it fast and trying to get it to register. But yeah, when you're in that moment, my immediate reaction was to try to do it as quickly as I can to get this creature away from me. And especially with the punching part, you know, punching slow is also counterintuitive. But yeah, I guess that's the instruction is to kind of do it more of a Qigong or meditative type of movement, or how do you describe it to people?
[00:37:55.560] Celine Tricart: How do we describe this to people now? We just say, focus on your gesture, don't do it too fast. And if you do the gesture well, actually it's very easy to keep the shadows at bay. Because once you've done that circle of protection, they're all pushed away. And then you can think, you look at the next shadow, so I'm going to get this one. And then if they all come at once, well, you do it again. And once you've gotten that gesture down, then it actually gives you way more time to be in the moment. But to be perfectly honest, if we want, we can just switch to controllers. The experience works great with controllers. And then you can kick and punch and do it as fast as you want. But we've been resisting doing this because it feels too easy. I mean, it's just not what the piece is about. If we want people to not have any trouble with hand tracking and just go through the piece and get the story and go home, then we should just give them the controllers. But we made the decision to make it more challenging and complicated but we want people to have that hand tracking thing and tracking experience. When you achieve a new skill you have those little symbols that appear and float and they like come and touch your hands and they appear on your hands and it's a very nice feeling to really feel those skills coming to you so it was important to keep that. the mistake we made with hand tracking that we're going to correct for the final version is right now there is too many if conditions for a gesture to be validated and i think that's a problem so for example the circle of protection you mentioned the ifs are you need to have two hands flat with fingers glued together joints in the shape of a triangle and then once that symbol appears you need to wait one or two seconds and then you need to slowly open the arms and it's like there's too many if conditions for that gesture to work so we're going to remove half of them. So the danger in that is that this gesture might be detected wrongly and happen when we don't want it to happen. So we need to find that middle ground. Same for the shield. The condition right now is closed fist and then crossed. Well, really, we just want to cross. So we're going to remove that condition of crossed fist. And if you leave your hands open and you cross your arm to protect yourself, it should still work. So right now we're in the process of removing a lot of those ifs, all the way to when it becomes a problem and the gesture is just happening without you wanting it to happen. So it's a balancing act, but we've been working on it. And actually for the bigger version of the piece that we are working on, we are collaborating very closely with the Meta hands team, because they obviously are very interested in that experience. There's not that many hands first, we call it hands first, in an experience that is meant to be played with hand tracking and not controller and maybe hand tracking on the side. So we will be working hand in hand with the hands team at Meta to try to make those gestures work better.
[00:41:07.655] Kent Bye: Yeah, it's hard for me to comment too much on the mechanics of the game because there were a number of bugs. But one thing that I would have liked to have seen was potentially, I mean, maybe when these other fixes are implemented, it's not necessary, but at least to have some indication of a hit bar of how far you're progressing. Because I had one bug where I was sort of doing and doing and doing, I must have done at least 15 different protective safety shields and another 10 shields up close and it didn't die and end up being a bug. But just to know kind of where you're at along that's it and that's one of those things where if I don't know if the progress is happening and if it's actually working, then it can get frustrating because it feels like I'm doing all these repetitive behaviors but not seeing anything of progression and then it just feels like I'm repeating the same thing over and over and then sometimes if it has trouble detecting it then it's another thing of just feeling like I get frustrated with the core mechanic of the piece that it just doesn't feel satisfying after a certain point if I don't feel like I'm making progress and so I feel like some indication within the character or within the environment to show me that I am moving forward and then that there's a an end goal that I'm getting towards because some of those levels with multiple characters, it was really difficult to know where I was at and almost wanting to nope out and be like, all right, I've had enough. But I still wanted to finish to see the ending, and I'm glad that I did. But one of those things where giving some feedback to the user that whatever they're doing is working on some capacity. So I don't know if that's something that's come up.
[00:42:39.860] Celine Tricart: Yeah, this piece for me has been one of the most, I think I've made the most mistakes on this piece that I've never made in my previous project and I've learned so much but also because of that ambition that came with the piece. So the two main things that we need to fix that we realized were big problems is one that you touched upon which is the lack of feedback So that was a choice not having, we didn't want any interface, right? I don't want a life bar, I don't want scores, I don't want it to look like a video game at all. But what we're working on for the next version is we're gonna redesign the shadows and when you hit them, so when you have a successful hit, they will lose a piece, so they will actually shrink. And so when you look around, you'll see big shadows, small shadows. And you know, OK, I'm going to hit that small shadow because it's almost dead. And then you can strategize how you're going to take care of the enemies. And you'll have actual feedback of your hit. So that's number one. And the other mistake that I made right now, it has me coming from cinema. And I'm still kind of stuck in that world where I want to tell my story. And I want people to pay attention. we have moments where the powers are activated which is the tutorial moments and the fights and the rest of the time when there is cut scenes and there is talking etc we deactivated those powers and i think that's a big mistake because a lot of people actually need that time to practice to like okay the shield how do i do the shield and they try to and it doesn't work and so they think it's a bug or they think it doesn't work because they're not doing it well and they don't quite understand in video game when there is a cut scene that takes away your agency it's very obvious the camera changes It's super obvious it's a cutscene, there's nothing you can do. In VR, it's not obvious at all, you're still yourself, you're still there. And it was my mistake to say I'm gonna do a video game cutscene, well it shouldn't exist in VR and that's not how it works. But I wanted as a storyteller that people stop playing around with the powers and pay attention to what's being told to them. So those are the two main things that I need to fix before we release the piece.
[00:44:55.765] Kent Bye: Yeah, I think especially at the beginning because I had that experience of being taught the first gesture and then it worked and then I was like, I'm fine. And then actually when I tried it again, it wasn't working and then I was like, oh, and then when I got into the actual fight, it still wasn't working. So then I had that impression of like it was broken the whole time. So yeah, having it active to build that confidence and to do it multiple times and not just like do it once and kind of move on. There were some other changes you made to the environment, adding an IR light and putting a black carpet, and there was other things, some other lights. I don't know if there's other things you figured out in terms of optimizing hand tracking.
[00:45:30.818] Celine Tricart: Yeah, sorry, I forgot to mention this. Originally, we had white curtains all around our booth with white carpets, and that was a complete and utter disaster. The hand tracking was completely lost, so we swapped that for a black carpet. And also, we opened those curtains, so then the headset would be tracking better and understanding the space better. And then to take it to the next level, somebody here from the Venice technical team, they actually brought us a little infrared light, because the Quest 10 tracking and tracking is based on detection of those cameras that you see on the headset, especially using the infrared wave frequency. thank you and so adding extra infrared was helping the tracking a lot. This morning I found a new setting that I felt really improved also tracking but maybe it's all in my head who knows is you can change the frequency so if you're in Europe here it's 50 Hertz electricity is based on 50 Hertz and in the US it's 60 and That matters when you are indoors because all the lights, all the practical lights or all the lighting in the piece are based on that frequency. And so the headset tracking by default is, I think, automatic. So it will switch, it will look for 50 or 60 and it will switch. This morning I put it to 50 Hertz and it seems to improve a little bit.
[00:46:58.705] Kent Bye: Oh, so it was on 60 for the last few days?
[00:47:00.887] Celine Tricart: Automatic. Automatic. So... So yeah, those are all the things. The other things that we haven't tried, because here people attending the Venice Film Festival on the island happen to be very white, but there is one big mystery to me is how does it behave with very dark skins. In LA, I know I tried it with some of my friends with dark skins and it was fine, but there is some people out there saying hand tracking is finicky with people with darker skins. So that's something else that we need to test because it's quite important that nobody gets discriminated upon.
[00:47:36.193] Kent Bye: Yeah, and that may be on the meta side to be able to do the training of their algorithms on the core services that they're providing there. Well, yeah, I guess it sounds like you're going to go back and make some changes. What's the next phase for you as you move forward with this project?
[00:47:52.061] Celine Tricart: Yeah, so the goal is to release it for free on the Oculus App Lab by the end of the year. So our number one priority is to go back to our partners, which are France Television. Well, that's it, because the rest of the funding came from French government, the CNC. We have a little bit of funding from FNC, the Festival Nouveau Cinéma in Montreal, and the Région Ronalpe, which is a region in France, that also we got a little bit of funding from them. We're going to try to raise, because right now we're completely out of money, we're going to try to raise additional money. We'll see if we can, and hopefully get all those fixes implemented before we release it. It's very important to me that we get the best possible experience to the people out there. But at the moment it's just a difficult question because everything costs money. We finished that piece a couple of weeks ago and technically there is not more we can do. So we'll see, we'll do our best.
[00:48:52.359] Kent Bye: Awesome. And finally, what do you think the ultimate potential of virtual reality, immersive storytelling, and interactive gaming might be, and what it might be able to enable?
[00:49:04.349] Celine Tricart: You mean in the context of self-defense and empowerment?
[00:49:07.672] Kent Bye: Yeah, or just in general, or just where you think this is all going.
[00:49:12.339] Celine Tricart: You know, I've seen a lot of my friends who have, in the past year, who have zero connection with the VR world of video games or anything, actually buying Quest 2. Usually it's to play Beat Saber or Vader Immortal or Supernatural, which I think is awesome. And now those people, this is a brand new audience of people who just want more and they are very curious about stuff. So it's interesting. I think our audience has changed. When I look at the data for the key that has been on the store for two or three years now, it's still going strong. We have over 800,000 users. And what happened is the morning of December 25th of last year, so the Christmas morning, I had a giant uptick of female users. so they're probably not all women they might be women connecting their facebook account so their kids can play the quest you know so i'm sure those numbers are not exactly all female but the crazy thing was i had maybe 23 of my users being women on the key and Since the day after Christmas, all the way to now, it's 50%. So big, big, big influx of female users, supposedly female users. And that, I think, is pretty encouraging. And I'm very excited to see how women will embrace that sector and what they will do and how they will feel by experiencing everything that's available right now.
[00:50:47.542] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:50:52.641] Celine Tricart: No, I think it's been very challenging, it's been very interesting, but I'm really grateful that we did fight back and we did hand tracking, and we're going to continue to try to improve it. And I think the future is hand tracking, honestly. Yes, it's complicated to play games and interact with other people when you don't have a lot of buttons and stuff to use. It is challenging, but it's not impossible. And I see a future where there will be less and less physical separation between ourselves and the virtual world. So I'm really excited to see how other creators will use hand tracking for their own piece.
[00:51:31.386] Kent Bye: Yeah, and I think one of the things that I take away from being at a lot of these festivals, sometimes I feel like a QA tester on a number of different experiences, not just on this piece. But there's a certain way of when you're on the bleeding edge, then there's things that don't always go right. And for me, I was the first in line, and I think over time it gets better. And I suspect that this is just the process of being in VR, is that things can be in a process of getting better and better. And hand tracking, like we're talking about here, feels like, you know, you said that it's two years in and maybe another half year or a year to get to the point where it's going to be really solid. And I tend to prefer using the controller because most of the times the hand tracking has been so finicky. So I tend to avoid it. I don't tend to do a lot of the games with the hand tracking. But this is a type of experience that I feel like it starts to make sense because it's built into the Story, but it also gives me a deeper sense of actually being embodied in my body in the same world as these entities And it gives me this deeper sense of that embodied presence and a little bit less abstract in that way because I can see my fingers moving and actually there's another piece here called a Lele where I had an experience of phantom touch from Feeling such a deep sense of presence embodied presence so I think there's actually going to be a lot of really interesting things as we move forward with hand tracking and Yeah, it's sometimes difficult to be on the bleeding edge, but I have faith that you'll be able to persist and figure it out. And when it comes out, it's going to be unlike anything else that's out there, because there's probably a lot of other people who have not been willing to help fix all those bugs that are out there. So I'll also just say I just really enjoyed the narrative aspects of fight back because it feels like you do go on a journey. And there is a very satisfying ending scene where you get to learn about all these amazing women who have been using different techniques of martial arts to be able to fight back. And it's just really cool to have an aggregation of those and to go on this journey. So again, congratulations for finishing it and for being here. And thanks for sitting down to help unpack it all.
[00:53:28.016] Celine Tricart: Thank you so much, Gens.
[00:53:30.507] Kent Bye: So that was Celine Charcart. She's the creator of Fight Back, which was showing at Venice Immersive 2022. So, if you want more context for the wrap-ups, then I'd recommend checking out the episode 1121, where I talk about all the 30 pieces in competition. And in episode 1144, there's an immersive panel that I did at Venice with some other immersive critics talking about the art of reviewing immersive art and immersive entertainment. I recommend checking that out in order to dig into a little bit of my own process of what I'm trying to do with these larger series and trying to unpack and discuss the art and science of immersive storytelling with a lot of these different pieces that we're showing at Venice Immersive 2022. So that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
