I spoke with producer Victor Agulhon of Targo Stories about their collaboration with TIME Studios to produce D-Day: The Camera Soldier, which he calls the most technologically advance immersive documentary produced up to this point. They took footage from D-Day and spatialized it in a number of different ways, and really pushed the frontiers of innovation for spatial storytelling. It released on the Apple Vision Pro last Tuesday, and it’s definitely a project that you should check out if you have access to an AVP.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.458] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of special computing. You can support the podcast at patreon.com slash voicesofvr. So in today's episode, I'm going to be covering one of the best immersive stories that I've seen on the Apple Vision Pro. It's called D-Day, the Camera Soldier, and it's by Targo Stories, who've has traditionally been super innovative and using all the different volumetric storytelling techniques and they've combined all those together to recreate different scenes from the D-Day invasion they've take you onto the boat they've got the footage and actually are telling a really powerful story of a daughter who finds out her father actually was at D-Day and shot all this iconic footage And so it's about her discovery about who her father was, but it also uses that personal story to tell a much broader story of D-Day and taking what's ordinarily a 2D film and then translate it into a whole spatial and interactive experience. So we're covering all that and more on today's episode of the Voices of VR podcast. So this interview with Victor happened on Thursday, May 22nd, 2025. So with that, let's go ahead and dive right in.
[00:01:21.209] Victor Agulhon: Hi Kent. First, let me start with saying that I'm happy, you know, we're having this conversation. We've had them, you know, a few times and it's always a pleasure to chat with you. So I'm Victor Agulon. I'm the co-founder of Targo and a producer of Immersive Experiences. I've been doing this for the past eight years now. You know, we're slowly going to 10 years. And what I love about this and, you know, what we've been focusing on for all this time is really trying to make this technology blossom into the mainstream, right? And how we can, through stories, make people connect more with the technology and how we can use the technology to make people connect more with stories. So it's really this kind of conversation that we've been trying to encourage with this company and with all the projects that we've been working on.
[00:02:04.172] Kent Bye: Great. Maybe you could give a bit more context as to your background and your journey into the space.
[00:02:09.031] Victor Agulhon: Absolutely. So Tago really comes from when Chloe Rocher, who's my co-founder and the director of all the documentaries, we started to work together really on this. At the beginning, I think it comes from her vision that she wanted to make people, you know, she was a journalist at the beginning and she wanted to bring people with her on the field. And I was always interested in technology. I had started first experiments with VR, first small ventures with VR. And I went to see her and I showed her the technology and it just like came together very nicely. for the both of us. And so for the past eight years, my role has been to be the producer of all the documentaries that we do. So we focus only on virtual reality experiences, right? So things that you watch in a headset, I think it's always been the central element for us. and on documentaries, because we think that this technology is such a powerful bridge between individuals and between stories. And so my role has always been to try to find the right people to work on the projects, find the right technologies, and find the right amount of funding also to make them possible to do. And so across all the more than 15, even 20 documentaries that we've produced now, We've had the chance to work with Meta a lot on productions and with lots of partners, Time Studios for this new documentary. And one of the key elements that I think has been a recurring theme in all of the projects that we've had is the relationship between time and space and something that's almost like philosophical in a way. But we love the idea of how we can transport people to different times and to different places with immersive technologies.
[00:03:40.717] Kent Bye: Yeah, and there's a number of pieces that we've talked about over the years and a couple that come to mind in terms of 9-11 and then also the JFK documentary that you did in Immersive. And so each of these are taking archival photos or footage and then adding stereoscopic effects and CG components where you're able to go into the places or taking maybe a monoscopic 360 and making a little bit more stereoscopic, but kind of blending together all the tools and the tool set from taking archival photos, giving a stereoscopic twist or more of an immersive. And this documentary of D-Day, The Camera Soldier, there's some more interactive components. And so maybe you could just start from with D-Day, The Camera Soldier. Where did you begin in terms of this as a project and how do you see it evolving from these other projects that you're also working on?
[00:04:31.906] Victor Agulhon: Yeah, absolutely. I think what you're describing, so we've done a lot of historical documentaries. So I think you mentioned the 9-11 documentary. We told them the story of the last survivor, the Notre Dame documentary, the JFK experience, also where we tell really the events minute by minute. One thing that's absolutely passionate when you work on these projects is how do you transform themes that everyone knows, footage that everyone knows, scenes that everyone has seen on TV documentaries, in the movies. How do you make them relevant with the world of VR? How do you make it relevant in immersive? And what do you bring that's radically new, that's worth putting on a VR headset and dedicate 20 minutes, 30 minutes of your time into an experience like this? For us, there's been something that's always very VR native in that way, which is looking at a 2D footage and trying to understand what it would mean to be in this footage with a headset and to physically be present in the scene. There's something that's almost chronological in the way we've done this. When you look at our first documentary, it was Notre Dame. And then we went on to the 9-11 documentary, so 20 years after. And then we went on to the JFK documentary, which was 60 years after the event. And now we're back with a documentary on the Second World War, on the end of the Second World War, on the D-Day landings in 1944. And now we're 80 years back. And so it's always been an absolute honor to work on all these themes and to find a way to make the stories that they hold feel very relevant now. And I think that's really what the core fashion lies here. It's really about you look at these old dusty pictures, all of these materials that look aged, which seem difficult to connect with. And with the right treatment, with the right approach, with the right story, we can make it feel like it's here and now. And so the D-Day documentary is very much about this whole journey. But I think it even goes further than this. So just maybe at a high level for anyone who's listening is the D-Day documentary is about the story of a woman, Jennifer Taylor Russell, who discovers very late in her life, so when she was around 60, that her dad was truly involved in the D-Day landings. It was something that was part of the family story, but it was never clear what he had really done then. When she was 60, she receives a Facebook message from a historian who reaches out to her and asks her if she's seen the footage that her dad has taken. And so she had never seen it. And so she received that Facebook message and she clicks on that link. And then she sees for the first time at 60, the footage that her dad had taken on D-Day. And she had seen tons of documentaries around D-Day, but she didn't know that all of the clips, all of the photos that she had seen, were actually her dad's and so it sends her on a quest of trying to reconnect with her dad with his story understanding you know why he was the way he was and so the documentary really blends jennifer's story as an immersive journey of you know trying to reconnect more and more with her dad and getting more and more involved into his memory and you as a viewer getting more and more immersed into jennifer's story and so physically the more you progress into the story the more you get back to the past the more you know the window of immersion expands and you start with a 2d frame and you end up then with interactive scenes and then you are in fully immersive video and then you're in fully cgi reconstructed scenes and it's really that journey which i think it's the first time that it's been done to this level where the technology blends with the story to this point and it's a story about immersion that uses immersion as the storytelling platform and so I think for us on this idea of like remastering archive footage, this was also the perfect story because her dad was a cameraman during the DDL endings. So he filmed the DDL endings and it was just, everything just came together very perfectly in terms of the right story, the technology and the character. So it's really a project that I think is the most, you know, polished in terms of why it belongs to immersive media. It's the one where we've thought through the most, you know, why it belongs there. Yeah.
[00:08:23.103] Kent Bye: Yeah, and so I know that you had more of a tech demo of this project that released about a year ago where I had a chance to see it and we may have even recorded an interview about it, but it kind of got into the vaults because you went back and added a bunch more. So I just watched the final version that's gonna be releasing next week by the time this episode comes out. And it's got a whole story that is much more developed and the way that you're integrating everything, you're kind of pushing the edge on a lot of different fronts. And so I'm just curious to hear a little bit about like what happened from the first initial tech demo, the proof of concept that this would be like a possible idea to use the Apple Vision Pro to start to do these kind of more photogrammetry slash CG scenes that's adding not only the stereoscopic effects to 2D photos, but to translate them into full 3D scenes and then add the whole story arc that ties it all together. And then since last time we talked to you, you have Time Studios that have come on board to help bring it home as well. So maybe you could just take me back to the initial demo that you were doing and then everything that's happened over the last year to really tie it all together.
[00:09:28.086] Victor Agulhon: Yeah, I think you've seen, you know, the whole arc of like, you know, creating a project. We started thinking about this project, you know, probably two, three years ago. After JFK, the JFK Memento documentary, I think we settled on the idea that we wanted to rely on the archive footage and the archive materials. So we started pitching around, starting to think about what a prototype would look like. During that process, I think it was in June, there was the Apple conference and Apple announced the Vision Pro. And so we looked at this new project that we wanted to create and we looked at this new device and we were like, it's probably a good way for us to try to get involved with what this new platform allows us to do. And at core, you know, when we look at a device like the Vision Pro, it's a device that fits exactly what we think the technology should be used for. It's a sit back and enjoy kind of experience, super high quality, very much focused on the video side of things, targeting, I think, more mature audiences than other platforms which will target more gamers. So it just seemed like a lot of elements were coming together for us and which seemed like the Vision Pro would be the right platform for us to target. We started by putting together a tech demo, which I think was more of a tech demo. It was a very loose storyline, which was already around the camera soldiers, right? But it was about all of the soldiers who took photos and even journalists who took photos and videos of the detail and things. And the goal was to go between, you know, from the perspective from one to the other and so on and so on. And so that's what the tech demo was about, was getting our hands on Vision Pro, seeing what was the response out there. And even as close beta, we were able to get some pretty good feedback. And so what we got from this was, from the tech demo we released a year ago, was there's a huge interest around this kind of media for Vision Pro. Even though the device was very new, very few people have it, it still felt like there was much more excitement. And to speak in product terms, it looked like product market fit. It looked like you have a documentary which can reach the right audience because it's on the right platform. And so after we released that first demo, we're like, this is something we need to go through. This is a project that we think can be a real game changer to show what documentaries can be on Vision Pro. And we just committed to it. We just published the tech demo. We're like, this is something we want to go for. And we went for it. And so that's really what the journey was. We release the tech demo, we see how people respond, and then we commit to it. And in the process of committing to it, we realized that we wanted to focus the story on one character because it's always what we've done in our past documentaries is we try to focus on one very individual story. And we got in touch with Jennifer. She was super interested also in, you know, finding ways to bring her father's story to life. And everything just came together also super nicely here. So we were able to get some funding, also public funding, because we are a French-based company. So we were able to get some public funding for it, which helped significantly into making this project. And as it came together, we were presenting it to different partners all across the world, whether they're tech companies or media companies. And in that process, we reached out to Time Studios and we presented them what we had. Time Studios had a connection with Richard Taylor's footage because the Time magazine in 1944 had published the photos. And so we presented them, you know, the whole story. Time magazine has, you know, having published the footage, the relationship that we've had with Matthew and Lauren, which has been, you know, just a great relationship because we're interested in the same ways, in the same stories. Flo is a journalist, you know, so it just kind of makes sense to work together. And then that's how we started working together on this, which will give this project a massive amplification. I think, and help us, you know, achieve more of our vision of trying to bring this project more to the mainstream and pushing immersive technologies to the general public. So when you look at the tech demo and when you look at what we have now, it's a completely different story. Even the tech stack is completely different. There's been so many innovations and, you know, so many new things that went into this, which is really insane. Like in terms of the complexity and all of the things that we had to achieve technically and narratively, it's been... Incredible. And so it's been an amazing journey, honestly, putting this project together. And we're super excited now to have it finally available next week on Vision Pro.
[00:13:49.352] Kent Bye: Yeah. And one of the big themes that was coming up at South by Southwest this past year was that, you know, this difference between 360 video and 180 video and how the Apple Vision Pro is really focused on more 180 video experiences. And, you know, there's some of the 360 videos that are telling the story of a place. And I know that Target Stories kind of started with the 360 video, but you've already been working with 180 video for a while. And especially, you like for Apple, there's the Apple immersive video, which feels like it's fully immersive 180. And then there's the more spatial video, which is in a windowed frame that has some stereoscopic effects. And so I feel like in this piece, it's a little bit more pronounced this switching back and forth between that more framed stereoscopic view versus like the more immersive 180 view that may or may not be Apple immersive video. I don't know if it's, you know, using that format or if it's just 180 video that feels like a proxy that seems very similar, if not totally equivalent. So maybe you could talk a little bit about this contrasting between like the 360 video and 180, and then also this kind of framed spatial video that you're switching back and forth between.
[00:14:57.745] Victor Agulhon: Yeah, I think what's super interesting with this industry is that, you know, every two years, there's like a complete direction switch in terms of what is the right thing to do. One thing that we are super proud of at Tago is that we've always been sticking to what we've wanted to do. And now we're happy because it seems that things are going in the direction we've always wanted them to go. So what I mean by that is we started with 360 videos because there's no way to capture the emotions as well as a video. I don't think today we have any better technology to capture human emotions than video, period. And so for JFK, Memento, our project was mostly based in fully CGI reconstructed environments. And we tried bits of video here and there, but the reality is technically it was extremely difficult to do. So we couldn't really get to the resolution and the quality we wanted to have. What Vision Pro allows for us... is the ability to switch from one media to the other at peak quality extremely easy. I mean, when I say extremely easily, there is a huge technical pipeline that's been built underneath to be able to go from spatial video to immersive to CGI to interactive back and forth. But it's possible. It's a lot of work, but it's doable. And so that's a massive game changer. When you listen to interviews of creators, directors, producers, I think everyone says always the same thing, which is, what is the best medium to tell this story? And the response has always been, oh, immersive is the best way to tell this story. And that's true, but that's not always true. And for this project, I think we were able to be scene-specific in terms of which medium is the best way to tell this part of the story instead of a whole story. When you're doing an interview with someone, you don't need to be fully immersed with them. It depends on what you want to say. And for this documentary, we were able to be like nothing specific of, okay, when you arrive at Jennifer's house, do you want to be fully immersed or do you want to be, you know, in a frame? And for very specific reasons, for each of these shots, we've been able to make the decision between fully immersive, spatial video, interactive or not. And for us, it's been completely different from what we've been able to do in the past because then every shot had to be like creatively designed from the ground up to match the story. Right. And so that's been a huge change compared to the previous experience that we've had. And I think it ties to what you were saying about the Apple immersive video. I think it's an incredible format for once we have revived interest in video, which is spectacular, but it doesn't allow for interactivity or for more enhancements. And yeah, For that project, for D-Day specifically, we wanted to be able to switch back and forth between different types of media. We have a coming project that will be only in one format or on a different format. But for this story, it just felt right. And for us, we've been loving video for such a long time. Being able to display videos that feel transparent, right? Video where you don't see the pixels, when there's no screen door effects, when you barely see the resolution and where you barely see the pixels. It's incredible because it feels like this is the final vision of what we've dreamt of since we started with 360 video.
[00:18:05.737] Kent Bye: Yeah. And so is it actually using Apple immersive video or is it just 180 video that is a little bit more replicating it?
[00:18:12.975] Victor Agulhon: It's immersive video that's replicating the resolution of Apple immersive video to a certain level. I think if you speak to the folks that develop Apple immersive video, they will see the tiny pixel level difference. But I think in terms of resolution and the clarity of the pixels, we have something that looks extremely good for something that's been built internally. And it's not Apple immersive video. From my understanding, it's a completely different format. And we'd love to experiment with the format because we've loved video from the very beginning, but it's not proper Apple immersive video by trademark.
[00:18:45.185] Kent Bye: Okay. Yeah. And it would have been the first time that I've seen like Apple immersive video within the context of an app other than, you know, some of the other ways that you can download it and play it. The streaming aspect of it is different. And so in terms of the heart of the experience, I think the immersive pieces that they're at the end that were part of the tech demo where you're taking some of these scenes and recreating it. I noticed at the end, there was a disclosure that you were using artificial intelligence in a variety of different ways. And I know that when we talked, there was a lecture that was at the MIT Open Doc Lab, featured the Archival Producers Alliance. And they had put out a whole statement where You know, there's some larger concern within the broader documentary industry with the Archival Producers Alliance was looking at how some folks at like Netflix were starting to just completely do generative AI to create completely made up scenes and then intermix them within the context of a larger film that had actual archival footage. And so there's this question of like the pollution of the archive. So this is an issue where you're taking the archival images, but you're doing like a expansion from 2D to 3D and adding more detail resolution. And so I noticed at the end in the credits, there was this disclosure that details all the different ways that you were using artificial intelligence in this piece. And so I'd love to pass it over to you to talk around this process of disclosure, but also like the use of artificial intelligence in this piece.
[00:20:16.564] Victor Agulhon: I think this production came at a super interesting time for this because in the past we've used, even for JFK Memento, we've used, you know, scaling and technologies that use artificial intelligence. But I think what you said is right. I think artificial intelligence reached new levels of what it was capable of doing in terms of like video generation and so on, which I think was much more threatening to the actual truth. And with, you know, what you were saying about polluting the actual, you know, historical records. And so, you know, this paper of the Archival Producers Alliance is sort of like the Bible that we tried to stick to because the fears are real. And I think that we as creators have to be extremely ethical and transparent in ways we use AI, which is why we wanted to put out the complete disclaimer at the end of the documentary, say, if you are interested in how we use AI technology, we will detail everything we've used on our website. But I also want to say that there's a huge difference between generating pictures with a prompt and trying to remaster footage by algorithms that now rely on AI that previously didn't rely on AI and were just algorithms. And so I think for us, it was an effort of trying to be completely transparent. okay, this is what we've been doing and how we've been treating the media. And this is sort of like the before and after, and we show this on our website very clearly. A key element for us was because we tell the story of a cameraman or for someone who actually captured film, whenever we show remastered footage, we always show the actual photos of film before. And so you look through the documentary and you'll see that At any single moment, when you see a picture that's been remastered, whether it's AI or not, just even if it's hand coloring, for instance, we always show the media. So if you look at the family photo album at the beginning, you'll see in front of you that there is the actual photo album with the photo as it was and as it is in Jennifer's family photo album. And as you go through the pages, you'll see in front of you a screen on which you can see remastered version. And so we believe that the audience is, everyone's intelligent and will understand that what you're looking at is the actual media. And in front of you, you will see the remastered version. The reality is also that AI isn't completely ready for production to operate on its own. It's very like specific tasks that you will use AI for. You mentioned it upscaling, which is, you know, enlarging the resolution of media. And then you have to make sure that, you know, it is accurate, right? So we work with a historian all along the production to make sure that whatever came out of the AIs was first for us, that it looked good and that it was also historically accurate based on the originals. So it's been very much trying to use AI in an ethical way. And I think we've succeeded in that aspect and into being very transparent with viewers. The question of polluting the historical record is a massive one. And I've been doing some research and when I was just looking for footage of D-Day, you can already see that some of the actual photos on stock websites are just filled with fake photos. So there's a real risk for producers. But I've never seen ethics really change in a way that ethics are not media dependent. Ethics depend on what you want to say as a producer, as a creator, as a director. But ethics is, when you're doing a documentary, is truth-telling. And so there's been plenty of ways already to... alter the historical record when you do manual retouching you're still doing retouching to a picture and if you want to lie about something that's happened you're going to lie whatever the technology is and so i think for us it's been about trying to transmit you know that transparency to the audience trying to show you know how we apply care and how we are extremely picky about how we use ai just for everyone to understand that first it's not something that it's it's not a you know one-click solution for anything. And that is just a very powerful tool for artists. When you look at what the algorithm now can do in terms of denoising some of the old photos and film, it's incredible. And so it's been a very interesting journey for us to try to see how we can integrate more improvements of AI, knowing that we are in production. What's ready is the version that we are most likely out one or two years ago because the bleeding edge of AI doesn't give us the level of control that we want on the production.
[00:24:25.401] Kent Bye: Yeah, it was interesting to hear how you were also using it for concept art and storyboarding and pre-visualization. And so as you were developing it, it sounds like that you're using ChatGPT for prompt generation, but also Luma Dream Machine to actually create a animatic is what some people in the previous world call it, a rough cut as it were. Yeah, exactly. Had you already recorded the interviews or was this also just a process of like, what would it look like to kind of have all this footage tied together in a piece?
[00:24:55.941] Victor Agulhon: Yep. So we are a small team. You know, we are 10 people now at Targo. For us, the question of having more certainty on what we're going to be doing, what kind of shots we want to have, making sure that we film exactly the things that we will include in the documentary is super interesting. The documentary is an interesting art form because it's... between planning and not planning, right? You have to create moments, but then you won't control what happens. What you will control is where you want to film. So if you want to go on the beach, if you want to go film an interview at home, if you want to go film in the basement, but then it's the art of letting life happen as it happens. And so we used AI as a creative partner at the beginning of the production, before we filmed anything, to get to a rough cut of the documentary, basically. And so we used it, and that's what we would do before. Before, we would just use pictures here and there of different locations, and we would just put them on a storyboard and add some edits and record the voiceover ourselves to try to invent what the documentary will be. For this one, we realized that maybe we could be a bit more specific than having sort of the broad strokes of the documentary but we could actually try to plan the shots we really wanted to include especially as we wanted to change and transition in and out of different types of media right so we wanted to have like a level of control for the user which is like okay when you're going to be starting to interact with an object we want to have a slow sliding shot from a specific angle that will make you understand that you're going to an interactive moment and so There's some shots that we knew we really wanted and being able to visualize them with AI was a complete game changer. It helped us really be more specific about the kind of shots we wanted. It helped us time also the project to 20 minutes, which is what it ended up being roughly. So it's just been a very helpful way for us to visualize what the documentary would be. And I think it helps us be like, better storytellers in the end, because we can just iterate more. Even before we do the shoot, we realize what's going to work, what's not going to work. So the process for us was, I think, we clearly wrote down the entire documentary the way she envisioned it. And then she started getting with ChatGPT and Luma Dream Machine to get AI to generate some photos to put in the edit. Then she started to do her own voiceover to cover this. And we realized that there were AI voice generators so we could edit the script much faster without having to just re-record it every time. So it's just been a very, very helpful tool to be able to really iterate before we go on the field and to know much more what we were trying to get done. out of the shoot days.
[00:27:35.226] Kent Bye: In this footage that you're generating, are you doing stereoscopic conversion on the AI generated or are you just doing a 2D version of that?
[00:27:41.851] Victor Agulhon: No, all of these like creative footage was just like, none of it is in the documentary, right? All of it was just used before we went, you know, in Connecticut to film and in Normandy. All of it was just like placeholder that we use on the 2D edit for us because all of the 3D work is too specific depending on the cameras and so on. And we didn't need that level of control over the 3D. I mean, not yet, maybe in upcoming productions we will. But right now it was more as just for us, it was the ability to convert to pictures what we would usually just have in words or just in pictures. And then we had like a moving image that helped us, you know, even pick some of the music that we wanted to use before and so on. So it was just I think it just made the pre-production much more realistic compared to what we would get in the end. Honestly, it makes a small studio like us much more efficient and rational in terms of what we actually do when we are on the field.
[00:28:36.650] Kent Bye: Yeah, and the beginning of this D-Day documentary is really quite stunning in the sense that you have like a portaled window, but yet you're also kind of subtly painting with like these film reels that are in the environment. It reminds me of my experience of watching the Gucci documentary on Apple Vision Pro, which was just a 2D video portal, but the way that they were using the environmental effects gave so much of an impact of giving a level of immersion. Even though I was just watching a video, it created a whole... special environment and context that was really transportive. And I feel like the way that this documentary begins does that same type of thing where you're able to add effects to the environment that make it so much more immersive.
[00:29:16.997] Victor Agulhon: Yeah, I'm glad you loved it. It was difficult, you know, to say the least, to like try to get to something that looked good with sort of like the rough shapes that we would get. But we're thrilled with the way it looks. The vision at the beginning was to say D-Day is part of our heritage. You know, it's sort of something that we all live in a post-D-Day world, right? without this attack, I don't think that we would live in the world we live in. And so we just wanted to start by saying, making a statement, which is, we don't see the legacy of D-Day, but it's all around us. It's in everybody's home and it's just surrounding everyone. And so we wanted to find something that was very visual and which would bring the legacy of D-Day directly in the user's surrounding. And so that's really how it started then. from a production perspective, the goal was also to make people understand very early on in the project that it will turn into a more immersive experience. And when you start mapping effects around viewers, I think it gets them into a mindset, okay, there's going to be more than just a 2D window in front of me. And finally, You know, the D-Day documentary is almost like a genre in itself, right? It's almost like a TV documentary, a genre that you could watch on TV. And we liked the idea of trying to really turn the people's environments, people living rooms into a 3D screen, but like properly, you know, you know how you have like some of these TVs that have like ambient lighting that go from the back. We're like, what's the next generation of this? And we felt like it just... It was this, you know, it's like actually being able to map elements, photos filmed around viewers directly in their surrounding. And so we love this effect. I think it's also reminiscent of the Matrix look, but in a subtle way. So it's been one of these things also that wasn't possible before Vision Pro, right? Just getting access to this sort of data and being able to process it in this way. So lots of new elements, which I think contribute to making the project feel like a truly immersive first experience.
[00:31:13.059] Kent Bye: And in the past, you were starting and just doing like video formats. So 360, 180 video. And then eventually I believe with the JFK, you might've used something like Unity, but in this, I'm assuming that you're using like native Apple Vision Pro, Vision OS, all the, or is it a Unity project?
[00:31:30.373] Victor Agulhon: Yeah, it's a Unity project. And I have to tell you one thing is like when we started doing this project, we got a lot of pushback from everyone everywhere who was telling us that it wasn't possible to do what we wanted to do. Like really everyone was like, they're never going to be able to do it because the tools are not ready. It's just not possible. And so we went around and we're pitching, you know, tons of people everywhere. It's like, no not really and that's part of why we did the tech demo right we're like is it really impossible or is this actually possible people think you know that it's impossible and so turns out it's possible turns out it's difficult to do but it's possible we used unity for the entire experience i think our experience of having worked with unity on the jfk momentum project made us understand the very specifics of all the elements that we could use leverage and just all the ramifications but that we could use. But it's a Unity-based project, and we're extremely proud of the results we got. And also, it would have never been possible without Unity. We started to look into the Reality Composer Pro, but it's just not developed to the level where you could be able to do all of these effects and transitions and immersive interaction. So it's a Unity-built project. But for us, it's more like, it's almost like we built a new media engine inside Unity, right? Because this ability to change between media is not something that's like out of the box. And so it was a ton of micro decisions that we made along the way that enabled us to get there. But it's a Unity-based project and we're super proud of it.
[00:32:56.092] Kent Bye: Okay. Okay. That makes sense. And so there's a number of interactive opponents that we can get into. But before that, I want to really highlight the heart of the piece, which is that you're taking these scenes from D-Day, a 2D photo, and then creating a fully spatial immersive scene. And I'm wondering like what kind of techniques you're using in terms of AI to help create the scene or if you were just reconstructing the scene and like a blender model or like what was the process from doing this from from 2D to 3D translation?
[00:33:27.890] Victor Agulhon: So what we had done in JFK is completely different from what we've done here. I think in JFK, we were like, we need to use the media and only the media, the source media to get to the final results. The problem that we had, and that's what we had in the tech demo, is we used the famous picture everyone knows, Into the Jaws, where you have the men leaving the boats and going toward the beaches of Normandy. The problem is that the resolution of these assets was way too low. And with Vision Pro, you could see what it would look like. So we were faced with a decision. Either we had to make it look like photographic memory, but then we had to lean into something that's not very our identity. We love something that's photorealistic. We love when people feel like they're back in the past. And if we wanted to stick to the source media and only the source media, it meant that we would have had to embrace a much more like artistic style of suggesting what a photo you know you that you're walking into a photo and so on and so that's what we did for the tech demo but then we're like we can go so much further now that it would be a shame not to use the resolution we can unlock and so We still start from the same element, which is the exact photo. And we do a first camera mapping of all the elements that we see in the photo. So we're going to apply the textures to all the different elements. So we start by really remodeling the entire scene in 3D based on the photo with the right angle camera and so on. Then we worked with the historian and coloring artists to use the reference and to turn them into colors. And then started a whole new process of, okay, we replace the textures with the historical references that we have. So we went and we scanned boats that are preserved into some of the museums. We used all of the super highly detailed archive photos that we could find to get the... right colors, right photos. I mean, we just try to look at all the materials that existed that would allow us to create a 3D version of that moment specifically. Oh, you know, the seconds before, after, but like the scenes of the detail endings in 3D. And so everything is completely rebuilt. And that's what bugs me with AI, because we always tend to say, oh, it's AI, it's AI. The reality here is like, it's a huge work made by actual artists. They will, you know, model hand by hand the assets that will, you know, listen to feedback, that will look at the references. So it's a massive undertaking, but mostly human made. AI comes in, and that's something we also detail on our website. It comes in really at the end to upscale the textures and to really to try to bake in more details, to try to make the lighting be more natural, but on very select and specific elements. I will say 90%, 95% of the work is purely manual 3D work. So maybe it can be automated more in the future, but Because of the level of complexity of optimization that you have to put into all this sense, you can't really just use AI to do any of this. And a few months ago, I was at an Apple workshop, you know, in Cupertino, where they were sort of like lifting the veil, you know, the way they build their own immersive environments. And the reality is also, I mean, they pitch all of these tools that automate the process and so on. But for the amount of environments we needed to do, we didn't need any of these automation tools. Not yet. And the reality is the process is very much the same. There's no black magic. There's no secret skill that makes something look good in 3D. You need to know your craft to be able to make this. But once you know the craft of how do you optimize, how do you make something look good, you can do it. And it's also something that everyone was telling us is you'll never be able to reach the level of quality and resolution of the Apple environments. I don't think we're exactly there, but I don't think we're too far from the resolution that you have natively in the headset for an experience that has animations, that has a story running, that has Unity running. So it's mostly 3D work, tons of historians, journalistic comparisons and inputs, and just making sure that we get it right to the detail and the colors of the lifeboats and to everything.
[00:37:23.476] Kent Bye: and that's what makes it look good and the reality is that right now it's it's just highly skilled work that goes into this yeah i really enjoyed those scenes that you have these like tracking shots where you're moving forward and you can really start to see the it's an immersive like you're you feel like you're in the scene but then you're kind of moving forward and so you get to see more of the spatial aspect of it because there's the interactive components that you have after you watch the piece then you have the ability to go back to those scenes And I had a fun little experiment where I was sitting still and I was like, oh, I wonder if I could get up and walk out and see. And I was able to actually do that. But in the film, you don't need to get up and walk because you are able to actually do this tracking shots and just create a much more deeper immersion. But there's also some other interactive components where you have these objects that are being talked about in the context of the film where there's these artifacts from the war that the daughter that you're featuring is coming across. And you have the ability to actually pick up the object and look around them as well. And so in those objects, did you just do like a photogrammetry scan or talk about the process of translating those into more interactive parts? And what was the thought in terms of wanting to add this more interactive agency component where you could chase your curiosity a little bit more as the story was being told?
[00:38:42.358] Victor Agulhon: Yeah. First, on the interactive scene that you described, we feel like, because in the documentary, you know, they serve the story, but they're also, you know, there's like fully immersive scenes. But the reality is, it's something you almost want to get back to, to just watch and take it in and just look. And so that's what we did. We just created these interactive moments where you can just spend your time and you can look at the photo and compare if you wish. But you can just, you know, let your thoughts run free and just be there in the moment and just think about what that moment meant in history to the life of the soldiers. So that's really what we wanted to highlight. Baking interactivity into the documentary is what a completely different A Fourth Press. It comes back to the story of telling Jennifer's journey, right? It's She was slowly putting together her dad's involvement in the war. She had all of this stuff. And it was really about connecting with her in the way that she tries to connect with her father. And so that's what we mean by the fact that we've been able to truly blend technology and the story. You see her do some of the actions. and we wanted you to be able to it's almost automatic when you see these boxes you want to grab the objects and so we wanted you to have the same experience as her to try to connect with her in the way that you know when she talks about grabbing these objects about connecting with history about you know reading the letters from 43 44 and We wanted you to be able to replicate that to connect more with her because I think it's a very universal journey that she's going through. Not all parents have done some incredible achievement in their life, but there's always a side of everyone's parents that we don't know. And so that quest of trying to reconnect with family felt very much universal. And we felt like, if you do the things that she does you will also you know join her on this journey so it was very much about mimicking replicating and also just like the ability to create close interactions with some of these like historical objects i think it's something that's Just very powerful, even just, you know, from a museum perspective, it's like there's no boxes around, you know, there's no glass window in front of these objects. You can just grab them, look at them, and it's very powerful. And so that's from the story, like in the production side of why we wanted to do it. In the technical aspect, it starts with scans because we wanted to have the right proportions and everything. But then it's same, it's photographic references. We had tons of photos of each of these individual objects and you have no idea how many rounds of feedback we've made to have them look exactly the way they looked. You'd be shocked to see how many rounds of feedback went into making sure that the metals, that the purple heart looks like a purple heart and the reflections are right, that the grain on the camera looks right, that the pattern on the helmet is correct the way it's reflective. mostly 3D photographic references, video references to be able to match it. And I mean, we wish it would be possible to do, you know, like Gaussian splatting or photogrammetry, but the reality is right now, because of the optimization needed, it's still, you know, human work and artist work. And it's beautiful in a way.
[00:41:45.609] Kent Bye: Yeah, in terms of the other interactive parts, and when I'm flipping through the photos, I noticed that I think I had gotten ahead a bit where he starts to talk about the bullet wound and I had already crossed that photo. And so I went back. And so it's kind of an interesting like balance between like letting the user go at their own pace. But yet, if they follow what's being told, then there's ways that the story is augmenting and amplifying what you're seeing. And so there's that moment where I had that I got off track, but I got back on track very quickly.
[00:42:12.071] Victor Agulhon: Yeah. I think for us, what bugs us is these kind of experiences where you have a notion of failing or succeeding, and we didn't want to make interactions compelling in any way. If you just stand back and look, you'll be happy. You'll have a great experience. If you want to interact with the objects when you're invited to, you'll have a great time as well. And there's not even a pace at which you're supposed to change the photos. We wanted to give people enough time to also get into their own thoughts looking at a photo because you'll pick up a detail and you'll think about this detail and then you'll go through the next pictures and then you'll keep on listening to what she's saying. But we felt like... you know, it's also a way to let you control the story in your own way. It's like, we don't want to, the only thing we didn't want to like you spending five minutes into this. So it's timed in the experience, but then it's, you know, it's what we say is that we value users attention more than their actions. And I think it's very true in this documentary. It's, as long as you pay attention you will be fine there is no need to flip the pages at a specific time there is no need to grab an object if you don't want to grab it these are more like additions that enhance the story rather than like requirements for you to move forward and what we try to stay away from was this idea of you know you have to click on next if you want to go to the next this is a documentary it has a starting point and an end point we're taking to the story at a pace we think is going to enhance you know your experience
[00:43:33.527] Kent Bye: Yeah, one other interesting technique that you use that I haven't seen you use before or very many other people, which is that you're looking through a viewfinder. So you're creating like a portal view, but you're watching this 2D footage and it's somewhat occluded. So you can kind of look around and get a sense that you're painting around. So it's taking a 2D artifact of a film that's already been shot, replaying it, and then giving you a sense of shooting it. And I thought it's an interesting interaction part just from the viewing experience, but also potentially serving the function of dealing with footage that could potentially make people motion sick because it's, on a boat and it's moving around and having that frame helps to mitigate that. And I don't know if that was part of the initial motivation for that, or if you also wanted to just find new ways of taking something and making it slightly interactive, even though it's not really having that much of a difference as people are moving around.
[00:44:24.932] Victor Agulhon: I think it's a game changer for people in the way that they think that now you realize that he really filmed it. And so that's something that he saw. I think it makes you realize that, oh, this is what he saw because we are sort of like, not insensitive to like the footage, but you know, we're used to seeing World War II footage. We're used to seeing all these things on like TV. And so you're like, Yeah, whatever. Like just more World War II footage. And just by this simple technique of you put a viewfinder around it, then you're like, oh my God, that's what he saw. That's what he went through. And I think it just puts you in a much more open-minded position to like think, okay, that's someone went through this experience and that's... what he went through. I think it was much more as a way for you to get into the mindset of real people went through this and just understanding that there's always people behind the camera. So it was like, it started really as a narrative push. And then it's actually funny that you say that you think it makes people less sick, but because our realization, it made people more sick at the beginning. And so we had to work a lot on stabilizing the footage a lot just to be able to have like sort of like a horizon that's that steady at the horizon because when we just put the raw footage it was like you know because obviously he was you know capturing this movement on the boat it was extremely shaky and i mean when you're filming going into today you know you're not going to be like extremely steady so we had to stabilize a lot this footage just to make it look steady because especially with the viewfinder it made it like you had way less understanding of you know the fact that the scene had borders and so on and one example of how we wanted to use ai and how we didn't end up using ai is this footage at the beginning we wanted this footage to be converted to stereoscopy and we wanted it in color but then we realized that if we did this then we'd be presenting people with a version that was probably closer to what he saw but that's not closer to what the actual footage is and you know we would probably be putting like layers between the viewers and the actual media and so we made the decision to stick to the original footage only to stabilize it but to show it as is and that's one of the ways where i think this documentary really tries to be transparent with the archive that he chose and the treatment and there's just something very powerful about thinking it's sort of like you know that contradiction between now you're seeing what he saw And you're being into his mind, into his body almost, in terms of what you're thinking. You're really getting into his mindset. But you're not seeing what he actually saw. You're seeing what he filmed. And it's actually different. And this paradox is what creates a tension here. You see the footage, but you're thinking about what he went through when he filmed it. And so it's this kind of tension where we think that interaction really highlights the power of the story and this journey of connection with Jennifer's father.
[00:47:06.345] Kent Bye: Yeah, my final thought is just that the way that you were able to tie all these media together from like the photo of him having a wound from D-Day into like the letters, reading the letters, having a bit of a narrative of him leading up to it. And then, you know, when you actually get to the D-Day, going through all of that and The moment when you have this frozen shot of the bullet going towards him, kind of in the bullet time, slow motion. And just the process of knowing that he was wounded, shooting all this and going through each of those different scenes. And to have the daughter there also telling her story and the emotion she brought to the overall story, I think just overall is all tied together in a way that focusing on this one character, you're able to get the full journey and recontextualize it and make it feel like you were... I don't want to say make it feel like you were there, but feel like there's a proxy of having a little bit more of an immersive experience of the context of what that experience must have been like. So yeah, it was really great to see that all tied together at the end. And yeah, it was just a really beautiful piece.
[00:48:09.605] Victor Agulhon: Thank you. No, I think, I mean, that's really what we tried to achieve for us. It was very much, it's paradoxical because I think it's probably one of the biggest events that we've covered just in terms of the amount of people who were killed on the D-Day landings, you know, just on that day. But it's also the one probably that feels the most personal, you know, because you have this story and I think the level of connection that you have with Jennifer all along the documentary is very transparent. But also she was an incredible character to interview because she was really going through this journey of going back to Normandy. It was like, for her, it was a true document. Like for us, from our perspective, it was a true documentary where you're filming her going back to Normandy, discovering all of this. And it's just a very, very personal story and a very, you know, big historical story at the same time. And I think that connection and how the technology enables to bridge, you know, the interval story and the larger story, I think was incredible to work with. So, you know what, now we hope that it's, in somehow showing a way for documentaries. We think it's the most advanced, immersive documentary ever produced just because of the richness of the media and the diversity of media and the power of the story. So we hope that it will show a way forward for more documentaries like this. And above all, we hope that people will enjoy it.
[00:49:30.794] Kent Bye: Awesome. And finally, what do you think the ultimate potential of immersive storytelling might be and what it might be able to enable?
[00:49:39.324] Victor Agulhon: It's the final question, right? So... And I think we've talked about this before, but it's really about that bridging time and space. And it's going to be very French. But when I was a young student, I was reading a lot of Sartre, the philosopher. And I love his thoughts about existence and time and space and phenomenology. And just looking back, I really feel like this is what immersive technologies are about. They create a universal experience that... can potentially get rid of time and space when used in the right way. And it's paradoxical for many people, but I think it's the technology that connects the most people in the world because you're able to have access to the raw emotions, the stories of anyone, anywhere, in a way that feels very personal. And so I think it's the ultimate connection device on a personal level. And at least that's what we intend to prove with it.
[00:50:40.450] Kent Bye: Nice. And is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:50:47.397] Victor Agulhon: Keep going, keep going. It's such a challenging space. There's not much money to be made. There's not much investments right now in the space, but keep going. I hope these projects help, you know, contributes to proving that we can create great stories that like make a difference and Yeah, you know, just stay in the space because even though it might be hard, you know, on some days, I think the power of what we're able to create with these technologies is just immense. So thank you very much, Kent. It's always such a pleasure, you know, to talk with you just because we can get specific on, you know, what we want to talk about and we can just cover, you know, in depth. And I know that you understand everything. So it's just always great to have this conversation with you. So thank you for the interest and for taking the time.
[00:51:33.834] Kent Bye: Yeah, well, I know Targo Stories has, over the years, consistently been on the leading edge of pushing the structures and forms of immersive storytelling, especially in the documentary form. And so, yeah, it's just great to see your continued evolution and to see how you're continuing to refine and upscale and continuing to innovate on so many different levels. And in this one, I think the thing I'm really taking away is just the way that the heartfelt story is also tying in together. So it's just a really beautiful way of diving into history. So... Thanks again for taking the time to join me here today to break down some of your latest innovations here on D-Day, The Camera Soldier.
[00:52:06.494] Victor Agulhon: Thank you, Kent.
[00:52:08.034] Kent Bye: Thanks again for listening to this episode of the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
