Targo Stories continues to innovate with novel spatial storytelling techniques in both of the projects that were submitted and selected to SXSW. The Behind the Dish series uses micro lenses to give a uniquely fresh and giant look at food being produced by female chefs. And then JFK: Memento is experimenting will all types of translations of spatial footage from 2D into 3D, that’s then juxtaposed within a volumetric representation. It steps you through the Kennedy assassination, and gives you a much deeper spatial context of the events as they unfolded. I had a chance to break it all down with Victor Agulhon (co-founder of Targo, producer of projects) as well as Chloe Rochereuil (co-founder of Targo, director and creative director of projects).
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. It's a podcast that looks at the future of spatial computing and the structures and forms of immersive storytelling. You can support the podcast at patreon.com slash voices of VR. So, continuing on my series of 24 episodes from South by Southwest, today's episode is with Targo Stories, which they actually had two pieces at South by Southwest. In competition, they had JFK Memento, as well as a piece that's called Behind the Dish, which is a three-part episodic series that is already available on the MetaQuest store. You can go watch it. So they follow three female chefs and their process of making their food, and they have this real interesting micro-stereoscopic view of food. So the food is, like, gigantic, and they have this unique look of trying to display food. And it's just around the culture of food and the history and these women's stories. So that's behind the dish that we talk about first. And then we talk about JFK Memento, which is produced in collaboration with the Sixth Floor Museum in Dallas. And they're coming up on the 60th anniversary of the JFK assassination. That was on November 22nd, 1963. And so they're doing a six-part episodic series that they're going to be having at least showing there at the museum in Dallas, Texas, but also may be available more broadly. That's still to be determined, but we have a chance to get an early look at the first two chapters at least. some of the ways that they're taking what they usually do with 360 video storytelling and start to translate that into more of a spatialized 6DoF experience. And they're doing lots of really interesting things with recreating a spatial context and then overlaying on top of that different archival footage where they're able to translate 2D archival footage into more of a stereoscopic 3D look. And you have this portal view that is moving around these different spatialized contexts. And so they're doing a lot of really innovative and interesting things of trying to blend the forms of immersive storytelling that they've learned from 360 video, but blending that into more of a 6DoF experience and trying to create this new fusion of techniques. So they're really pushing the edge of what's possible with the future of immersive storytelling. So we'll be covering all that and more on today's episode of The Voices of VR Podcast. So this interview with Chloe and Victor happened on Monday, March 13 2023 at South by Southwest in Austin, Texas. So with that, let's go ahead and dive right in.
[00:02:22.873] Victor Agulhon: So my name is Victor Aguilon. I'm the co founder of Targo and producer of virtual reality documentaries.
[00:02:29.061] Chloe Rochereuil: And I'm Chloé Rocheret, and I'm also the co-founder of Targo, and I'm the one in charge of the creative side of the documentaries we produce. And here at Top of Bio, we are showcasing two experiences that I directed, JFK Memento and Behind the Dish.
[00:02:44.251] Kent Bye: Great. So maybe you could each give a bit more context to your backgrounds and your journey into making VR documentaries.
[00:02:50.290] Victor Agulhon: Yes, so the journey to getting to VR documentaries, for me the starting point of getting into VR was the interest in considering VR, AR and spatial computing in general at the next interface with technology. And the reason I got into this field was that I wanted to showcase the power of this technology to mainstream audiences and very early on across conversations with Chloe we realized that documentaries was really one of the ways that people could understand the power of this technology to connect with other people with locations and so you know how life works you know you have conversations you start making projects and in the end we really settled on that style of documentaries that allow people to meet extraordinary characters to explore places that either don't exist anymore that are out of reach or that have just explore the past. So this is the journey of how we do VR and I think right now our sites are really focused on becoming the leading non-fiction production company in virtual reality.
[00:03:44.160] Chloe Rochereuil: And yeah, so I met Victor in college and I was doing a master in journalism. So my background is more focused on journalism that led me to storytelling in general. And I think when Victor came to me and talked to me about VR, it was early back in 2015. So VR was not something that people would be familiar with. And when I tried it for the first time, I immediately thought that this could be a so powerful way to just bring people into the story. And I think that's, There is a need today for VR to connect people with the real world because a lot of people assume that VR is for games or like virtual worlds but I feel that with documentaries there is definitely this connection to the real world that is super interesting. And yeah, so then we co-founded the company and we started to work with different kind of media. Starting with the New York Times, Al Jazeera, USA Today, mostly producing 360 video content. And then we decided to make our own documentaries. And I think that one of the first pieces that made us exist in the XR community is Rebuilding Notre Dame. You know, one of the first pieces we made with this specific format of like 20 minutes documentaries. And for this one we had the opportunity to film inside the cathedral before the fire and then after the fire. So I think it was one of the first pieces that was really standing out. And then, of course, When We Stay Home. And those two got nominated at the Emmy Awards. And after that we produced many more documentaries on mainstream topics to address mainstream emergencies in VR.
[00:05:21.237] Kent Bye: Yeah, you certainly have already quite a filmography of different pieces that have appeared at different festivals that we've crossed past a number of times, and so I guess this time you have a piece that has already been released onto the Oculus Quest, or the Quest Store, that people are able to watch with Behind the Dish, and then you have the JFK Memento, so maybe we can go with the one that's available so people can go watch that and then listen up to the point and then they can go and either decide to listen on or eventually see the JFK Memento. But let's talk about behind the dish. How did this project come about and what were some of the technological innovations that you had to do to even produce some of this micro photography?
[00:05:58.800] Victor Agulhon: Yeah, so the story behind Behind the Dish is actually that food is one of these topics that we always wanted to talk about in VR. It has a sort of like mainstream appeal. It's some of the most consumed content all across the internet and we were really fascinated even with the shows on Netflix, you know, Chef's Table. So there's something really culturally popular that we loved around food and there had been nothing that had been done properly in VR on food. And even though we had like groomed that desire to focus on food and gastronomy, we always found that the technology that was available at that time was not good enough to say something new around the food. So we really gathered and we kind of like got together to brainstorm and think about all the ways we could showcase the food to make it worth watching in VR. And in the end, we kind of like landed on this idea of creating a rig that would allow us to change the scale of food. So the idea would be that instead of looking at food as sort of something that you're about to eat on the plate, make it change scale, change perspective on it, and have it look like a sculpture that's like three or five meters wide, and really look at it like a piece of art. And so that's the starting point of Behind the Dish. It was really this kind of like switch of perspective on food that got us started, and that was something that we could only achieve in virtual reality. So the technical innovation on that side was probably one of the main driving forces on how we can, you know, tell the story around gastronomy. And then, obviously, the stories fell in place with the stories of chefs.
[00:07:17.502] Chloe Rochereuil: Yeah, so when we had this amazing technology allowing us to show food like never before, we also had to find interesting stories, stories that could really fit into a category of... I mean, we wanted to tell stories that would be compelling to everyone and I think that the female chef came naturally to us because this is an industry which is widely male-dominated and I think that with Targo we've always wanted to focus on people who are kind of extraordinary and special and unique in their own way. And I feel that putting a spotlight on the female chef was a way to just focus on stories that would be, you know, full of challenges, of things that would be out of the extraordinary. And so we decided to focus on three female chefs in three different countries. One in France, Hélène Darroze, one in Japan, Yumi Shiba, and the fourth one, Deborah Ventress in Atlanta. And all of them, they obviously have very different stories, but they all face the same challenges in terms of, you know, finding their own voice into this industry and also facing an industry which is, as I say, male-dominated. So I feel that with their story, we obviously have this amazing technology showing these gigantic dishes, but then we also have this special connection with women who really make their own way and who and incredible stories, so blending the two really creates this unique content and this special connection with the character that we love to do with Targo.
[00:08:47.733] Kent Bye: How did you find these three chef protagonists?
[00:08:51.310] Chloe Rochereuil: Yeah, so I feel like at the beginning we wanted to showcase different kind of foods because we felt that different types of food such as sushi, soul food or gastronomy food would really showcase different kind of images, textures, reflection and indeed like the three episodes are very different like from one another and then we thought that We also wanted to showcase the culinary tradition all over the world. So it was not a documentary only on food and the food the chefs were creating, but also on the tradition, the cultures of the different kinds of food we were showing all over the world. So then, of course, naturally, the three different continents, the US, Europe, and Japan, we thought about them. They're just appealing to everyone. And it's funny to see that at South by people, they all want to see Japan because I guess there is something exotic in it. And then we find this chef who had amazing career, amazing journey. And especially, I mean, all of them really faced different challenges. And I feel that these three countries are a great mix of culinary tradition and amazing dishes that really look amazing with this technology.
[00:10:01.834] Kent Bye: Yeah, the thing that was really striking to me is I watched them, I guess in, I don't know if it was in order when they were made, but at least when they were showing up in the VR headset of the USA and the Soul Food, and then the Japan, the sushi, and then the gastronomic six-star Michelin chef that's in Paris, France. And so just to have the history and the context and the way that the food is situated into a very specific location, I feel like it's a really interesting connection that you're making there between place and the power of VR to cover place and the ability to tie that into the different foods that emerge out of these cultural contexts. So I'd love to hear that process of trying to both cover the stories of individuals, but also the broader cultural context of how to tie all those different aspects of the place, but also the history of places.
[00:10:47.579] Chloe Rochereuil: Yeah, I mean, it was super important for us because, like, of course, the macro shot creates this special connection. You feel like, you know, you're a small ingredients like in the middle of the plates and you're almost like inside the hands of the chef. So it's very special to be there, but also being able to see the behind the scenes of how these dishes were created from the local ingredients at the farmers to the kitchens, the back kitchen of the restaurant. I think this we also wanted to show, you know, how we coming from ingredients to final dishes as well as the biography of the chef and how they became who they are today. So these storylines just intertwined, these two storylines intertwined with each other during the episode and I think it works well because there is this natural balance between the macro shots and the 360 environment.
[00:11:38.545] Kent Bye: Yeah, I'd love to have you maybe elaborate on some of the different technical aspects of the camera because I know that Canon came out with like a fisheye lens that you put onto like a DSLR camera that is like a high-end camera. Is that kind of a similar approach or did you have to basically build something from scratch that is like a much smaller scale?
[00:11:55.183] Victor Agulhon: Yeah, we did have to build something from scratch. At the beginning, once we had that idea of what if we made food look gigantic, we had to explore several options. The first thing we thought about was what if we did photogrammetry of the food. So the thing is, if you do photogrammetry of the food, you're going to have a very highly detailed 3D model, but you're not going to have all the reflections, all the smoke, all the detail that you want to see when you're looking at the food and the things you're used to when you're looking at food in traditional formats like TV. So the photogrammetry, quickly we thought that it was not going to be something that we could manage. Then we looked at ways to kind of like create what we call hypostereo, which is basically reducing the distance between people's eyes to change the sense of scale. Put simply, because it's a bit technical, but put simply our sense of scale today is defined by the distance between our eyes. It's called the interpupillar distance and in average it's 64 millimeters per person. if you change that scale, if we decrease that distance, you will have the sensation that everything around you is larger because that's how we perceive depth. And so what the camera does to your eyes when you're looking in VR headset is that it decreases that distance to a point where you have actually just one centimeter between your two eyes and that's what gives you the sense that, you know, the food is scaled five, six times, seven times. So that's how we play with the sense of scale. But to do that, when you want to put two cameras at one centimeter distance, you can't put them together too close, because when you think about the body of the cameras, it's too big. So what we had to do, we worked with German studio, we worked with lots of consultants, stereographers, to actually assemble and create a technology, a rig, that would allow us to create overlap between the cameras and this is a technology that's used in Hollywood in studios and our challenge here was to make it compact and something that we could carry and bring with us in the kitchens with the chef. So now we have a rig that's really custom specifically for the needs of this production that allows us to film the food in close-up in a controlled environment and that allowed us to get all these shots. Now we have that rig and we just think that we can apply it to so many other topics, everything that's about craftsmanship, everything that's around high details, like there's so much potential around it. But in terms of technology, and I think that's one of the driving forces in everything we do, is when we think about a topic in VR, We want to bring something that's never been done before. We want to bring a kind of visual and experience that people have not been able to experience otherwise, and that is only possible through the lenses of a VR headset. So for that project, for Behind the Dish, that camera was a big enabler to reach that point. And all the technical learnings, I think, contribute to making the future productions also that we're working on much higher quality.
[00:14:23.089] Kent Bye: I had an experience after watching these, and I'm curious if other people have had this, of wanting to come out eating a lot of food. Is that a common reaction?
[00:14:30.285] Chloe Rochereuil: Yeah, always. Especially if you're watching the content just before lunch, it can get really hard then. But yeah, a lot of people also ask me, it would be amazing to be able to smell the dishes and have extra sensation with the experience. And of course, I think that because you're so close to the food, you really feel that it's there. And I feel that it's something that has never been done before. And because it's so close, you also want to have more. So it's creating this frustration of wanting to eat the food, smell the food. And I think it's a success to get this feedback, because then we managed to achieve a goal, which was getting you the closest to the food possible and being able to see dishes like never before. So it's a pretty good feedback, I would say, to feel hungry after that.
[00:15:17.670] Kent Bye: I know back in either 2018 or 2019, I heard of one chef that was doing an immersive experience and then having a whole eating experience that happened afterwards. Is that something that's kind of on your radar in terms of how you could transform this into a location-based entertainment? Because you're really getting the backstory of these different people in these restaurants, and I could see how people coming in and have an immersive experience and maybe that's how they help decide what they want to eat or at least have some component where you get more of like where does this wasabi come from like in the Japanese episode that you had that really goes into the depths of taking you into a specific location. I think it's really powerful for me at least to see what the process is that goes into making the food and how restaurants could be a part of having people appreciate the care that they take in curating and all the stuff that maybe is invisible but making it more explicitly visible for people. So, yeah, I don't know if you've thought about if there's pieces like that that could be more explicitly tied to either some of these restaurants specifically or other location-based entertainment experiences that would be traveling around things rather than tied to any specific restaurant.
[00:16:21.808] Victor Agulhon: Yes, I think regarding the distribution landscape in VR, it's evolving very quickly. And what is so interesting, now we see the emergence of all these domes, all these planetariums that get repurposed into distributing narrative content. And we went to domes with Chloe and we tried it out, we tried the content in the domes. And if you take the US episode, for instance, you have the mushrooms and they're absolutely massive in front of you. And when you display that on the dome, you can like feel that it's almost surrounding you and you feel like you're in the food. So there are literal ways for people to like eat the food, to place tables in domes and display the contents all around. I think the starting point for us was making this available in the VR headset to show what was possible. around it, and now we're exploring how we could actually distribute and enhance the experience around it. And for the US Chef, for instance, it would be very easy to organize an event in one of the cities where there's a dome and having the content being displayed in the background during a tasting experience. I think it's just a starting point and what's also so interesting about this documentary is since we've released it so many companies that either are involved with food that are either like restaurants have reached out to us and have been like thinking of how they could also create a food experience for themselves. So I think we're evolving naturally as a society toward more and more immersive experiences. I think there's a desire for it and we think that behind the dish as a production is a good showcase of what we can do around a theme like food and we've seen the excitement because we get like a lot of people reach out to us about how we can apply our learnings to what they're doing and so it's definitely exciting. We haven't found a way today to do it but in the next year and next 18 months it's realistic to think that there's gonna be like sort of like a tasting experience where you have all the food that's around you based on behind the dish or on a spin-off content.
[00:18:05.505] Kent Bye: Yeah, and what has been the reactions to this piece so far now that it's been out on the Oculus, or the, I keep saying Oculus, but it's MetaQuest. It's still the Oculus to me, so. But what's been some of the reactions so far?
[00:18:17.875] Chloe Rochereuil: Yeah, well, I mean, the reactions were amazing, especially showing it at South By. It's great to hear, like, real comments from real people, you know, because in VR it can be quite frustrating to not having feedback, because, of course, like, you can't really put comments or discuss with people. So the feedbacks were amazing, and I feel that a lot of people are coming out from the experience telling us, wow, the imaging are outstanding, like, the quality of the images are incredible. And a few years earlier, You know, people were coming out of the experiences and were like, yeah, it was good, but a bit blurry, you know. So now we don't really have these feedbacks anymore, which is good because we're improving the quality, of course. And I feel like the macro shots are also something that people have never seen in VR before. So it's definitely something that lets a mark on you. And yeah, here people are very interested in the Japanese episode. And I feel that people coming from Asia are more interested in the French episode. It's also interesting to see what kind of interests we have regarding the people watching the experience and where they come from. So, very interested feedback and we're amazed by the people just discovering our experience. It's absolutely amazing.
[00:19:27.548] Kent Bye: Yeah, I thought they were all really strong episodes. And for me, the USA episode about soul food was, I think it had a lot of really powerful emotional arc as well, just hearing that story and the history. And yeah, I just thought it was really, really well done. And I'm curious if you had a chance to actually show it to some of the chefs and what was it like for them to start to see their food in such a new perspective?
[00:19:49.152] Victor Agulhon: Yes, so, I mean, we haven't been able to show it in person to the chefs that are in Japan and in the US, but we shipped them a headset and they saw it. And some of them were used to having, like, their food being filmed, but I think seeing it like this, they were all completely impressed by the technology. Because think of it like, already when you know VR, you're impressed by this, because I think it's a technology that feels really new. But when it's your food and you don't know anything about VR, and it might be your first touchpoint with VR, they were completely blown away. And for the chef that's in Paris, we went there, we showed it to the entire team at the restaurant, and they were also blown away because even if they've been on many shows, especially in the Michelin-starred restaurants, they've seen their food many times on camera, but something like this, they had never seen it before, and it was also very powerful for us to see that the people that we filmed had such an impressive reaction to the food and to the content, so it was very rewarding, even as creators, to show that we're not doing something that only talks to the people in the VR community, but that it really speaks to everyone. And just to add on to what Chloe was saying about the reception and what we've learned about the feedback we got, I think it's the first time we do a documentary that's also very entertainment-focused and that is such a universal topic and there's no side note to it. I think these documentaries are about food, I think they're very uplifting, all of them, they have all a very positive message. and everyone can relate to them. I think it's the first time for us to be able to do a documentary which is a bit more light-hearted like this one. So it's a very satisfying experience also to kind of like bring that much joy into people's lives through a content like this. So that's been one of the learnings for us. And back to your question regarding like the chef's reaction. They were really impressed by, you know, even for them seeing that amount of details on the food and, you know, seeing their hand, like, seeing the detail, the precision, I think that makes sense, especially in the Michelin star one, where, you know, they use, like, the tweezers, I think, just, like, to place the flowers very specifically on the plates. That's a very satisfying experience, even for them to, like, look at their craft this way.
[00:21:39.595] Kent Bye: Yeah, well, I just want to say that I really enjoyed these, and they are available. So I highly recommend folks go check them out, because I think it's one of those episodes that I think you could probably show to a broad range of different people and help also introduce the power of VR to folks, because it gives them a new perspective on food and also telling these stories. But yeah, maybe let's go on to the other piece that you have here, the JFK Memento. So maybe you could give a bit more context for how this project came about.
[00:22:03.578] Victor Agulhon: Yeah, so I think it ties back to what we do as a company, which is trying to speak about topics that people can relate to. So the food is a good example, and I think JFK Memento is really a second good example for this, because this year, we are 2023, and this is the 60th commemoration of the assassination. And one of the big learnings we've had as a company in the past years is that with VR, we can make people travel across time, and we can put them back into places that don't exist anymore, and we can give them emotions and understanding about events that happened in the past. And so for JFK Memento, the goal here was really to allow people to dive back into 1963 to understand what happened on that day. And one of the starting points on this was collaborating with the Sixth Floor Museum. When we had the idea, we reached out to them very early on to present them our idea of doing a documentary around the Kennedy assassination and the investigation that followed. And so we got in touch with them, we presented them our idea, and we realized that they had a huge amount of collections, films, photos, pictures, like archive, you know, materials that were related to the assassination and the investigation. And we just thought that, you know, this was such a great starting point for us to kind of like base a documentary of it. And so the goal of this documentary is to allow people to live and to explore the 48 hours that follow basically the moment when Kennedy was shot. And the goal is to really tell it through the eyes of some of the witnesses of that day, some of the detectives, the journalists, but mainly to base it off the archives, the films, the photos that are reprojected in 3D and give you a spatial understanding of the events.
[00:23:30.748] Kent Bye: Yeah, I thought it was really quite effective because while I've seen the Zapruder film and some of the other shots, I didn't have a broader sense of the other geographic context. I could go and do Google Earth VR and see it, but I don't know all the different spaces, but I felt like it's able to give me that broader spatial context of that event because you're doing a lot of compositing or you know creating this collage like effect of like creating and setting the broader spatial context but then overlaying a 2D film that is like a window but then also within the 2D film you're giving a specialization in that so it's like a window into 3D of what happened but you're able to I guess it's like our fovea is able to track at a very high resolution area but you're able all the rest of our eyes that is more of a blurry and so in that kind of a metaphor, you're setting the larger context of the eye, but then having this kind of a fovea-specific area, which is the archival footage that's overlaid and animated and moved. And so I thought it was a really interesting compositing technique. I'm not sure if you've used that specifically thing before, where you're having it as existing spatial context and overlaying some of the 2D images like that. But I'd love to hear your process of kind of using that as a conceit to help ground people into that place in time.
[00:24:44.886] Chloe Rochereuil: Yeah, I mean, what you're saying about specializing and having this special understanding of the event was very important for us because we realized that when you're watching the films and photos, you usually don't understand the context surrounding these photos and where they've been taken and what the sequence was. So we felt it was important for this documentary to be able to stand in the exact position from where these films and photos were taken. And then, of course, we thought about how should we integrate this photo. And we realized that we have different techniques to make sure that these photos bring this special context to the story. And, of course, there is these kind of build boards, 2D build boards, all around the place. And these are people extracted from the films and photos that we put onto the plaza and inside the Texas Building repository to give you the understanding of what was the atmosphere of the event when it happened and then we thought a lot about what should we do with the videos because we first thought that it could be good to have like different layers inside the videos and reproject them onto the environment but then we realized that if we break the frame of the video you kind of lose this connection with the film And I think that there is an emotional connection with the films and the entire frame of the film, which is important to just understand the context. And I think it's way more powerful to have the entire frame than just different layers. So we mix these two approaches of the films and photos, and the films and photos are really The connection between all the sequences and all the witnesses, it's really the visual effect that just leads you through the experience. So we thought it would be our central, our core effect in the experience to just make sure that people, of course, are in a 3D experience, but they connect to the event with the real material that was taken that day.
[00:26:42.842] Kent Bye: Yeah, and you know, in this piece you have like this combination of both 2D photos and then videos and then overlaying them into the spatial context and recreating different scenes and in that spatialization process I know you've done that before in like Survive in 9-11 where you have a little bit more of a 2D panoramic but you're able to give more of a spatialized depth to these and I'm wondering if as time has gone on and you've had an increased ability to have artificial intelligence to be able to use some of this stuff or if it's still largely manual rotoscoping process where you have to do a lot of like painting in the different missing spots that would be normally occluded from a monoscopic perspective but have to recreate what that stereoscopic perspective would be. So I know as you're doing that it's obviously in the past been a lot of manual work but I'm just wondering if that's been reduced with some of the advents of artificial intelligence and if you've streamlined your process for doing this type of 3D spatialization.
[00:27:36.930] Victor Agulhon: So I think what you're saying about surviving 9-11 is interesting because that was really the starting point for us, about understanding that we could treat archive pictures and give them a new significance, a new meaning today through the work that we're applying to them in 3D. The key difference being that this experience, JFK Memento, is really the first time that we also venture into the six degrees of freedom world, so it's all 3D recreated. And so all the work that had to be applied to these archives had to be redesigned from the ground up because we could not do the same treatment, otherwise it would not look as convincing in that space. So we tried a lot of pipelines to kind of be able to improve the look of the archives to make them fit in the frame, create like 3D conversions. So we tried a lot of different pipelines. What we realized was that all the tools to convert to 3D using AI were still too young to be able to be used in production. They would look good for 80% of the frame, but the remaining 20% would be impossible to do by hand. And then we will have to do the entire process manually. I don't think we're there yet for that use case. I think it's maybe a matter of like two years before we can use that in production, really. But we've used AI as part of the process to kind of like clean up the pictures, remove all the noise, all the grain, so all these kind of like scratches and stuff. AI has been really good at it, but it requires also some kind of like heavy monitoring to make sure that it doesn't alter the source media. So we've used it in the pipeline but it hasn't been the sort of like a magic button that we press that creates all of this and we did have a lot of like R&D and tests to create a sort of like convincing way to reproject the archives and the way you've described it is really what it is. It's that we have these 2D layers, we have these 3D kind of like frame that's converted into 3D so It's a blend of all different techniques and we kind of had to adjust the different levels to which we put technology to create an environment that's convincing. But AI, unfortunately, still isn't to the point where it can be used in production like this completely. And I do think it's going to land there at some point, but just in production right now, it's not working yet.
[00:29:33.502] Kent Bye: Yeah, so you have a number of different chapters. You have other chapters that you're going to be working on. So maybe you could walk through what the chapters that you've done already and what's to come. And as you come up onto this 60th anniversary, then what's going to be revealed there at this sixth floor museum in Texas?
[00:29:49.615] Chloe Rochereuil: Yes, so here at South by Southwest we are showing the first two chapters of the experience and the full experience will be released in November for the 60th anniversary. So basically we are trying to chronicle the assassination from the moment JFK was shot until the arrestation and then killing of Oswald. I'm a bit spoiling the experience right now but you know. So the first two chapters are really about the assassination and the first chapter. And they're all told by witnesses who actually saw the events or participated into the investigation. So the first chapter is happening on Dili Plaza. It's told by Marianne Moorman, who was a young lady in 1963, and she actually took the picture of JFK right before he was shot. So she was really like front row witnessing the assassination and she remembers this moment. So she really, you know, take you through the path of the motorcade in Dili Plaza, coming from Main Street to M Street and, you know, passing in front of the Texas military. So she was wonderful because she remembers very vividly that day. And it was very powerful to hear testimony and be able to you know, just dive into their memories, really. So the first chapter is that, so you really follow the motorcade. And of course, we didn't want to show any graphic or violent content. So the archives and film photos didn't show any of this graphic content. And so the second chapter is happening in the Texas Book Depository, which is a building from which Oswald shot. And it's told by Helmut Boy, who is actually the last living detective who worked on the investigation. So he's telling you how the police got to the scene, how they searched the floor. And, you know, this sixth floor, which is now the museum at Dili Plaza, it was full of boxes. So you're among the boxes and you're searching the floor with the detective. We have some arcade photos reprojected. And they actually found the rifle and some of the shell casings on that floor. So you're being able to have the view Oswald was supposed to have when he shot the president. So, we're leading you through this search, and the end of these two chapters are when some of the policemen realize that there is an employee missing of the Texas Coup d'Etat, which is, of course, Lee Harvey Oswald. So, the next chapter is going to be about how they arrested Oswald and how they searched his houses and understood, you know, who was this guy who was living in the Soviet Union. So it's all about the investigation, how the detectives gather evidence. And the last chapter is going to be at the museum in 2023 with the curator of the museum, Stefan Fagin. And this one is going to be more about understanding the context surrounding the assassination, especially diving into the different conspiracy theories that were created and invented after the assassination and also the official investigation, of course, especially the Warren Commission, who was commissioned by Johnson after the assassination. So with him, we're just going to dive into the legacy of the assassination at the museum right now with, of course, an augmented experience at the museum, being able to dive into the panels, having this kind of temporal gate to some of the scenes in Dallas. So yes, and these last chapters are going to be available at the museum. We're going to show them there. So I feel that people would have the opportunity to just understand these events in a completely different way. And of course, also explore some of the locations around Dallas that are not at Dilley Plaza.
[00:33:24.535] Kent Bye: Yeah, I felt like that there was, especially the second chapter of being able to be on the scene and the translation of the investigation, but to kind of feel like being in that space, I thought that was a component of that story that I'd never experienced in that way at all. And so that was nice to be situated into that and see how that unfolded. And also just the interviews that you're doing, I noticed that you're doing this interesting hybrid approach where I'm in what feels like a photogrammetry captured, but it's a CGI created, but yet you have some sort of either billboarded or volumetric capture of the people that are being interviewed and so maybe you could talk about that decision as to you know have this more grounded in the CGI rather than 360 video because you know you do obviously a lot of 360 video but at the same time this is a piece that's maybe a center of gravity of having a CGI that maybe you would find too much of a break of presence across jumping back and forth between CGI and 360 so yeah I'd love to hear you elaborate the decision there to go more of the CGI route with a bit of a billboarded video.
[00:34:24.356] Victor Agulhon: I think the starting point of this is, this is our first 6DOF experience and we believe that once you've given people the ability to move their head around and to explore the place in an interactive way in the sense that they can move their head around, they can look through the book boxes, they can look through the window, they can try to like they feel free in that space and they choose their own perspective if you wish in some sense. It was difficult to kind of like alternate between this amount of freedom and being stuck inside a 360 film. We like the idea of being in the 360 film if that's what you're in, but transitioning from one to the other we thought would have been a bit difficult to like explain to the people why it would change at some point and from a user's perspective, from a viewer's perspective, it was important to retain that kind of like sixed off aspect across the entire experience. Then the question comes into when we do 6DOF, when we do CGI, 3D recreation, we still want to show the real people. We still want to connect with history, we still want to connect with the witnesses, so recreating them in 3D and having them animated was not an option. And so we thought that the best option would be to actually film them and integrate them into their environments that are recreated in CGI. So if you were with us on the shoot, you would notice that the places where you see them in the documentary are actually their living rooms. And we found them in a green screen setup in their living rooms, but then we recreated the entire setup in VR. So it's kind of like a weird experience to go through, but... In the end, we just wanted to have the ability for the moments that you have them on screen to just see them in person, having them moving naturally, you know. Every detail of it is so important because you can't have an experience that's only about the location. It has to be about the people who lived through these stories. And so we had to remain into this entire interactive 3D environment. That's the best solution we found today.
[00:36:04.924] Kent Bye: And they were far enough away that I really couldn't see much stereoscopic effects, but did you end up shooting in 2D and just billboarding it into the virtual space, or did you actually shoot volumetrically?
[00:36:14.044] Victor Agulhon: So they are shot in 3D, so in stereoscopic video, but then at that distance it creates a small sense of better integration in a space, but obviously if we had volumetric capabilities it would be even better integrated. When we started that production that technology was not yet available and in terms of performance we thought that this The danger is always to kind of like end up in a sort of like zone where it looks unreal. And this is the best way to have people looking real. So it's always trade-offs that you have to make, especially when you're doing 3D. But in that sense, the most important element for us was that connection with the real people feeling like they're alive in front of you. And that definitely was the best choice.
[00:36:49.341] Kent Bye: Is this the first 6DOF piece that you've been a part of helping to direct and produce?
[00:36:53.869] Chloe Rochereuil: Yeah, absolutely. So it's the first 6DOF experience. We've done a lot of 360 stereoscopic video at Targo. And I feel that there is a certain kind of topic that are really powerful into this kind of content. And some topics are, you know, better in 360 video and some other topics are better in 6DOF. So I feel that there is no, like, one or the other. But it's more like, for example, for Behind the Dish, it would have been impossible to have, you know, this connection, being able to be in the environment, being able to see the food as it is in 6DOF. But for JFK and especially I think for historical content broadly, 6DOF really is a good fit because we have this ability to recreate environments and there is not a need to show live action or things that would be better in 360 video. So I feel that it was great to work on the 6DOF project and I feel that working in 360 video is also super powerful for other topics but for this one being able to just recreate the location as it was in 1963 it's something we couldn't have done in video so it was just a natural way to do it in 6DOF because there was this need to be back in time and to recreate this location very accurately so it was a quite natural choice for us to do it in 6DOF.
[00:38:12.652] Victor Agulhon: Just to add on this, because I think once, when we did that transition to 6DOF, the question was also how interactive does a piece like this has to be? And at the beginning, we had a much more interactive approach, like you could grab some of the objects, so not obviously not the rifle, not anything that would be kind of like sensitive in terms of like narrative, but you know, maybe you could have like grabbed some books here and there, and there was like, we had an approach which was much more focused around recreating the investigation, maybe you could take some of the pictures that the police took, so we did some iteration and tests around it, and then we really realized that our type of documentaries, our style laid like really in documentaries and that means a linear editing. And so even though I think naturally when you have the ability to use game technology people expect interaction, we realized that this documentary would be much more powerful if there was no interaction. So today the only interaction that we have and I think this is one of the decision process that was probably the longest for us was to land and be happy with the fact that there would be no interaction, and that the only interaction that people will have is the ability to tilt their head, move around, and it's a slight interaction, but I think it's what makes the entire difference. Because when you're in the sniper's nest, and when you're looking at all the details on the floor, when you're looking at where the bullets are, you can like move your head to look through the window to see, you know, was it possible? Did he shoot from there? How far is it? All these things, these are things that you can only understand if you are able to move your head and play with the parallax around that. And so these are small interactions that don't require you to use your hands or a controller, but that's what makes the piece come alive. And so for that reason, we've been always, you know, thinking about what are the pros of going six off and what are the cons. And the pros is definitely that for a topic like this, it gives people the freedom to like look where they want in the way they want. And for a topic like JFK, like the assassination of Kennedy, it is like, I think a very important sensation for people to have to feel like they're kind of making this story their own as well.
[00:40:01.097] Kent Bye: Will this experience of JFK Memento only be available as a location-based experience in the museum, or is this going to be also widely available for people to see?
[00:40:08.859] Victor Agulhon: It's also going to be available more broadly. We're looking at the specific details today of how we're going to be distributing it. There's obviously options of going video and 3DOF online. There's also the option of bringing it to the Quest store, but all of these are like right now decisions that we are making. But the goal is obviously to have everyone being able to experience the entire 6DOF experience on the store. That's definitely our main goal.
[00:40:28.542] Kent Bye: Awesome. Yeah, looking forward to having that out there for more folks to be able to see it at some point as they come up on the 60th anniversary. So yeah, I guess as we wrap up, I'm curious what each of you think the ultimate potential of virtual reality and immersive storytelling might be and what it might be able to enable.
[00:40:44.971] Victor Agulhon: It was so unexpected. We really didn't expect that question. Well, now that you ask it, I will say experience more in less time. It's really what we do. We contract time and space with virtual reality. We don't aim at replacing any of the dimensions, any of the experiences of the real world. But we do think that we can give people a glimpse into more things in the world thanks to virtual reality. So that's what we are doing today with our documentaries. But if there's one ultimate potential, I would say contract space and time for people.
[00:41:14.112] Chloe Rochereuil: And I would say that connect with the real world is something that we are very dedicated to because I feel that there is a need today to be able to focus like 20 minutes in a row, do not see any other screens around you, like really being focused on something is really something that VR can do and there is this demand, this need of attention from the people and I feel that being able to connect them with the real world and taking VR as a way to really put them into real environments, meet real people is going to be probably like one of the ultimate potential of VR because like today VR is obviously something that works really well for games and experimental experiences. artistic experiences. But I feel that documentaries really feel something special in VR. And I feel that it could be the ultimate potential of VR.
[00:42:05.905] Kent Bye: Awesome. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:42:12.150] Victor Agulhon: I think, as usual, great conversation. And we've covered everything. So I don't think we need, I mean, I don't personally don't need to add anything. Thank you.
[00:42:19.641] Chloe Rochereuil: Yeah, same thing. I mean, thank you so much for taking the time to speak with us. It was great to be able to just be there, talk to the creators, and being able to just talk with you is always a pleasure.
[00:42:31.791] Kent Bye: Yeah, well, you say you aim with Targo Stories to be one of the leading immersive documentary creators. And I think you're certainly right up there in the leading pack alongside with Felix and Paul and all the stuff they've been doing. I feel like both of the work each of you are doing are kind of pushing the medium forward in different ways. So I really appreciate always seeing what you're cooking up next, especially with the 360 video and stereoscopic and now moving into 6DOF as we move forward. So looking forward to seeing the rest of JFK Memento and other experiences that you have that I'm sure you're already working on that we'll be seeing at some point. So thanks again for joining us here on the podcast today.
[00:43:03.059] Victor Agulhon: Thank you, Kent.
[00:43:04.220] Chloe Rochereuil: Thank you, Sumit.
[00:43:06.201] Kent Bye: So that was Victor Argelon. He's the co-founder and producer of projects at Targo Stories, as well as Chloe Rochele, who is also the co-founder of Targo Stories, but also the creative director of all the different projects. And so, yeah, they're producer, director, and all these different pieces from Targo Stories. And they have a lot of really amazing pieces that they've been doing over the years. When I had a chance to talk to Blake Hemendiener about this year's selection, he says it's kind of rare to have two projects by the same director-producer combination, but the Behind the Dish was such a strong piece to include in the retrospective look, as well as the JFK memento was also doing all this really innovative stuff when it comes to telling the story of the JFK assassination. So I've a number of takeaways about this interview is that first of all well target stories is aiming to really be pushing the edge of 360 video immersive storytelling and I think year after year they continue to innovate with the structures and forms of storytelling and Really glad to see them moving into six to off experiences to see how they're able to create these new fusion of techniques and so they for doing an interview for the different subjects in GFK memento and They ended up actually creating photogrammetry, 6DOF experience of these different places, and they would shoot in that same location with 360 video or with a stereoscopic lens. And so you have the spatialized view of them, but they're far enough away that it kind of feels like a 2D billboarded experience of them anyway. It's a physical capture of their embodiment, but in the environmental context of their world And so they're fusing these different techniques also doing lots of fascinating things with taking 2d photos and images and creating like a whole 3d rotoscoping process where they're able to create this stereoscopic effect by you know filling in the gaps of what would be occluded from one the eye or next but to just give this sense of space and depth and They were doing that both with the photos and images. They're not relying upon any AI right now. This is all a custom handcrafted beast Oh process I'm imagining at some point the type of stuff that they're able to do with their hands They're able to potentially redo with this process with AI at this point with this archival footage whenever you're going through this processes you have the risk of potentially changing or altering it in some significant way so They're able to get like 70, 80% there, but they still would have to do it manually anyway. And so both for maintaining the integrity of the archival nature of this footage, but also to not have to replicate some of the labor that they would already have to fill in the gaps of what wasn't done properly by the AI. But this is potentially a type of process that in the future that will be more widely distributed. at the bleeding edge of trying to push forward what is possible. So they're able to create this spatial context of what happened with the assassination. And I'll be curious to see what the next four episodes go into arresting Lee Harvey Oswald and to go into some of the different conspiracy theories, because there's a lot of stuff that actually hasn't been published about what actually happened with that and all the different conspiracy theories that have come up. They're going to be covering that as well. But it's done in collaboration with the Sixth Floor Museum that was happening in Dallas, Texas, which you can actually see from this Point of view and have folks come in and have this whole history around that and they kind of take you into Seeing that a little bit when you're in this immersive experience So it creates a piece that could be shown on site and be able to see it, but you could also see it at home But then still have the desire to go see all the additional information and context at the museum So yeah, really interesting to see how they continue to push the structures and forms for they did the surviving 9-11 piece that also was very innovative and have a whole breakdown of that piece as well with them and And also just their whole series of Behind the Dish, I think, just really solid storytelling in each of these different character portrayals and to both highlight their process of creating food, but also to dig into more of the place and location that's unique to how this food is being made, like going up into the hills. Wasabi harvesting was an amazing use of 360 video to be able to go and see these really epic locations. Yeah, sometimes the video that they create is more of a 180 type of thing where there's stuff that's happening behind you but there's not much stuff interesting so you end up just kind of looking forward. But other times I would stand up and look around because it's easier for me to do that while standing than it is to sit down. So I found myself actually standing and sitting when I was watching this piece. I would stand up when I want to kind of get a sense of a place and then I would sit down when it was just watching what was happening unfolding in front of me. So the behind-the-dish is something that you can go watch and check out and I highly recommend doing that and then yeah the JFK memento We'll see if that comes out, but they have aspirations of being leaders of immersive storytelling Especially with 360 video and also now getting into sixth off storytelling And so yeah, definitely keep an eye on what target stories is doing with these because yeah both Victor and Chloe are really pushing Having a lot of different innovations that they're doing that are really pushing the forms of storytelling forward And yeah, really just excited to see how they're doing that in the context of these pieces So, that's all I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
