Kusunda is a spatial documentary about a indigenous language in Nepal that is on the brink of going dormant and falling asleep. It’s an incredibly powerful story that uses photogrammetry, Tilt Brush interpretations of oral history, and interactive natural language processing actions that help teach the interactor a number of Kununda vocabulary terms. It’s a production that had to adapt to COVID-19 by innovating other metaphoric & spatial ways of telling this story.
I had a chance to talk with Now Here Media co-founders Gayatri Parameswaran and Felix Gaedtke after their Tribeca World Premier in June, 2021. We explore their journey producing this piece, the special considerations telling this story spatially, and the deep listening involved in producing a piece like this.
Kusunda won the Tribeca Storyscapes Grand Jury prize, and is currently featured out of competition at the Venice Film Festival VR Expanded selection until September 19th.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. So I'm going to be doing a series of interviews of projects that were featured at the Venice Film Festival 2021, starting with Kasunda, which actually won the grand jury prize of the Storyscapes back at Tribeca of 2021. And it wasn't generally available to be able to be seen, you had to actually go physically see it at the installation, but at Venice Film Festival. It's actually in competition, and you'll be able to see it if you get accreditation with the Venice Film Festival. But Kasunda is a really amazing story about a language in Nepal that is going dormant. It's falling asleep. And it's actually a story that's told across multiple generations, and they're able to actually capture it in a really compelling spatial way, what I describe as a form of spatial anthropology. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Felix and Gayatri happened on Friday, June 11th, 2021. So with that, let's go ahead and dive right in.
[00:01:10.332] Gayatri Parameswaran: My name is Gayatri Parameswaran and I am co-founder of Nowhere Media, an immersive storytelling studio that explores how we can achieve social impact through storytelling and in immersive formats. Yeah, my background is in journalism and documentary making and approximately six or seven years ago, I happened to kind of discover virtual reality as a medium and as a format and fell in love with it for the potential that it offers for the kind of stories that I want to tell and we want to tell at the studio. And there has been no looking back. And I'm also co-director and producer of Kusunda, Speak to Awaken, which is a voice-driven virtual reality experience about the sleeping Kusunda language in Nepal, which is currently having its world premiere at Tribeca Storyscapes.
[00:02:12.203] Felix Gaedtke: Yeah, my name is Felix Gilke. It will be a bit repetitive. I'm also co-founder now here. I actually have a background in journalism and documentaries and yeah, fell in love with VR mainly because I really love this idea of like exchanging perspectives. And that's something I have been trying to do in my work even prior and yeah, love to use this opportunity to do that in this room as well. And I'm also the other director producer for Kusunda and excited to see it now come to life or be presented to the world.
[00:02:49.688] Kent Bye: Great. Well, I had a chance to watch Kusunda and it's a really amazing story. And before we get into some of the aspects of the piece that may be a spoiler, I'd love to maybe kind of do a transition here after we kind of cover a little bit more of the generalized background. Maybe we could get a little bit more of a context and backstory for how this project came about. Cause I actually think I, I saw you give a pitch for this. It was in January. It was one of the Kaleidoscope VRs, which now is Artisan. They were doing a number of different pitches of different projects. And I saw you give a pitch for the project. And so where the pitch was and where it ended up was two different things, but maybe we could dive into like the evolution of how this project came about.
[00:03:30.360] Gayatri Parameswaran: Sure. So actually, Felix and I have, as we mentioned, we've been working as journalists in many different parts of the world for the last 12 years. And in 2012, we were in Nepal and we were based in Nepal for a few months and working in the region. And through a chance encounter, we got introduced to a linguist in Kathmandu who had been working and researching on this language. He told us, oh, it's a very special language. It's called Kusunda. It's a language isolate, which means it's a language that's unrelated to any other languages of the world. And there are very few speakers left. At that time, he said there's only one speaker left. And it's a 75 year old woman who lives in this village. You have to go and meet her. And the idea that there would be only one speaker of a language at that time made us think about, you know, what it might feel like to be that person. And, you know, is it a sense of loneliness? Is it a sense of not being, I couldn't imagine not being able to speak my mother tongue to anyone. So we kind of, you know, said, Oh, let's go and meet this woman. And then we took like a 14 hour bus ride, and then we had to cross a river on foot. And then we arrived at her home and met Ganymede. And she struck us as this really amazing, inspiring woman. And Felix, maybe you want to take it from here, because I don't want to explain all the backstory, but give you also the chance to to
[00:05:13.005] Felix Gaedtke: Sure, so Gyanimea at the time told us that she grew up in the forests as a child as part of this hunter-gatherer community, Kusunda, and then eventually settled with her parents when she was a teenager in the village, and how she tried to blend in into the society in the village in Nepal by dismissing or neglecting her Kusunda identity. But she obviously knew how to speak the language and kept it a secret for many, many years before linguists heard about her. And she then went on this path where she really wanted to revitalize the language and help people to understand better the Kusunda language and culture. And yeah, at the time she was also just an inspiring woman because she was always busy working. She was climbing on trees and plugging mushrooms in the forest. She was just an amazing character and person. And at the time we did a small TV reportage about her, but we knew that we always wanted to do something bigger, longer with her. So we kind of decade or so. And slowly this idea developed once we were working more with VR, that that would be a great way to approach this. Also because a lot of what defines the story happened in the past and it's already over and has to be kind of reconstructed. And yeah, so we started speaking with her and she at the time had also started then to give language classes to like village children in Kasunda. So she was super excited to do something like this and we developed the story idea together. It took a long time to get there and to also raise the funding and so on. Once we were at the point that we were going into production to Nepal, basically the day before we left, Gyanimeya passed away. which was a big blow to the whole project and to us personally as well. So we were not sure what to do, but since we already had flights booked and a Nepali crew waiting in Nepal to work with us, we went ahead and went there not being sure what we would do. So we first went to her family to pay our condolences and then it became pretty clear that they really wanted us to continue with the project. So they were really pushing and saying like, she really was excited about this, so please go ahead and do something. Which was easier said than done, because we basically were in some ways back to square one. So we were trying to see how we could tell the story. And then we remembered that there was one student that she had been speaking about, Shima, that was like one of her best students. And we thought, maybe we can work with her. And then it turned out that Hima's grandfather was a tsundere shaman who has lived in the forest for over 40 years and was also an extraordinary character. So we kind of met with both of them and discussed the project with them. And then, yes, picked it up from there. And then, in a way, we're able to pivot the whole story and idea to something a little bit different while still keeping the main idea and concept alive.
[00:08:24.803] Gayatri Parameswaran: Yeah, if I can just add to that, I think the biggest lesson as a documentary maker that we've learned as documentary makers is that you can't control reality. And it's about letting the story guide you and letting the process guide you and keeping faith in that rather than trying to control. I mean, there was nothing in our control at this point. after we had learned of this huge loss. I mean, you know, our project is subordinate to the larger scheme of things. We were working on this theme and the subject of disappearing languages. And as we were documenting it, we had witnessed in some ways, the loss of someone with so much knowledge. And we hadn't, we felt so bad that we hadn't managed to archive that prior to this event, this tragic event, and we were completely lost. But sometimes just keeping faith and going forward and trusting your instinct that there's more pays off. And we were so, so fortunate and grateful to Hima and Lil Bahadur, who there's a kind of a triangle over here where Lil Bahadur had been Gyanimeya's shaman and had been treating her through her illness. And Hema had been learning the language from Gyanimeya. So it was all kind of an interconnected net that we were able to show somehow through the piece. And I really hope that it comes across. And basically, also in the larger context of language disappearance or languages going dormant and falling asleep, Every two weeks, a language falls asleep. The language that we use around this is very deliberate. We don't say that languages are going extinct because languages are not like dinosaurs. They don't go extinct. They can be revived. Dinosaurs cannot come back to our planet, but actually there are so many cases of languages being revived. Until now, we know in the century, at least 20 languages have been revived. but it takes a lot of effort. And there is such a close connection to what's happening in our Anthropocene area. You know, it's a symptom of what we are doing when it comes to climate change, when it comes to cultural eradication. We are destroying, as human beings, we are destroying a lot of wealth on the planet. And if you overlap the map of the world that's affected by lack of biodiversity or where biodiversity is threatened at the moment and where cultural diversity is at threat, there is a correlation over there. Places where biodiversity is at threat are also places where culture is at threat. And a lot of the times people who are at the front lines of this are indigenous communities fighting for their land as well as their culture. So, you know, in the broader sense of things, I just wanted to put that out there.
[00:11:42.797] Kent Bye: Yeah, so I think we're certainly clearly into the part of the story that's going to dive even deeper into some of the potential spoilers. So I highly recommend folks, if you haven't had a chance to see Kasundi yet, definitely try to go and watch it somewhere. Well, before we go too much further in that, I'm curious, is this project, do you have plans for it to be widely available for people to actually be able to watch it?
[00:12:04.103] Felix Gaedtke: Yes, we plan to distribute it online once we are a little bit further in the festival circuit.
[00:12:10.219] Kent Bye: Okay, great. So if you're listening to it now, this is probably a good place to stop and then you can dive in. Cause I do think that there's part of the story that we really have to kind of directly experience and we're kind of already to the big first turning point, which when I had first heard you pitch, I was like, wow, this is like an ambitious project that seems to be kind of hinged on this one person. And then the day before you leave to go start this project, she happens to pass away. But then, you know, I have to say that the project ended up taking a turn and maybe went in direction that. pushed each of you to be able to maybe use the medium of virtual reality in a new way or ways that you weren't originally planning on. I'm really struck with how you were using a range of different types of volumetric techniques. You have the depth kit type of volumetric capture where you're doing the interviews, but then you have the photogrammetry of the different scenes that you're in, which I thought were quite evocative in terms of where you were at and the landscape of where you were at. really helped to ground the conversations into a very particular place. But then you use the Tilt Brush-esque animations to be able to paint a larger picture of the context of the forest and going back into these childhood memories and stories and trying to, what I describe as rather than just visual anthropology, where you're taking photos or videos, almost like a spatial anthropology of an entire culture, that you're trying to describe what it was like to be a part of this Kusunda tribe. So maybe you could talk about that evolution of this turning point and maybe a little bit of what you had originally planned for what kind of volumetric techniques that you were going to use. And then when you hit this turning point in the project, how it pushed you to go in this other direction that may be exploring other volumetric techniques.
[00:13:54.094] Gayatri Parameswaran: Yeah, sure. I think we both feel, and I'm speaking for you Felix here, I think we both find, you know, coming from this background of non-fiction or creative non-fiction, we find that realistic representations of environments in virtual reality is very, very powerful. And this is a tool that allows us as creators to take our audiences and take them to places where we have been and make them participants in the story. And I don't want to underestimate that power. I think photogrammetry is something that I wish, you know, that's something that I feel is a very, very powerful component of the story, which allows for this as what you call a spatial anthropology. And I mean, you know, for us to be able to even do that, I think it was this close collaboration and co-creation with Lil Bahadur and Hema, like two of our co-creators in this project, that allowed us to even develop the concept and the idea and the way that we did that. And it became very clear from them that Hema's need and her generation's need is to understand their history and their tradition while still looking at the future. We had to adopt a visual language that could help this generation. Hema is 16 years old and her generation, we had to find a language that would enable her generation to look back at something proudly and look optimistically towards the future. So that's when we thought, okay, Tilt Brush, together with our art director, we identified that after a lot of experiments, Tilt Brush offers a very organic way of representing images. So the organic nature of brushstrokes fit very well with the area and the geographical region where we were working and their way of expressing themselves and being quite close to as natural as possible. And we used motion capture. We had support from the Zurich University of Arts to help use their motion capture facilities. I mean, we wouldn't have been able to use motion capture, otherwise it would have been very expensive. And then we got into these motion capture suits and started recreating the past. And that's also when we realized, oh, that was one of these moments we thought of the digital divide between the community in Nepal and us, you know, us as creators in the West, in some ways. Although I'm not a Western storyteller, but I'm based in the West and the kind of access that we have to technology is very different from the kind of access that they have to technology. And that was a really complicated process where we had to be sure that, is this how we need to represent something? And a lot of collaboration and close working with them made it possible to get over that hurdle, but that wasn't easy. And maybe Felix, you want to speak a little bit about the volumetric gap?
[00:17:16.979] Felix Gaedtke: I just wanted to add something regarding the animation. So the Kusunda no longer live their traditional hunter-gatherer life. So the world we were trying to recreate was mainly based on anecdotes and stories we heard from Gyanimea, as well as the interview with Lil Bahadur. And especially Lil Bahadur, I mean, he's 86 years old. He's an older man. his memories are kind of blurry and not that clear and not that focused and kind of insular, if you can say that. So that was something we really wanted to also convey in the animation, rather than trying to pretend that we have a full picture of how the past was, being able to maybe allow for some little glimpses into a world that no longer exists as such. So that was, I think, also really behind the idea of how the animations look, because they're very fragmented. Yeah, and the volumetric capture for us was to try to see how we can get both Hema and Lil into their own space when they tell the story. We were working with Azure Connect and DevKit hooked onto a camera and well, the whole process was quite challenging because initially we thought maybe we can get even a bigger setup and then we realized, well, where we're going to do the interviews, like some small hut on the top of a mountain where you need to get, and setting up the makeshift volumetric studio there was very challenging. And we had constant issues with electricity blowing out and goats walking through our green screen and all these kinds of things. And yeah, so that was a very interesting way of working and allowed us to really put both of them in the space that they actually live in.
[00:19:10.690] Kent Bye: Wow. Yeah, that's really fascinating to hear how you're talking about the photorealistic aspect of photogrammetry. And then as someone who's really interested in capturing the realism, you're forced by the very nature of the story to dive into the aspects of like Tilt Brush to be able to abstract these memories. I think it works really quite well because it gave me a sense of spatial context as you're able to even go up to different scales and really show perhaps some shots that you could have maybe gotten with the drone footage. But I think you're able to compose with the animation and tilt brush and create scenes that really give me a sense of this space and how different people, different places are related to each other. But there are a couple of moments in the piece where you ask me to make a choice. Which is interesting because I watched all the way through and then I didn't go back and watch it all the way through the second time because it was like, I don't know, I would have loved to be able to just kind of dive into those other scenes that I didn't watch as someone who's somewhat of a completionist as I go through these different pieces. But maybe we could talk about that decision to be able to make a choice and show different aspects of the story while including other aspects of the story. And I have one version of the story where I made those choices, but, you know, I'm kind of curious as you're creating this piece, what led you to kind of have this choice point for the creators and what those different trade-offs are when you start to do something like that in a piece like this.
[00:20:33.691] Gayatri Parameswaran: Yeah, we had to make choices artistically as well as quite practically. And initially, we had the idea of just a very linear version of the story. So first of all, I guess it's important to know that it's a voice-driven piece, which means participants have to learn certain words in the Kusunda language and say those words out loud to move further in the narrative. And this choice was also guided through our co-creators where Ganymede was a teacher of the Kusumda language and she wanted people to learn the language as well as Hima, her student, who said, I want people to say these words in my language if they want to access our stories. So it's a kind of a give and take. So both these intentions and wishes that they have or the requests that they had be consolidated in this voice interaction. Initially, in the prototype, we had started creating a neural network that was trained on these words on the Kusunda language and that would be able to recognize these words and then that would function on its own. However, very soon we realized, and this also reflects on the phenomenon that we are discussing here, Artificial intelligence is not for minority languages or low resource languages, such as Cusumda, especially as Cusumda who have only a handful of speakers left. It relies on massive amounts of data. which is impossible to do as an independent team. And very soon, you know, after the prototype, we realized, okay, we ran a campaign where we had a website that was collecting these particular words in the Kusumda language. And then we trained the neural network on that, based on that. And we realized very soon, oh, it's not working. We need to just hack the system. And it shows the imbalance in this field as well, in linguistics, when it comes to using artificial intelligence, you're pitted against a system that relies on majority, you know, on something being spoken widely, such as English or Spanish or on any of these big languages. And this is also the reason why languages fall asleep is when societies, groups, and communities are forced to integrate into the mainstream and speak and adopt languages that are spoken by a majority. And basically, we said, okay, let's then try to use the English Microsoft speech recognition and then just adapt it to be able to recognize Kacinda words, which works pretty all right. But it also kind of shows how in order to fit into the system, we just need to hack the system. There's no way we can build our own system being who we are and working on the subjects that we are working on. And therefore, so basically there are two points where you mentioned, Kent, there are choices that people are allowed to make. And Quite practically, we realized that if we are taking the piece to museums or festivals, people would want to have a footfall. And 20 to 25 minutes was the sweet spot that we found we needed to tell this story. So on a very practical level, we broke down the script and said, OK, what are the parts of the story that offer a smooth narrative arc and allow you to understand the story completely, even without having to do all of it? And we realized that there were some parts of the past where, like Khalil Bahadur is speaking about his life in the forest and the kind of challenges that he had to face. We thought that audiences would understand the gist of it if they had access to just one arm of the story and they didn't need to have. Also, similarly, it was about whether he was going to be alone in the forest or the moment that he met his lover or then later wife. So both aspects of the story allowed you to understand the different side of Lil Bahadur's life and his personality. But you didn't need both to understand the larger picture. And for at-home users, we've given the option of chapters. So people can go back and go back to the chapter that they want to do the other branch, and then be still able to do that.
[00:25:06.014] Felix Gaedtke: Also, just to add to this, conceptually, I think there was the idea to not like reveal everything in once, because it is kind of like, anyway, we don't show the Kusumda culture as a whole, would never claim to do that, but like in a way how these cultures and story is like a little bit hidden in the same way. We also wanted to create some sense of mystery for what else is there and some curiosity later. And yeah, that was also part of the idea.
[00:25:34.902] Kent Bye: Oh, okay. Yeah, that makes sense. And thanks Gayatri for going into the voice activation aspect, because I do think that that's a significant part of the story. And I'm struck by how, when I watch virtual reality documentaries and I'm a native English speaker, often things will be dubbed over or with at least English subtitles. So in some ways I'm in this privileged position to be able to consume some of these stories, but yet at the same time, with virtual reality being such a nascent technology in order to really sustain yourself and making these type of projects, it always has to be dubbed back into at least English. At least that's the current culture in terms of where things are at, which in some ways starts to like, just like you said, how the artificial intelligence doesn't recognize the language. And so it puts people in position if they want to start to use these new features. It's sort of like a digital colonization in some ways of having aspects of the culture, in this case, the English language, spread through the technology that in order to engage with the technology, you have to speak the English. And so it's really quite interesting to hear you speak a little bit more about how you had to really break down the language down into the phonetic spellings and then from there still kind of hack the system just to give it to even work. But I know that I had a conversation with Elena Herzog, who did a piece that was at Sundance a number of years ago called Last Whispers, which was also looking at this aspect of languages going to sleep. Although she called it going extinct in her context, but I think I appreciate the aspect of these languages that go dormant and go to sleep and that they can be revived. And so I guess as I was watching the piece, that was the thing that was really striking to me was just this reflection of death and loss and there's a part of the larger context of the coronavirus. And it was almost like the story that you were capturing was also in its own way, mirroring what was happening in the outside world. And so you kind of have the breaking of the fourth wall of the context under which that you're making the story and the global pandemic that was happening in the midst of all of this. And then I guess the themes of death and dying, but then the Phoenix rising from the revitalization of something where there's hope that it's not not quite dead yet, as they would say in Monty Python, but there's this revitalization and this rebirth aspect of it. So there's these themes of death and rebirth, but also you were dealing with the pandemic and how the pandemic was very directly impacting the production of this piece, but also the themes of death and dying reflected within the piece itself. So yeah, I'd love to hear any reflections on that as you're making this piece and trying to really focus on this story, but yet how aspects of the wider world were infiltrating into it and these other aspects of this digital colonization that can tend to happen with these new emerging technologies.
[00:28:22.580] Gayatri Parameswaran: Yeah, I think one thing, so basically just the context, we were in a timeline. We were in Nepal in February 2020 and we just basically got one of the last flights out of Nepal before the travel bans around the pandemic started. So it was a very rushed kind of exit that we had to make. And our plan was to go back and forth many, many times to be able to work closely and in a collaborative, co-creative manner with the community. And we couldn't do that in ways that we had planned because of the pandemic. And everyone planned things that didn't work around the pandemic, so that it's nothing different with this production. One thing we realized is that, once again, something I spoke about, this digital divide. We live on one planet, but we have very, very different realities. I've been trying now to contact Hema for an interview or a discussion, panel discussion that's coming up next week. And although she has a phone and she has internet, but the internet doesn't work half the time. And I mean, I do believe that communities can choose not to be connected. I'm not saying that the prescribed model of development from the West that's handed down needs to be applied and adopted by communities around the world. That's not what I'm saying. But I just think that it's really good to acknowledge the privilege that we have all of us have in our ways and to recognize that there are communities and people who do not have access to the same kind of technologies, the same kind of access and connectivity. So when we were developing the story and going back and forth, and on top of that, virtual reality is so inaccessible and exclusive, right? So when we were creating the piece, we cannot even buy a headset in Nepal. We tried for three weeks to get a headset in Nepal, and it costs three times the money that it would cost in the US or in the EU. which already shows who is advantaged and who is not advantaged, and that the people who actually have access to resources are also the people who are advantaged in the pricing of these devices. Anyway, sorry, I'm just really upset about this inequality is something that really upsets me. But so we found hacks and ways to go around it. So with Hema, what we did was we realized we needed to create a video version of the piece where we were going to kind of narrow down on what parts of the story is told and how it was going to be told. And we started sharing things in video. So we have like a linear video version of the piece. which enabled us to go back and forth with them in getting green lights and saying that, oh no, can we instead do it this way or that way? And once again, to say that Hima lives in a town now, she goes to school where there is more accessibility, whereas Lil Bahadur lives in the village where the filming happens. The film is located and situated, which is even more remote, right? So he doesn't have a phone, there is no internet. He's not someone who is willing to have those, to even engage with these technologies. And the way he looks at technology is also really interesting. For instance, when we were using the drone to capture the valley, he called the drone an eagle vehicle. So he has his own analogies to look at technology and make sense of it. which I think is really interesting because he connects that to nature, whereas we are so disconnected from our natural surroundings that we won't even think of a drone as an eagle vehicle, but that's what he calls it. So we have this linear video version that allowed us to actually make the script concrete And then we started working in virtual reality. So everything we produced in VR, we always had a video version of it because we were going back and forth with people who didn't have access to a headset. And in this process with Lil Bahadur, what we realized is we had phone calls with him. And at one point, we just have to send letters and written texts. So we were dealing with a very high end technology. And at the other hand, we were actually writing letters, describing things in Nepali to Lil Bahadur and having, he also can't read or write. So there was someone else actually reading out things to him on the other side. There were a lot of bridges to cross and the pandemic just made it, you know, if we had been in the same place, this process would have been much easier. But the pandemic in some ways, I'm grateful, helped us reflect on what we were producing. And I hope that that also comes across in the piece because We think that as documentary makers, an important responsibility that we have, especially working with communities who are marginalized, who don't have the access to the same kind of resources that we have even as creators, it becomes very important to be transparent about the process and to acknowledge our flaws and our privileges in the process.
[00:33:45.558] Felix Gaedtke: Yeah, I think that's a very important point. And that's, I think, why we brought us into the story or like kind of the making of into the main narrative, because it didn't happen in a vacuum, you know, like, it's such a unusual thing for them. And for us to work with these kinds of technologies in spaces like this village. So we kind of like wanted to be transparent about it and also show what that actually means and that it's not picture perfect and that there are these little moments that indicate this is like something we are creating together and it's not something that they are developed in a vacuum and then it's just done and finished.
[00:34:28.089] Kent Bye: Yeah, I definitely think it comes through and it kind of reflects the quality of the moment of the time of both your own personal journey in terms of creating this piece, but also all the ways in which that you have this digital divide manifest in this particular context. And I wanted to dive into what I thought is the emotional climax of the piece where Ahima is singing a song that's written in Kusunda to her grandfather, who grew up speaking Kusunda up until he was 18, but yet stop speaking it. And he expresses the sadness and remorse that he's essentially forgotten his mother tongue because it's a language that's been falling asleep. He hasn't had a chance to be able to continually speak the language. And if you don't speak a language, then it kind of fades away from your memory. And so here's someone who is taking care of someone who is one of the last fluent speakers as she's dying and reflecting on his own inability to speak his mother tongue while he is listening to Hima who was taught by the last fluent speaker to be able to sing a song and then You get through this amazing, beautiful song that just is so poetic. And as you're going through the forest, and then you talk about how you stumble upon this footage that was almost like a prophecy for how this was all going to play out. I guess the question I have is what was it like for you to go back and to capture the song? And did you forget that you had that clip or what was the moment that you realized that you had this full circle moment where you had some footage from 2012 that was predicting a moment that was unfolding in front of you?
[00:36:03.068] Gayatri Parameswaran: Yeah, actually, the last footage is actually from 2018. At the back of our minds, we already knew about Hema because she was learning the language from Gyanimeyam. And when we met Hema, we didn't know that Lil Bahadur was her. We had met Hema and Lil Bahadur separately because Hema was at the time in school and in the town and not up in the village. And separately, we met Lil Bahadur. And it was only a few days later that we realized, oh, they are related. It's a grandfather and granddaughter, and he's forgotten the language, and she's trying to learn it and revitalize it. And we thought, wow, this is turning out to be an intergenerational story. And when you zoom in and in and in into a phenomenon, this is probably a great place and a great way to show what's happening at a macro level. We started working with Lil Bahadur and Hema while we were on the ground. And we spoke to Hema and while we were conceptualizing, brainstorming, doing some workshopping, having conversations, Hema said, hey, I want to sing a song in Kusunda. And we said, of course. She said she wants to write a song for her grandfather. And we said, oh, sure. How can we help you? How long do you need? And she said, oh, just give me two days. And we were like, wow, that's fast. And we didn't interfere in that process at all. And we thought, OK, it's going to be a song about the future of the language. And she sees the Kusunda language and her efforts at revitalizing it. And Lil Bahadur hadn't heard the song before. And we wanted this moment to take shape in front of the camera and for it to be a surprise for Lil Bahadur as well. And when we realized she, you know, she ended up writing a song for her grandfather, but in his words, because he didn't have the words anymore, you know, which is something that touched me. No, it was a moment where I almost had tears where I thought, wow, this is, you know, I really have hope and faith in the next generation. So thoughtful of her to give this gift to Lil Bahadur and, you know, in some ways substitute his forgetting and his loss of the language with her words and her music and her song. And then we went back and during the post-production, we were, you know, reviewing all the footage that we had from 2012 through all the trips that we had made. And then found this bit from Ganymede where she says, oh, we don't have a song recorded in our language, but, you know, Hema, she's going to save the language. And she says, she's going to sing the song recorded in the Kusumda language. And we were like, wow. That's when we thought when we were reviewing the footage, we thought, okay, Yeah, this was prophetic. And, you know, how we wish Yannie Mae was around to see what she had envisioned. But yeah, maybe, maybe she does see it from somewhere up there, you know?
[00:39:15.720] Kent Bye: Yeah, yeah. It's such a, it's such a powerful moment. And overall, I mean, just the whole arc that you go on the film, it just feels, So complete in that way. And I don't know, Felix, do you have any reflections on that? Cause it's such a emotional climax. It just, I guess it really hit me in a way that was. Surprising in a way of, you know, just the, I guess it shouldn't be surprising, but just the, the power of story to be able to really tap into the emotional flow of a, of a story and how it culminates in this moment. I'm just curious to hear from your perspective, you know, how all that unfolded and all came together in the end.
[00:39:51.532] Felix Gaedtke: Yeah, I mean, Hema was a singer she made clear early on and that was already something we liked and that she wanted to do a song. I really liked this idea and I was anyway always so amazed by her being like a teenage girl that instead of learning English or language that would professionally maybe help her to get a better job or something like this, she like is really enthusiastic about learning Kusunda. which also led to some kind of backlash in her surroundings, you know, where people are saying, why are you wasting your time with that? Why don't you do something that brings you more later? So I already admired her a lot. And when we then found this footage, I mean, it's often this kind of footage, you have gone through it many times, but you kind of overlooked something. When we realized that we had that, for me, it was a big surprise. And I think we were very happy to see that it would also bring Gyanimea back in because she was really the one that sent us on this path. And yeah, it really made it feel complete for us.
[00:40:55.383] Gayatri Parameswaran: And I just want to say, I was back home in India visiting my parents and I could make a trip before the second wave hit India and Nepal. I could make a trip to Nepal. There was a short window. And I was able to do screenings in the community of the film and the virtual reality piece. And, you know, Hema was so excited. And she said, Hey, I have a new song now. And I was just like, wow, this is going to be an album. So yeah, really, really happy to have helped her. You know, one of our dreams is to be a singer. And if she can sing in the Kusumda language, that will put the Kusumda language on the map of Nepal and also inspire other younger indigenous artists, creators, activists to move ahead in their desired intended direction. So I think she's going to be in some ways inspiring a lot of people to follow her.
[00:41:53.046] Kent Bye: Yeah, well, it's such a beautiful song, like not even knowing what the words are and what they mean. I mean, she explains it later, but as you're listening to it, it just, I just felt like it kind of washed over me. And it feels like, you know, like you're saying, there's this potential future power of that moment that you're capturing, but also how that could continue to ripple through the culture there and bring about larger change when it comes to the relationship to some of these languages that go to sleep like Kasunda. But just to start to wrap up here, I'm curious for each of you, what do you think the ultimate potential of virtual reality and immersive storytelling might be? And what am I able to enable?
[00:42:32.097] Gayatri Parameswaran: Having worked on this piece now, I think that has been a question of, you know, having, so we showed the piece to Ganymede as family. And as I mentioned, there was the video version of the piece as well as the virtual reality version of the piece and we screened both in the community and Ganymede's granddaughter was doing the piece as well. And she was very moved by the video because we screened the video prior to the VR piece. Because we had everything, as I told you, the process involved creating a linear version and that's the piece that's going to be used by the community for their advocacy efforts and for their work within the context of language revitalization and using storytelling for language revitalization. So we screened that piece and then she got into the headset and she saw her grandmother and she got out of the headset and started crying. She was very touched by the video, but she just couldn't control her emotions after the virtual reality piece. So that once again, underlined the potential of virtual reality as a medium that is very powerful. And I started wondering, what does it do? Like, what if we were using virtual reality for collecting memories as an archival place, not just for culture, not just for culture, not just for language as this, you know, as you described it, creating the spatial anthropology for this community, but also for personal memories. It's very, very powerful. I don't have the answers to this, but I have just been wondering what it might do, right? If you could see your loved ones, you know, who have passed away in this space, spatially, volumetrically, what it does to memories. And there's something there that we are still figuring out. And I know there have been some experiments and there have been some leads research wise in that direction, but I'm really, really interested in that aspect and, and wonder what that potential is of that potential is of virtual reality.
[00:44:41.923] Felix Gaedtke: Yeah. I actually really liked this spatial anthropology term that you phrased, Ken. I'm going to use that from now on. So for me, really what I said in the beginning, the strongest thing VR can do for the work that I'm doing is really enable people to exchange perspectives and access stories, places, or people and minds that they otherwise would not be able to. That's something I find very, very exciting and interesting. And also this, um, I don't really like the term conservation, but like to kind of preserve like little snippets of the world and make them accessible to others also over time is something that I find very exciting.
[00:45:29.632] Kent Bye: Hmm. Great. Well, is there anything else that's left and said that you'd like to say to the immersive community?
[00:45:38.959] Gayatri Parameswaran: Maybe just one thing probably is that, you know, we're entering, we're in a time and space where it's very crucial for us to be vigilant of what's happening in this space and to know, you know, a lot of the work that you've been doing, I've actually been following and been concerned about how the technology is developing, what are the consequences for a future society? And is that really the direction that you want to go in? Yeah, I think there should be more open discussions and spaces to be able to have those discussions and raise those valid doubts and questions that we have. So, yeah.
[00:46:19.938] Felix Gaedtke: Yeah, maybe one general thing about Kusunda. It might be obvious to some, but not to others. I think this has been really an amazing teamwork and wouldn't have been possible without the help and support of so many different people. So we might be the two giving you the interview today, but it would as well be a lot of other people who deserve that attention.
[00:46:55.567] Kent Bye: It really just shows the power of virtual reality in a way to not only capture a moment in time, but also, like I said, sort of do this spatial anthropology of this culture that is fading away. And like you said, going to sleep is maybe a good way to phrase it and going dormant because we have no idea how things will continue to unfold in the future. And so, yeah, I think it's such a powerful piece and a testament to what the medium of VR can do. Anyway, thanks for joining me here on the podcast and telling more of the story and your, of your journey on this project and just sharing some of your insights and process as you do it all. So thank you so much for joining me today on the podcast.
[00:47:32.441] Gayatri Parameswaran: Thank you. Thanks a lot for having us and thanks for the great work that you do.
[00:47:37.482] Kent Bye: So that was Gayatri Parameshwaran, as well as Felix Gittke. They're the co-founders of Now Here Media. So just a couple of quick takeaways from this interview is that I feel like there's certain aspects of the photogrammetry mixed with the tilt brush that gives you this real spatial sense of this story. I mean, they had original plans for this story, which was going to be a lot different had COVID not happened. And had their intended main protagonist not passed away right before they were going to start shooting. But I think what that actually forced them to do was look at other ways to be able to tell the story that explored more abstract concepts like Tilt Brush and to be able to take stories and start to translate them. And I think just overall, the arc of the film was really impactful, because I think they were able to actually capture a really magical aspect of this song that was constructed in the Cosunda language that was kind of prophesized by one of the main protagonists that ended up passing away. So, I don't know, it's just a really emotionally impactful piece and I'm so glad that it was able to win the Grand Jury Prize at Tribeca and that it's available again for folks to see if you want to catch it at the Venice Film Festival. It's going to be there until September 19th. So, definitely try to check it out. And I guess it's also worth mentioning the fact that there's just a lot of voice-activated aspects to this piece where you're able to, in some ways, learn how to speak certain words within the casino language and how that, within itself, required getting away from the machine learning training that may not always be friendly to some of these more indigenous languages. Just this whole concept of bringing back these languages and that they're not fully extinct, that they're just going to sleep or dormant, and that there's a possibility for them to be revitalized. Perhaps it's through pieces like this that capture the deeper story of a language and its culture that it represents that the wider community is able to keep it alive in some fashion. So that's all I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, so I do rely upon donations from people like yourself in order to continue bringing this coverage. So you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
