Last Whispers is a virtual reality experience that visualizes the extinction of languages around the world. Director Lena Herzog says that there are about 7,000 active languages on the planet, but that we’re loosing one to two languages a week due to climate change displacement as well as different factors of economic and cultural colonialism. She says that we’re on pace to live in a world that only has 30 different languages, and so she wanted to try to tackle the problem of “How do you tell the story of silence?”
I had a chance to talk to Herzog at the IDFA DocLab about the process of designing and developing Last Whispers, her collaboration with Emblematic Group, why she felt like this piece needed to have the full immersion of VR, and the need to move beyond binary thinking in being able to preserve cultural heritage through pride of language while also being worldly and engaged with the rest of the world. We also cover some of the more philosophical aspects with Herzog saying that language is our first creative act, that it’s extremely democratic, and how it is able to embed many aspects of culture and understanding.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
Here’s an introduction to her VR film that she gave leading up to Sundance this year:
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So continuing on in my series of looking at some of the storytelling highlights from the IDFA DocLab, today's interview is with Lena Herzog. Lena did a piece that premiered at Sundance this year called Last Whispers. So Last Whispers is about the loss of language. There's about 7,000 languages in the world, but Lena talks about how there's about one a week that are gone extinct. either through cultural colonialism or climate change, the destruction of different communities, or just the economic consolidation of these different languages. And within those languages are embedded lots of different culture and whole ways of being and inheritances for different communities. And so she wanted to address this challenge of how do you do an immersive experience about silence, about the death of these languages. And so her answer to that was to do this experience called Last Whispers, which we talk about here in this interview, and then we can unpack it here a little bit more as well. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Lena happened on Saturday, November 23rd, 2019 at the IDFA Doc Lab in Amsterdam, Netherlands. So with that, let's go ahead and dive right in.
[00:01:28.168] Lena Herzog: My name is Lena Herzog and the only thing I've done so far is Lost Whisper's virtual reality in seven minutes. Immersive, I guess that's what you call it.
[00:01:40.378] Kent Bye: So maybe you can give me a bit more context as to your background and your journey into creating this piece.
[00:01:47.372] Lena Herzog: Well, I began as a philologist, philologia, studying language and literature in Russia, in Leningrad University at the time. When I came to the United States in 1990, I studied philosophy and became an artist, a photographer, primarily working in the medium of image. I embarked on the work that's later and now has been called Last Whispers. About 20 years ago I did some projects for a very long time. Conceptual work needs time to really get absorbed. And it went through several iterations, including a photographic one, when I took pictures of the last speakers of extinct and endangered languages. You couldn't take pictures of the speakers of extinct languages. And then I realized that what had to be done was a work in which a person immersed himself, herself, Because the project was about extinction and absence. So how do you eliminate an extinction, the form of which is silence? Which is the form a linguistic extinction takes. People stop speaking a language. Well, you sound what has gone silent, which is an obvious answer. But how do you do that? You do it in such a way that the presence, the reality of what has gone is so apparent that it cannot be denied. So I want from a medium which is essentially in a frame an image. to several steps out of the frame. And the first time I stepped out of it was with sound. So the original work which premiered at the British Museum at the Living and Dying Gallery in 2016 was a work in flat image and an immersive sound. An 8.1 sound which made people neurologically register. that the voices are present. Our brain registers 8.1 octophonic sound and sometimes binaural sound as present, which is a very interesting understanding. And then I remember people saying in the Living and Dying Gallery, oh my God, I want all of that forest and all of this imagery of outer space to be all over the walls. They want it to be immersed, in other words. So the next logical step of immersion was stepping into the entire reality, not only sound, but image as well. And so I did the 7-minute virtual reality with Emblematic Group and Noni De La Pagna and her engineer Jonathan Yamauza.
[00:05:04.432] Kent Bye: Yeah, I had a chance to see Last Whispers at Sundance in 2019 and it was very striking, very evocative because you're talking about the death of a language and death of a culture and you show this point cloud representation of the world and geolocate where the languages were spoken and then you have this rotating globe and so you're kind of situated in outer space looking onto the earth but as it rotates you're able to locate where that person is speaking by animating this dot that is able to locate me in a time and space because time either at some point whenever the recording was made but in space where it was located and I thought that it was both very beautiful but also very sad. that there's this loss of these languages and trying to do this preservation. And it looked like you were collaborating with quite a lot of different organizations that were already trying to preserve the languages either through oral recordings or written just a number of different ways and you were trying to create a digital archive and experience of that. And so, for me, adding space to this issue really grounded it for me in a very specific way. Where did you interest start with this distinct languages? What was the catalyst for what was striking for you about this phenomenon? Why you wanted to try to preserve it in this way?
[00:06:28.733] Lena Herzog: Well, I think in a way that's a long story. Personally, I was always a bit language crazy. I also find that language is our first creative act. It's highly complex, highly abstract. How do you go from sounds to justice, love, blow? It's an extraordinary leap that we all do, and it's a highly democratic act. We all do language. And every language is a world. It's a highly complex web of understanding and inheritance. Because, of course, language is passed on to us. And what is embedded in every language are cultures. It's not entire culture, but there's an extremely important connection between language and culture. And the idea of a mass extinction, and we're talking an extinction of languages that is more rapid than the extinction of species. We're losing a language every two weeks, some experts say every week. So out of 7,000 languages, we're going to rapidly end up with 30. And the scale, the sheer scale of this extinction is mind-boggling. And also the fact that so very few people know about it. But of course, our ignorance of this mass extinction, the mass ignorance of this mass extinction is intrinsic to the form of this extinction, which is silence again. That's how it takes place. And so that was a huge challenge and a foundational challenge. How do you show silence? And also, how do you locate it? So when we, for example, located and created the geography, we combined the true geography, the dots that you see illuminating language locations are GPS correct to where they were collected on the outlines of the continents. But there's also a combination with imaginative geography, because we didn't want to do a Google map. So what I did, I subtracted the Google map out of the outlines of the continents and countries, and instead infused it with the imaginary map. except it's not quite imaginary. It's a map of the combination of catastrophic climate events, hurricanes, for example, floods that are taken by satellites of our Earth, because one of the main components of cultural and linguistic extinction is climate change. These are marginal communities that very quickly get evicted, not only by power, which is primary force that evicts them from their culture and from their language. It also is climate changes. So Brazilian forests that are burning up now and are expelling tribes. we will see these tribes that have left their habitat speaking Portuguese in 20-30 years. That's just one of the many, many examples. So the geography that you see inside the glowing marble, that's the marble of catastrophic climate events. And what I created in VR together with Jonathan Yamausa, an emblematic team, was what I called an empathetic vector. So I wanted us to feel identified and absorbed into this world of other voices, of others, to really feel the human chorus. So when you, in the beginning, see the globe, that you kind of recognize that's our planet, even though it's slightly strange, as I explained. You see it away from you, north of you. It comes closer, closer, in front of you, around you, you. And so this slow reveal of seven minutes from the others, the chorus of others, being estranged from you and away from you, to coming closer and closer to being around you, and the last what you hear is an exhale to being hopefully you, an identification. I have a feeling it's kind of, perhaps, if it works, I hope it does, maybe a helpful exercise for us to feel empathy towards others. To at least be able to listen and to hear these voices.
[00:11:57.698] Kent Bye: Yeah, I definitely felt it when I saw it. And just like hearing you share your process and your stories, it just like feel this tightness in my chest, just the sadness of what I see happening with this tension between the centralization and decentralization in the world where I'm at the American Philosophical Association Eastern meeting this year, I went and was talking to different philosophers and decolonization is a big trend within philosophy and the broader culture right now of finding how you can re-indigenize different cultures and to see the colonial impulse that we've had to create this monoculture, which has its economies of scale and advantages, especially with the internet that we've had. But we see the centralization of power with authoritarian governments, with the internet being able to be switched off in Iran that just happened last week, with the consolidation of wealth and power with a small handful of companies. And it feels like there's this need to have this decentralization and decolonization, which is to go back to those local communities, back to those local cultures, back to those languages, but yet all of the impulses of capital and our economic systems are pushing towards this using English as a global language. people who want to share in the culture of the entertainment will have to speak English to be able to get access to that. And also with all the different dynamics in China and even within the consolidation of power there, then people wanting to then censor themselves to be able to have the lowest common denominator that's going to serve the largest market, which I feel like your project is showing that we have this loss of language. And what does that mean? To me, it shows that it's the loss of that culture, but also this need to start to figure out what's it mean to actually put the power back in the hands of these local indigenous communities and to put oral histories at the forefront, even like Wikipedia puts the neutral point of view. So you have the New York Times or an anthropologist who walks into an indigenous community with a PhD and he writes a book. that is a more authoritative source than the stories from that community within itself. So what does the Wikipedia of oral history look like to see how the people's own stories of their own oral histories becomes the most authoritative? So I feel like you're tapping into these larger dynamics by talking about language. But yeah, I'm just curious to hear some of your thoughts about some of that.
[00:14:22.770] Lena Herzog: We should, I think, try and step away from binary thinking. It's not either-or proposition. And what happened with technology and the way that power proliferates, is that it is doing this at such a rapid pace and it has so captured the definition of what it is and why and stifled any real thinking about what it means for us that I think we're perhaps only now trying to understand how the forces that seem to be anonymous and obscure can be illuminated. But that's another subject. In terms of what it means to be rooted, inherited, and yet worldly at the same time, it shouldn't really be either or preposition. So one of the optimistic examples that I keep citing, because of course that comes up all the time, what's to be done, right? Is Gaelic. So after the bloodshed, euphemistically called the Troubles, the Irish people, after 800 years, essentially declared and made sure that they can study, speak, and flourish in their own Gaelic language. The Good Friday Accord settlement provided For that, Gaelic language and culture became one of the very first points of the settlement of the Irish Troubles. What did it mean? Policy and wealth was dedicated to this. Power allowed it and dedicated its real tools. not imaginary, not statements, but actual tools, which is policy and financing. People themselves, on their side, really wanted it. So although the prestige of English as a language of power remained, the pride of being Irish and knowing Gaelic also remained. These two directions met. And now Irish people are both inherited in their own culture and worldly at the same time. And no matter what worries the Irish have about their situation, economic, political, otherwise, it is an enormous strength. And comparatively to other communities where language was literally beaten out of them by power. The stories that we have, for example, from Aboriginal people from Australia is where children were kidnapped and literally beaten severely if they were caught speaking Walpiri. Or I just spoke to recently a Cheyenne tribe from Oklahoma. The man whose father studied in an American school, if he was heard speaking Cheyenne, his head was bashed against the wall. That just happened 40 years ago. That's really not long ago. What does it mean to have that inheritance of shame of speaking your own language and then now shame of not speaking your own language? And I think we really have to think about is who is it that we are? living in this world where boundaries are permeable. No matter what nationalist movements around the world are doing, the reality of it, the dynamic, is really on the side of the more permeable world. But do we really want to end up in a monochromatic, monolingual, homogeneous world where we are a medium average of dominant culture? Let me tell you what will happen. We will globally become utterly impoverished and provincial. Because what is the definition of being provincial? It's just not knowing anything outside your own self. and being fairly reduced to your spectrum, your cultural spectrum. Particularly, that's the case of dominant cultures. So, for example, some of the tribes in Africa, like in Gabon, speak nine languages, because there are 50 languages in a country of one million people. So this is extraordinary. Papua New Guinea, 800 languages, very tiny territory. Many of the people speak three, four languages that actually have nothing to do with each other. So it's only in dominant cultures where we are speaking, you know, one language or maximum two. So this always comes to surprise, especially to a lot of Americans, Brits or French, you know, that that's actually, you know, not the norm. And by the way, the neurological science is in. It is good to be multilingual. It's really good for us. Another interesting thing, Last Whispers was just showing in Montclair, New Jersey at the Alexander Kasser Theatre as part of Peak Performances. A woman came who was running a non-profit for the Lenape, Rimapao Lenape people, giving them possibilities to study language and revitalize certain cultural traditions. Obviously, they all speak English. All of these people. Now the average age in the United States for Native American men is 56. That's the longevity of an average individual Native American. That's truly shocking. The suicide rate is staggering. So this woman told me, she said that she can't really claim a really long history with her non-profit. But the results they have were extraordinarily promising. And we also have it from other activists and political solutions like that, that it turns out it's important for us to be inherited. To know where we come from. But of course, the current world also demands from us that we are also worldly at the same time.
[00:21:36.672] Kent Bye: Well, because you're working in the medium of virtual reality and you had written a manifesto starting to think about the medium a little bit in contrast to other mediums. When I think about language and the philosophy of language, I think of like Wittgenstein and other approaches of trying to think about like language and what it means. but do you see like film and virtual reality and more aesthetics or something that's different in language or do you see that as a part of like a new language like a visual storytelling language or a spatial storytelling language that you would consider as the same kind of thing as other languages?
[00:22:12.810] Lena Herzog: Well, we can be here for a few thousand years thinking what is language and And one of the things about Wittgenstein that I found extremely dizzying is that in the beginning he wrote an extremely convincing case regarding language and thought. And then later he debunked it and wrote an opposite take, just as convincing. about language and so on. This same man. And of course, the difficulty, philosophical and an epistemological one, is that we have a hard time thinking outside of language. Because that's a tool we think with. So it's very hard to step outside of ourselves. It's hard to step out the tool with which we think, by definition. It doesn't mean we shouldn't.
[00:23:10.842] Kent Bye: So do you consider a VR piece, like the language of storytelling within film or VR, as a form of language?
[00:23:18.569] Lena Herzog: I don't know if storytelling is necessarily what I would say VR is good for. And that's actually something that remains a question mark. And the reason why I have this undecided stand is that the question of agency of the visitor is so complicated and unexplored. And so when we tell a story, there's always a distance between us and the story. When we read it in a book, when we watch it on the screen in a frame, but when you lose a frame and you introduce the visitor inside that frame, what happens? And all exercises, for example, in film, like Lady and the Lake, where everything happens addressed to the viewer, as if the viewer is the detective Philip Marlowe, didn't really take off, didn't really work. There was a fashion in literature where several endings were offered, and the reader had to pick an ending. But that didn't take off either. The question about virtual reality and agency in virtual reality has to do with free will. And in virtual reality, it also has to do with perceptions and how our brain works. And we still don't know so much about it. We don't know really anything about it. We're only in the beginning. We hardly can explain why we're lifting our arm. And here we are throwing ourselves into virtual reality, creating world from scratch. Gravity. What is our perception of gravity? And massive also ethical questions. Because how are we designing the world where this visitor is making decisions or not making decisions? What are his chances in that world? What are her chances to make decisions? Well, we have clearly pre-designed this, right? And all kinds of also dark possibilities linger right there. Because power always had to do with depriving people of agency and limiting options. Because that's how power exercises power. And it is not a coincidence why all sorts of agencies that call themselves defense agencies, which is, of course, a linguistic trick. Who do they defend from, really? Why they're so interested in VR? All these VR conferences are constantly permeated by them. Why? What is it their interest? Of course, the interest has to do with understanding how to exercise agency, in other words, power. So where do we stand with that? What are our ethics in working with that? That's a very interesting question. And I think here we have to really set ourselves the task. What is our human task? And I would put it to you, humanism. and know what's going on. Try to really be critical, including ourselves. Really ask tough questions, but that shouldn't really stop us. Because the real key is what is our human task, particularly with technologies that can be so vivid, so convincing that anything terrible can be done and people can be convinced of the worst things about themselves and the world that can be habituated into the worst in themselves but of course also could be the best of us. The horizon is really open and really wide and this medium is, as somebody put it, in diapers. And I think we really must make choices that are responsible. And I actually sometimes really worry because a lot of people who are in VR come from gaming, where reality doesn't have any consequences. And truth or consequences seems just like a town somewhere and not a real thing. Well, it is a very real thing. And we'll pay dearly if we don't think about this. carefully and seriously.
[00:28:40.983] Kent Bye: So for you, what are the either biggest open questions you're trying to answer or biggest open problems you're trying to solve with your work?
[00:28:50.070] Lena Herzog: With the work in VR?
[00:28:51.972] Kent Bye: Yeah, the work that either with Last Whispers or the continuation of what comes after that.
[00:28:57.686] Lena Herzog: Well, that's too broad a question to really answer without looking foolish, if I haven't done that myself already. But, well, I would just really be interested in that exploration of where our humanism can be found. and try to see what is the best that we can do with the tools that we have. And not necessarily, maybe, most commercial. It's not necessarily that something that's most money-making or most entertaining, but something perhaps insightful. And work, beautiful work has been done. like Notes on Blindness, which is sort of the King Lear of VR, where you come out with a sense of extraordinary wonder and beauty from a story which is truly tragic and yet optimistic at the same time because you really sense the human potential. That would be good. I think that would be really good.
[00:30:17.557] Kent Bye: Great. And finally, what do you think the ultimate potential of these immersive technologies might be and what they might be able to enable?
[00:30:30.722] Lena Herzog: I think everything from very practical applications like in medicine, I understand immersive is being used for teaching, to extraordinary discoveries, to things that are, I'm hoping, insightful and revealing and fascinating. and also truly unique to virtual reality, that can do things that no other medium can. And what it is, is really key. What is the beating heart of technology? And you have to know for that, what is the beating heart of your idea and your project, that it demands a virtual reality. Could it rather be an essay written or a documentary movie in a frame? Why did it need to be VR? So that would be, for us, I think, a very important question to answer.
[00:31:38.543] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the members of the community?
[00:31:44.190] Lena Herzog: Be responsible, but go discover.
[00:31:47.551] Kent Bye: Great. Well, I just wanted to thank you for the work that you're doing with Last Whispers and for taking the time to talk to me about this very important topic. So yeah, just thank you for joining me today on the podcast. So thank you.
[00:31:59.015] Lena Herzog: Thank you.
[00:32:00.035] Kent Bye: So that was Lena Herzog. She's got a background in philology and studied language and philosophy, worked as a photographer, and she was talking about her first VR experience called Last Whispers. So I have a number of different takeaways about this interview is that, first of all, Well, in this piece, you know, it's this challenge of how do you show silence. And so the approach was to take this point cloud representation of the earth and trying to have different levels of abstraction within that representation that would show different elements of climate change and displacement. But this earth is kind of rotating around and that globe is moving towards you as you're listening to these different people speak these languages that are either in the process of being extinct or already extinct. It's kind of like the last people speaking those languages. They're geotagged, geolocated, so you're seeing this dot that is pulsing as they're speaking this language, and then they stop. And then you see this contrast between identifying these geographic regions and then seeing the extinction of these languages. that's actually a really powerful way to do that and it's got binaural audio track as well and you're kind of immersed within this abstractions and exploring the depth of language and it's kind of like a meditation and so just in the process of talking to Lena she talks about the importance of language of how it's like our first creative act it's very highly creative we can be able to say words like love and justice and blue and also very democratic. The language within itself, it has embedded within it all these different cultural meanings and it's our inheritances as a community and a collective. And so she says it's this tragedy that we have over 7,000 languages, but that about one or two a week, we're losing one of those languages. And that soon we'll only have 30 languages if that continues at this pace. So there's this death and this silence. And so how do you bring awareness to it? So there's another level of silence that she's trying to make aware of with this piece. And how do you locate it? How do you spatialize it? And I think the approach that she did actually, I think, is really effective of trying to actually give a larger context for where these languages are and through the act of showing them and having them disappear, showing that they're giving this symbolic death. And also in talking to Lena, she was talking about being careful about creating these false binaries between, you know, just thinking about these powers of the economic consolidation, cultural colonialism, the different ways that these languages that are exerting different levels of power and status, like the English as the language that I'm speaking right now to you, and as you're listening to this, and that, you know, as I travel around, there's a huge amount of privilege for me to be able to go to all these different places and for me to to be monolingual in the sense that English is my primary language and that I haven't been forced to be able to have to speak multiple languages but for most people in the world there are multiple languages that they have to speak and you know for some people they are in the regions where they have like nine languages that they speak because there's 50 to 100 languages in some of these very small regions. But she's saying that don't make this mistake of a false binary because there is this dynamic between trying to preserve your own cohesiveness of your culture and to have this pride that is like in Ireland with Gaelic trying to go to the opposite of being shamed and to transform that shame into pride and being able to have that part of your heritage and your culture, but at the same time being worldly. So you don't want to completely isolate yourself from the rest of the world and only have your small cultural context. They actually want to be able to cross-pollinate ideas and be able to connect to these larger things that are happening in the world. So there's this dual expectation to both be worldly, but at the same time, figure out how you can preserve different elements of those cultural contexts. And as we move forward into the metaverse and start to have more and more people have these exchanges of ideas and cultures, then you think about the languages that are being spoken and what kind of culture that's then going to be able to be embedded within that. and to see if there's going to be new types of languages that are forming, whether it's visual languages, memetic languages with memes, or the visual storytelling languages or different cultural symbols. They're going to be more universal ways of language that's going to be formed. And that's, for me, one of the more interesting philosophical aspects of the future of spatial computing is that it's going to be able to be formed into a whole robust language that we're going to be able to speak while we're in these immersive environments. We're already speaking the abstractions of our existing languages, but is there going to be a spatial language that's going to be formed and is it going to be serving a similar function to other types of languages that are maybe like poetry or with very high levels of metaphor or with film with all the different conceits around film. Lina said, you know, language is a very complicated topic. You could take thousands of years to be able to look at it. But it's also at the heart of so many different philosophical issues, especially because there's so much the way that our thinking that is getting translated into language. And so just having multilingual, multiple languages that you're speaking gives you many different channels to be able to explore different concepts and ideas. So. Lots to think about here and it was nice to be able to talk to Lina and she gave actually a presentation at IDFA DocLab talking about an update of the manifesto that she wrote with her husband Werner Herzog, you know, exploring what she sees as the affordances of virtual reality and some of the challenges that the medium has yet still to solve. And it gets into a lot of the same things around agency and narrative and having the interactivity within the process of telling a narrative and is narrative the best use case for what types of things that you can use the virtual reality medium for. So that's all that I have for today. And I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast. And so I do rely upon listeners like yourself in order to continue to bring you this coverage. So just $5 to $10 a month is a great amount to give and just allows me to continue to capture this real-time oral history of the evolution of the virtual reality medium. So, you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
