Once a Glacier is a 360 video that visualizes the before and after a glacier melts over the course of a single human life time. Director Jiabao then expanded this piece as a part of a live motion capture dance performance that showed at The ONX + DocLab MoCap Stage at IDFA 2022. I interviewed her at IDFA to talk about the evolution of this project, working with indigenous poet Joan Naviyuk Kane, recording sounds of a melting glacier, and how she wanted to juxtapose geological time with the time of a human lifespan to bring more ecological awareness to the changing world around us.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. And you can support the podcast at patreon.com slash Voices of VR. On today's episode, we're continuing my series of looking at different pieces that were at Infodoc Lab Now, at IWA Dock Lab, there was a whole Onyx Dock Lab motion capture stage experiment, where Onyx Studio came in with high-end motion capture technologies, and there was a number of different projects that were using these in different ways. And so, one of them was called Once a Glacier, which originally was a 360 video, which was telling the story of how melting glaciers are happening within the course of a lifetime. So, it shows a young girl who's kayaking into this big glacier, and over time, it melts down. And there's other dimensions to that story as well, as she tries to save and preserve the glaciers as they're melting away. But it's contrasting human time with geological time and talking about these larger issues of climate change. So they took this 360 video piece and did a translation. So they had one of the characters embody the glacier. And as she was dancing around, then there was reading of poetry from an indigenous poet. and just trying to experiment with how do you do the live dance performance on a motion capture stage that has projected in the background this virtual representation of what it's symbolically representing on top of having poetry to have another layer to the story. So I have a chance to talk to the creator Jabao Li about her process of creating this and where she plans on taking it next. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Jabal happened on Saturday, September 12th, 2022 at IFA DocLab in Amsterdam, Netherlands. So with that, let's go ahead and dive right in.
[00:01:58.442] Jiabao Li: I'm Jiabao Li. I'm an artist, designer, and creative technologist. I'm also an assistant professor at UT Austin. What I'm doing in VR space is I made a virtual reality film called Once a Glacier. It tells the story of a girl and a piece of glacier ice. And as the girl grows old, the existence of the ice is threatened.
[00:02:19.761] Kent Bye: And yeah, maybe you could give a bit more context as to your background and your journey into making virtual reality pieces.
[00:02:25.345] Jiabao Li: Yeah, I've been working in the VR space for a long time, since grad school, back to 2015, 2016. And also, professionally, I was at a tech company before, and working on future headsets. For the Glacier piece, I lived in Alaska for a year and worked on a series of Glacier work ranging from Glacier's Lament, where I turned the Glacier melting data in the past 60 years into music and dance and performed on the Glacier together with the Glacier's own sound. to reflections on the glacier, putting a mirror reflecting as if they were saying, this was me or this is my future. And then it progressed into a VR film, which is a story inspired by my own experience where during one of my expeditions on the glacier about kayaking, I collected this piece of ice and bring it back to home and try to grow it in the freezer myself, day by day, and I turned that into a VR film.
[00:03:25.649] Kent Bye: Okay, yeah, maybe you could talk a bit about how this specific piece, Once a Glacier, came about, and what was the starting point for you on this piece?
[00:03:32.934] Jiabao Li: Yeah, so I just talked about how I started the story, and I started by writing a script, and in UPI Poet, Joanne Kane, turned it into a beautiful poem, and I started to assemble a team, a great team, producer is Jenny and we start to make it as a 360 film and then we get to know that we can use the mocap stage with Onyx Studio then we adapt it into a live performance motion capture performance and we're also developing an interactive pcvr version where it will be a multi-sensor you can feel the cold and hot of the different temperature or when you hold the ice and the girl will at least help to try to water the ice in the freezer and you will smell the scent of glacier, swamp, freezer, museum and so on.
[00:04:27.817] Kent Bye: So let's start with the 360 video because it sounds like this piece has had an evolution as to what you're showing here at IDFA but you started with the 360 video so maybe start with the story there of what is essentially the same woman over time. So you have her as a teenager that's kayaking into the glacier and then you show her later and then through that time scale of a human you're able to see over the course of her lifetime how much of the glaciers have receded and I guess that's a way of showing the geological time that has happened, but it's very accelerated in trying to tie that into more of a personal narrative from one person's perspective of how that changed. So maybe talk about the development of that story and why you thought the immersive virtual reality aspects of that were particularly compelling to tell that story rather than other media.
[00:05:16.140] Jiabao Li: Yeah, so usually we think of human life as like a hundred and a hundred, and then nature's life, geological, deep time, it's so much more, millions of years. But glaciers are disappearing in our lifetime. Some of our kids won't see it. They're melting in an exponential rate. So I'm thinking, how can I juxtapose the human life with the nature's lifetime so that we can know this is happening so fast and it's not something that we should consider only far away. And so the story starts with a girl kayaking into the glacier. And there's this long, beautiful kayaking through and you can hear all the glacier sound that we recorded on the Matanuska Glacier in Alaska. And a piece of ice fall down in front of her. She pick it up, bring it back to her home. She kisses the ice and hugs the ice, but every time she does that, the ice melts a bit. And then inspired by actually Hindu farmers who bring a piece of female glacier and a piece of male glacier, trying to marry them and grow more glacier for irrigation, she started to grow the ice. So in the summer, she put it in the freezer and water it every day and let the cold grow it. And then in the winter, she put it in her backyard and let the snow and rain grow it. And then year after year, the glacier try to maintain its original size. But then she grows old. She kind of take on this all of her lifetime to do this futile, seemingly futile practice to try to save this what was once a glacier. So she tried to bring the ice back to its original glacier, but only to find out that all the glaciers has disappeared because of climate change. And out of sadness and guilty, she passed away on the glacier. And then later people find this piece of ice and the newspaper exploded. New York Times said, this is the last glacier in the world. Fox News saying, the glacier hotter. There's an old lady as a hotter of like, we found there's still one last glacier exists. But who knows, like after all this year of growing in the freezer, do you still call it a glacier? Or it's just like, we are just trying to self-condolence for that. and the bazillionaire about the glacier put it in their family name and named it after them and the last scene you see this piece of glacier ice sitting in the glass box in the white walls of museums just like those animals that went on extinction that we put them in the box and just like that and people swirling around and also like the speed of how the glacier see the world changed. Now people are becoming like fast forward and then the camera zoom in and you become that glacier and you see this cold world that doesn't really care about anything and people are still pretty busy on their phone walking around in the museum.
[00:08:03.470] Kent Bye: Yeah, I thought that being in an immersive environment and actually going into the glacier and then later in that same piece having a perspective of where the glacier used to be, now it's all melted away and you see it from a distance, but then you have an animated drawing as to what the structure of that was back when she was younger and able to actually go into that glacier. So I thought that having that immersive experience inside of the glacier and then later it being taken away I thought was effective. I think the other thing about this piece is that you have a poetic narration in that you have these motion graphic texts that are coming up and highlighting individual words but also having this swirly line that's going through and so maybe you could talk about that process of writing this piece but also trying to translate it into more of a poetic take that then is being narrated and that's what you're hearing and so you're hearing this poetry as you're in this immersive space immersed around with all the glacier ice surrounding you. So maybe you could talk about the process of developing that.
[00:09:03.747] Jiabao Li: Yeah, so in traditional Inupiaq stories, which is an indigenous tribe in Alaska, glaciers carry their memories through their songs. And as glaciers are disappearing, these song memories are disappearing. And so that really touched me and Why spend a long time talk about that's like a story, but then I want to tell it through the lens of the memory So that's why it's like the poem feels like fragmented and then the audience piece them together It's not like everything just present and tell right in front of you, but it's like piece by piece And yeah, there's a bit of a having to piece it together as you're watching it, right? Yeah, and same as in the motion capture. I heard there was a girl on the back pacing around, like last night, and then for this last piece, you know, it's a glacier. She finally paid attention and said, is that a glacier? Is that lady embodying a glacier? And yeah.
[00:09:57.020] Kent Bye: So you have the 360 video piece that you had produced but you also have the motion capture version. At what point did you hear about the motion capture as a potential option and to go down this other path which the piece, the 360 video piece, seems like it's fairly self-contained as to tell that story in that way and then what opened up into having this technology and then try to interpret it in a new way. So maybe talk about that process of becoming aware of the motion capture technology, this stage that's here with Onyx Studios here at IFA DocLab, and then that process of trying to translate your existing piece into a motion capture piece.
[00:10:34.423] Jiabao Li: Yeah, so it's about, I think, a month or six weeks ago that Onyx tell us that, oh, we can further develop it into a mocap piece. The time is really tight, but we have a great team to put this together. And I always want to tell more of the glacier from the glacier's perspective. That's why we added the first part, the solo dance, and the glacier is telling from their perspective about how it experienced the world from Ice Age all the way to the Anthropocene. So in the first piece of dance, in the poem, you can read that, like, I'm the glacier and start from how was the world like during the Ice Age and how it progressed and like how they grabbed the mammals and then the human take the chisel and dance on their face. And then we also have a one minute long AI generated video that's talking about, like we did all the research about actually the glaciers that now live in Alaska, they were here when human arrived. So they're not even that long ago. And you can see how humanity progressed through how the glacier had lived through the world. So what if we tell it, not from the human's perspective, but I was like feeling, okay, I'm the glacier, what do I see now? So we go through all that history and draw the timeline and say, like, what do we put in these visuals there?
[00:11:57.323] Kent Bye: And did you use the same poet for this version of the poem? And maybe could give a bit more context as to the poet that you work with.
[00:12:03.601] Jiabao Li: Yeah, same poet, Joanne Kane. She's a Inupiaq. She grew up in Alaska as well, as most of our team are. And she also teaches in Harvard and now lives in Cambridge. Yeah.
[00:12:16.685] Kent Bye: And yeah, maybe you could talk about your time in Alaska and what got you inspired to tell some of these stories about the glaciers there.
[00:12:24.310] Jiabao Li: Yeah, I lived there during the pandemic, very fortunately, and I go on these glacier expeditions every week. I've been to many different glaciers. So I talked to a lot of glaciologists as well. Some of them, I was surprised, especially the 3D modeler glaciologists, they don't go to glacier that much. They sit behind computers. And by going through these glaciers, you recognize that each glacier is so different. And they have their own characteristics. I don't want to personalize them. Every word I use to describe them feels personalized. But you see this gorgeous glacier and they are magnificent, but they are also vulnerable at the same time. They are disappearing, they are retreating in a super fast way. And the way that people are describing that they are healthy is they won't melt within their lifetime, which is maybe another, like, 50 years or so, but how do you even define that's healthy? And in different glaciers, like the exit glacier, that they have this sign saying, okay, this is where it was in 1920, where it was in 1980, and in 2000, and you just see it keep progressing, and you keep going, the glacier's still not there, and you see, oh, this sign, where it was before, that's a really sad experience. And when I was on Mendenhall Glacier with my friend who is also a glacier guide and we hiked for five, six hours and we still haven't gone to the glacier. But before, like where we started, that's where the glacier were. And that was a really sad moment. I was crying there on the glacier in the wind. And that's where it inspired me, this whole series of work.
[00:14:09.373] Kent Bye: Yeah, I actually was on a family vacation in Alaska this summer and went to the exit glacier and so saw there was the signs that as you are driving in you have like oh this is a sign where it was in like 1920 and then obviously you're driving on this place and so it has receded to the point where when you get there, it gets further and further receding back for as you compress time. So yeah, this exponential decrease of the glacier, you have this spatial experience of that as you're traveling through space over time, you see how much and how quickly they are going away. And so I feel like this VR piece is able to translate some of that. And that when you have the motion capture, I guess it's a little bit of a more what I see is at least that there's a dance performance that's happening. But then there is also the audio that's happening. But if I was only watching the dance performance, I wouldn't necessarily know what the story is. And so maybe you could talk about that process of trying to take this abstraction of this larger story you're trying to tell about the receding glaciers, and then have the narration that's trying to then augment this dance performance that's trying to embody this abstract entity, and then how to use human motions to try to communicate this larger story. Yeah, just using the motion capture technology, what was the process of trying to craft and direct and have some of the conceptual frameworks for how to use the dance and performance in a way to communicate these more abstract ideas.
[00:15:31.690] Jiabao Li: Yeah, so one way I think it's having the dancer embodying a glacier is that glacier is not static. It's moving, it's very dynamic, it changes every year. A glacier cave that you see today, you may not see it tomorrow or next week. So like really dancing the movement, how it carves through the rock over the years that can come through the dance. And then the mocap is augmentation of that dance movement. There's the particle system that translate the dance movement and go up as the poem progress through the ice age, the earlier time. And then when it goes beyond industrial revolution, then the particles are still going down. And to be honest, I would like to develop further on that particle system to have more effects and more fine-tuned, giving more time. Yeah.
[00:16:25.180] Kent Bye: What was the process of coming up with the more extended poem for this motion capture? Because you said your original process was to write a script or a story, and so what was the script or story that you gave to this indigenous poet to be able to translate it into what we saw today with the performance?
[00:16:40.672] Jiabao Li: Yeah, so this part of the story I write in collaboration with my husband, Cooper. He grew up in Alaska. And so we research a lot of the things that happen in this timeline. And imagine if we are that glacier. If we are that glacier, what would we write about it? Like really get into the truth of the glacier. It's funny to say the truth. And that already have a flow of poem. And then we talk through with Joan, and then that turns into the poem. Yeah. And there are so many interesting things that we learned along the way. For example, I don't know if you have seen the glacier moss. They are like tumble moss. They just tumble and go with the flow of the glacier and they can just absorb the water of wherever they go. And there's also that pink glacier in Alaska, which is the algae that has the pink color. But at the same time, those algae can also contribute to the disappearance of glacier. There's also the black worm that live underneath those ice. They also absorb more sunlight and contribute to their disappearance. And then there's that blood glacier, which because they have a lot of iron, that's why they give that color. And we included all these interesting things we learned about glacier into the poem.
[00:17:54.115] Kent Bye: Yeah, the visuals, you had this moment where you were showing different pictures of the glaciers and then translating those pictures into Pantone colors and then you come up with a color palette. And I know that when I visited there, there's like the color that you see of a glacier is different than what you might see in normal ice or even within the water that you're around. And so maybe talk about trying to map out the color palettes of the glaciers and how you were trying to visually represent that within the context of this performance.
[00:18:20.133] Jiabao Li: Yeah, so the color change every day, every moment, when it's sunny or when it's cloudy, it's all different. And I look up what's the pantone color describing glacier and there's only four, which was what we see is so different. There's so many different color. So I try to collect all the color. Of course, color perception is subjective if you use a different camera that will collect it differently. And again, I think all this ties together of this almost sad and futile action of trying to save this last distinctive characteristic of glacier, this last bit of glacier. like all this work, including the Glacier's Lament and Once a Glacier, where we model this glacier based on the satellite data of the Matanuska Glacier, trying to save the color, the model, and the sound part. If I can talk a bit more about it.
[00:19:11.957] Kent Bye: Yeah, you said that you were recording sound. So what was the process of trying to record what the soundscape of this glacier would be in this immersive experience?
[00:19:20.275] Jiabao Li: So I work with sound artist Matt McCarco and our team, we come to Alaska and we hike on different glaciers and record different parts of it. And there's this chirping sound that caught most of our attention. You thought it's a forest, you thought it's bird chirping, but it's none of those. It's simply just eyes break up and they squish each other when the sunlight coming out. And it's so little you have to really get down embracing the ice and put your ear next to it and you can hear it. So we have a contact microphone, we have a shotgun microphone and we point to it and then we also amplify that sound and we compose that into the film. And for the far part, like in a glacier cave that it's more dangerous to get into, it's almost like throwing the fishing line, we throw it into the cave so we can record a lot of the cave sound. There's also some sound that I'm like acting as the girl and like kiss or caressing or interact with the glacier and we collect those really soft little sound. Yeah.
[00:20:26.016] Kent Bye: Okay and you know when you show this performance here you had the new part that you had in the existing experience where you have the woman who's dancing as the glacier with the poem and then you transition into showing what I saw in the 360 video and you actually have a young actress who's actually saying the same poem that's in the immersive experience but you have a dance part that's sometimes just the dancers and then you have other times where they have motion capture that's engaging in a way so maybe you could talk about that process of recreating different aspects of the immersive experience but translating it into a live dance performance that has just the dancers but also has aspects of motion capture that's also having their actions be projected in a 2D representation that's on a screen as people are watching.
[00:21:11.077] Jiabao Li: Yeah, so for the interaction of the performers, the girl got to perform together with the dancer, which added a lot of dynamic between this relationship. Because in the 360 film, you don't get to see how the eyes come to life. It's the eyes in the film. But in this performance, they have such an intimate reaction. It's interesting because in my original script, there's this progression where the lonely girl see a lot more like a mother figure in the eyes, but then progress into, she become, like when she get older, she become the grandmother figure and taking care of the eyes. But in the film, there's not that part, but then it come back in the motion capture when the dancer is carrying the girl or later become the girl carrying the dancer. Those part come back. And then there's a part where the viewers see the POV of the girl when she says, I'm not alone. And what are the ice in the freezer? I'm not alone. And then see herself taking care of the ice in her backyard. Those repetitive parts, we really want people to get into the perspective of the girl and see how she's spending that whole lifetime taking this one piece of ice and see herself growing up throughout this stage. Yeah.
[00:22:28.312] Kent Bye: So in the 360 video there's the young girl and then there's later in the piece she's an older woman who has been cultivating this glacier that she's captured within the context of her refrigerator. But when you do the dance performance, was it meant that the young girl was interacting with the ice or that she was interacting with her older self? Because that was a little unclear as to whether or not she was the ice or whether or not she... Because when I first saw it, I thought she was kind of representing that older woman, but it sounds like that she was actually the young girl's interacting with the ice. Is that correct?
[00:23:00.898] Jiabao Li: she is progressing through her age during that repetitive I'm not a long part and she's interacting with us like you can see that water can motion capture tracked she's watering the dancer which is dice yeah and then when she see this way then that's the time that she see herself okay growing through all this age and become the grandma version and then the grandma's voice come in and their voice kind of intertwine each other and she set out of the stage and you become the grandma's lingering voice intertwined together with the ice.
[00:23:36.709] Kent Bye: Okay. Yeah, I think there's certain aspects where there's a lot of symbolic translation that it's not always clear to me when I was watching it like who was representing what in these different interactions and so yeah, I guess that's some of my feedback.
[00:23:47.845] Jiabao Li: Yeah, and I guess it's also open up to the audience interpretation. It doesn't have to be the way that we think you want to do it. Yeah. And also the voiceover is by another Inupiaq grandma, amazing lady called Caroline. Yeah. And we've heard a lot of story about glaciers and indigenous culture, and some of them you also get to include it in the film.
[00:24:13.653] Kent Bye: Okay, and so what's next for you in this project then? Now you have the 360 video, then you have the motion capture, so where do you want to take it next?
[00:24:21.379] Jiabao Li: Yeah, we are making the interactive PC one with the multi-sensory feeling. You can interact, you can feel the temperature, you can smell it, and also I'm going to still continue to expand the Glacier series.
[00:24:35.630] Kent Bye: And what do you think the ultimate potential of virtual reality and immersive storytelling might be, and what it might be able to enable?
[00:24:43.731] Jiabao Li: As we talk about metaverse a lot, there's of course the sustainability part of the metaverse and there are three major points. I'll just talk about the third point is it's able to tell the story that it's very strong, that you can get into the shoe, you are able to create that empathy and if The ultimate way to have climate action is this empathy, is this behavior change. Then VR has a lot of potential to help people do that.
[00:25:15.984] Kent Bye: What were the first two parts of the, when you say there are three points, what were the other two?
[00:25:20.217] Jiabao Li: The first one is how can we, at the very source, change the algorithm and change how do we run the backend with more renewable energy. And the second one, it's about if we can create digital double to replace some of the physical world with virtual world. We can do digital fashion, digital online meeting or concert in the metaverse. And maybe that can reduce a bit of material demand that we want to have.
[00:25:51.832] Kent Bye: Are these three points from a talk that you gave on sustainability? Or what was that from?
[00:25:56.315] Jiabao Li: Yeah, I talk about it in Google UX Con and also Tencent Design Week.
[00:26:01.579] Kent Bye: And what was the topic of the talk?
[00:26:03.662] Jiabao Li: how to build a sustainable metaverse, but I also expanded further as multi-species futures, where I also talk about work co-creation together with squid and octopus.
[00:26:15.120] Kent Bye: Yeah, and there's actually the last two interviews I've done were both talking about different aspects of the sustainability. Okawari, you know, was talking about the carbon footprint. But, you know, one of the points that they're making is that there really isn't a lot of data to really know the ecological impact of some of these things and to know what the existing processes are and if it's going to be something that's just added on versus if it's going to actually be replacing. And they say Most technologies that are being added are more in addition to rather than replacing stuff. So yeah, I don't know if you've looked at that... Looked at any of those sustainability aspects. Some of your thoughts and conclusions based upon your looking at some of these different dynamics. You know, what needs to be done in terms of sustainable development or living in right relationship to the world around us.
[00:27:01.647] Jiabao Li: So if you think about the species that we have impacted on, every nine minutes one species disappears on this earth forever. And we are causing so much havoc of the things that can't be replaced. And maybe we can recreate something in the virtual world, but of course it's not the same. Glacier is one of those. And Working in this field, I think there can be more content around climate consciousness, climate action, multi-species futures, and working as a designer of either the headset or the ecosystem or the hardware, then how do we make it more energy-efficient itself? Especially when metalware relies on so much heavy computation, how can we design that better?
[00:27:54.095] Kent Bye: I just want to note, as I was asking that question, there was a bird that dive-bombed right in the middle of this courtyard that we're sitting in as you're talking about the multi-species future. I'm just really struck by that. But is there anything else that's left unsaid that you'd like to say to the immersive community?
[00:28:12.633] Jiabao Li: I have to say this is a great community. People help each other a lot, very open. And we've seen that it evolved over the years since 2010-ish era. And we've seen the film become so much more mature. And as more headsets getting out and technology evolve, I'm so excited to see works progress alongside with it.
[00:28:38.600] Kent Bye: Once a Glacier is a 360 video, is it available anywhere for people to check out, or where can people get more information about this project?
[00:28:45.363] Jiabao Li: On our website, we have a screencast, we have the poem there, and it's now showing in Rindon Film Festival in London, here in IDFA, and we are touring around the world. There's more to come.
[00:28:59.235] Kent Bye: Awesome. Well, I really enjoyed the piece and yeah, excited to see where you continue to expand and take it with the different motion capture and other technologies. And yeah, thanks for joining me today on the podcast to help break it all down.
[00:29:10.411] Jiabao Li: Thank you so much.
[00:29:11.950] Kent Bye: So that was Xiaobao Li. She's an artist, designer, and creative technologist, as well as an assistant professor at UT Austin, and she created a piece called Once a Glacier. So I have a number of different takeaways about this piece. First of all, there's two main segments to this piece, I'd say. Starting off as a 360 video, which takes you into this spatial representation of the glacier. As a young girl, she's kayaking in pretty deep into this glacier, and she's reading off a poem written by an indigenous Inuit American poet named Joan Neviuk Kane. And the strongest piece of that 360 video for me was the spatial representation of being inside of the glacier, and then seeing later how, as this young girl has aged into an elder, that glacier is now gone. juxtaposing the length of a human lifetime into the death of a glacier that should be normally on a geological deep time, but is happening in the context of the human time span. That's a lot of what was the inspiration of this piece. Going out and actually capturing field recordings of the ice and then translating this into this spatial experience. Story-wise, at the end, the woman who is trying to capture and preserve the ice into a refrigerator and then is discovered later. I think the story falls apart a little bit in the end because it gets into this weird speculative fabulation where it's a human intervention trying to save and preserve the glacier. But I think the juxtaposition between the human time span and the geological time span, I think, was strong in terms of using the spatial medium, especially as you're taking inside of this canyon from the beginning. And then there's this other part that was at the IFADOC lab, which was the motion capture stage. And so it's essentially someone who is dancing on stage, and then you see in the background all these different particle effects that are kind of shooting off her body as she's dancing around. And so you're watching what's happening with the dance. And the very first line of the poem that was written by Joan Navia Kane is, I am a glacier, my interior inscribed by a million years of invisible language. I'm an emissary from a formidable ancient world to this callow, brash world with mar and garble." So, somehow, as I'm watching it, I miss that very first line, that I am a glacier. So, I miss that the person who was dancing was representing a glacier. So, this for me, I guess, goes back to this concept of show, don't tell. And so, there was nothing visually that I was seeing that made it intuitively obvious to me that I was watching a woman who was dancing as a glacier. Normally, when I think of a glacier, it's not something that's dynamic and moving, but I think part of the point is to create this juxtaposition of how quickly the ice is melting. You know, as she's spinning around, there's all these particle effects that are dancing down. But I think it gets back to this, you know, principle in film, which is show, don't tell. And so what ways can you start to tell the story visually in a way that makes it intuitively obvious? But again, maybe this is just a poetic interpretation of someone who's dancing and symbolically representing that they're a glacier. I find personally sometimes when I'm listening to audio narration and the visuals, the visuals always dominate for me. And so sometimes if there's something that's hooking on to the audio, and if there's a particular segment there that's key to kind of understanding what's happening, Then I think that's where the potential starts to lose some of the audience. There's also generally this juxtaposition to this type of dance with motion capture with this poetry. There is a bit of a cognitive load of trying to listen and watch at the same time and sometimes just listen to a poem. It takes my full cognitive capacity just to listen and to understand and to use my imagination to have that flourish of the poetry. And so how do you start to mix the modalities of poetry with the immersive technologies? There have been different pieces that I kind of describe as visual poems because I feel like it gets the essence of trying to capture those spatial metaphors in a way that is able to amplify the story that's being told. But here in this case, you have much more dynamic live performance that's happening right in front of you on top of the spatial representation that's being depicted in the virtual representation that you're watching on the screen, but you're also watching the dancer and it's also a live performance. So it's happening, it's dynamic, and there's this whole thread of the main narrative components are coming through the poem that's being read on top of it. So I think it's just a matter of mixing these different modalities. I mean, maybe somebody who could listen to a song and be able to interpret the lyrics and understand day in the meeting on the very first time of hearing a song, there are people that are out there that are able to hear a song and know immediately what it means. For me, it takes listening to it 30 or 40 times and going back and reading the lyrics and really studying it before I start to uncrack maybe what the deeper meaning of that is. I think that's not my strength. And so whenever there's immersive pieces that are kind of leaning on that, it's hard for me personally to understand. That doesn't mean that that's universal. I mean, some people may have that combination where it's just like perfect for them. I think it's just something to consider as people are creating these immersive experiences, how, as you start to blend all these different modalities together, what are ways that you can start to amplify the deeper message that's being told there? So this is a bit of an experimental space for the doc lab and onyx studio to be able to come in with these fusion of these different projects. And so it sounds like the next iteration is to try to take the more interactive components of this experience, and to add different layers of multi sensory experiences with hot and cold and maybe some different smells, and there's also different colors that were integrated into this piece, both into the poem but also in the color palette that's being used. And so, yeah, just trying to take this topic of the melting glaciers and find all these different various ways of fusing it together into these different modalities. And so we have the 360 video version, the live dance performance version, and now the next phase is the interactive version. For me, I see this as kind of an experiment of trying to mix all these different modalities. Actually, I should mention that at the actual performance, there was a handout of Once a Glacier that had both the poem that was being read live during the dance performance, as well as the version that was read during the 360 video. I should also say that the second part of the dance was a live reading of this poem. There are certain words that are highlighted. both the original 360 video, but also in the live performance, as you see on the screen, there's segments of the words that are being highlighted as the poem is being read. I think it actually does help to amplify different things that are being read. It's not every word, but it's just certain phrases and segments of the poem. But during the live performance, there was a young actress who was reciting that entire poem from memory that was overlaid onto the same visuals that you would see in the 360 video. So, yeah, just different ways that they were blending in. And during the live performance, there was some AI-generated imagery that was going through space and time as well, going from the Ice Age down into the current time. So just different ways of using just even the video and generative AI art on top of this live performance that was trying to set the original Ice Age context of the Dancing Glacier down into the current timeframe of this human life that goes from the young girl into growing old and kind of reflecting on her time with this glacier. So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
