Duncan Speakman’s Only Expansion was an augmented audio tour showing at the IDFA DocLab that was doing real-time modulation of the surrounding soundscape mixed with produced sounds in order to create a unique experience. The headphones had microphones attached in order to do the real-time modulation, and it also used a custom-built audio processing device to do the real-time processing that is impossible on existing phones and AR head-mounted displays. There was also a companion book for the people going through this experience that provided prompts for what they should pay attention to. The overall experience covered themes of climate change, and used a number of different real-time audio modulation techniques in order to amplify the building and releasing of narrative tension.
Only Expansion received a Special Jury Award for Creative Technology at the DocLab, and I had a chance to talk with Speakman about his music composer background, his journey into producing audio tours, Torsten Hagerstrand on time geography, Kristine Jørgensen on extradiagetic sound, Christina Kubisch’s Electrical Walks, time and our perception of time, and the different ways that he modulates sound in order create and release narrative tension.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So continuing on on my look at some of the narrative innovations that are coming out of the ITFADocLab, today's episode features Duncan Speakman and his piece called Only Expanse. So this was an augmented audio tour. So you have these headphones that have microphones on them, and he's got this special box that does real-time audio processing that he custom built. So you're walking around Amsterdam, and you're taking sounds from the environment, and they're being seamlessly blended in into recorded sounds that he's taking on in this whole guided audio walk. You've got this book that's giving you specific instructions for what to pay attention to as you're wandering around the city, and he creates this whole narrative that is focused on the theme of climate crisis that we're in right now. So this piece actually won one of the special jury awards for creative technology and definitely very compelling and really pushing the edge of what's possible with like real time augmented audio, you know, with a custom built system that does stuff that, you know, none of the other existing either phones or augmented reality HMDs can do. So this was definitely a unique experience in that sense. So we'll be covering that and more on today's episode of the VCSVR podcast. So this interview with Duncan happened on Friday, November 22nd, 2019 at the IDVA DocLab that was happening in Amsterdam, Netherlands. So with that, let's go ahead and dive right in.
[00:01:34.837] Duncan Speakman: So my name is Duncan Speakman. I'm a composer, I think, really. And a lot of my work is getting people to walk around with headphones on through urban environments, sometimes with pre-recorded media. Currently, I'm working with augmented audio. And sometimes I combine books and print with that. But mainly, it's always about trying to work outside of venues and gallery spaces and trying to sort of throw people into uncontrolled environments.
[00:02:04.943] Kent Bye: Great. So maybe you could give me a bit more context as to your background and your journey into this space.
[00:02:11.047] Duncan Speakman: So I'm an artist by accident, is the way I always kind of describe it, because originally I was a sound engineer and a musician, and then I started working on documentaries, traditional film-form documentaries, as a sound post-production and some directing. And then I moved to Bristol because it was kind of the home of documentary in the 90s and I had lots of records from Bristol and it seemed like a good kind of combination. But when I moved there, the industry shifted into sort of shutdown because there was this move from big, massive budget documentaries to sort of lower budget, reality style, following people around a hospital kind of thing. So at the same time there was this big surge in interactive documentary and broadband was just starting to appear and there was lots of different projects to work on interactive documentary and I worked with a few companies on prototypes for interactive documentaries but none of them could be broadcast, essentially, mainly because the technology wasn't there. I hope not because they weren't very good, but there weren't the sort of distribution methods or systems out there. But a gallery or an arts organization invited me to make one. And so I was like, Oh, okay, I'm an artist, I'm making art, I see. And so it kind of carried on from there, I sort of worked on a lot of interactive installation type work, projection mapping. And then I was slowly trying to bring sound and music back into the work over time. And The way I kind of got to Audio Walks was by being involved in the sort of early video blogging community, sort of pre-YouTube, and I was making all these short, one-minute observational films. They'd just be like a locked-off camera shot of somewhere and I'd write a small piece of music for it. And then I was invited by Mobile Bristol, which was part of Hewlett-Packard Labs at the time. They were doing research into locative media, and they invited me to make some pieces. And I tried out their GPS-triggered audio technology systems. And I tried it out with some of the soundtracks from my films, and I just sort of had that epiphany moment where I was like, oh, these are basically the films I've been making. I'm standing here in the street looking at a situation, listening to the soundtrack that I'd written for that situation. So it kind of just from there carried on with that sense of how to use sound and music. It's very music led, my work, how you use that to kind of frame what you're looking at and sort of leave space for the world to happen in a way. And so, yeah, I've kind of been doing variations on that for the last 10, 15 years.
[00:04:45.462] Kent Bye: Yeah, well, we're here at the IDFA DocLab 2019. And I had a chance to do your experience yesterday. And last year at DocLab, I had a chance to do Gabor Arora's and Lauren Hutchinson's Pilgrim, which was a little bit of a walking tour through Amsterdam. And you would stop. And if you stopped, then you would get another Pilgrim audio recording that would walk with you. So it was trying to recreate this feeling of walking on a trail with people. And this year, with your piece, it was augmented audio, so you actually have microphones on the headphones, and you're recording the sound around me and modulating in different ways. So maybe you could talk a bit about this project and what you were trying to achieve with it.
[00:05:27.217] Duncan Speakman: Yeah, so in some ways it's important, I guess, to talk about this project in relation to a previous one. So I made a piece of work called It Must Have Been Dark By Then, which used some of the same source material, and that was a geolocator GPS audio walk that brought you narratives from different places around the world undergoing rapid climate change. And you had a book that was much more narrative. It was these long travel journals of my journeys through those places. And the idea was to try and connect where you were here to those other places. So it would invite you to go to some water here and you would hear me by some water in Louisiana and you'd read the story of me being there. The problem I had with it was, I mean I really like the piece, I'm really pleased with it, but I really had this sense of audio isolation in it, in that you're, much as you're hearing these sounds from these other places and they are having a resonance with where you are, you're not hearing where you are at the moment. And the language sort of acted as a barrier as well in some ways. You were reading my experience. So I wanted to try and rethink it in a way that you're connected with the sound environment that you're in. Because one of the things about soundscapes is that you're always contributing to them. You know, you're not just listening to a soundscape. I really like this idea that Salome Vogelin talks about, that everything you're doing, every breath and movement you're making, is part of the soundscape. So there's this relationship, there's this kind of entanglement between you and what's going on around you. And that really spoke to these ideas around ecology that I was trying to get at, trying to rethink this sort of human-nature divide, this kind of subject-object, looking at pictures of melting glaciers, looking at lonely polar bears, and it all felt very distant. And I wanted to try and do something where you're hearing these other sounds, you're hearing wildfires in America, you're hearing flooding in Scandinavia, But you're hearing it at the same time as you're hearing where you are, and those two things start intermingling. So it hopefully speaks to that idea that your actions here have a relationship with all those things happening in these kind of distant other places. And as a composer, I also use music in the piece as a way of I'm trying to get to a sort of viscerality. I'm trying to get to a sense of, I think, horror and darkness, actually, in a way. I sort of feel like a lot of eco-critical and ecological thinking work is still dealing with the sublime. It's still showing us these incredibly beautiful images of terrible things, you know, and you look at these amazing disaster porn, in a way, you know, you look at these incredible photos of sort of bleached out landscapes or coal mines, and we talk a lot about what we're going to lose. And I sort of wanted to somehow tap into that, things might be really bad. And I think music and sort of harsh, dissonant kind of sounds are a way that you can do that. You know, you're physically invading someone's body with those sounds.
[00:08:26.615] Kent Bye: There's a number of different aspects to this piece. There's the actual content of the piece of the message of climate crisis that you're alluding to here. But there's also this element of the mechanics of getting a book that would give instructions for how to move, different ways to pay attention or to move through space. So maybe you could talk a bit about the book that you have that accompanies this experience that was trying to, in some ways, give some instructions for different modes of wandering.
[00:08:52.295] Duncan Speakman: Yeah, so the book, you know, there was a part of me that wondered if I could make this piece work entirely without any text, without any prompts, and it just, it just didn't quite work. It became, I really like ambiguity and I like some sort of openness and interpretation, but it was just, too much and then it would have been just a sound work which I wanted to be something a bit more than that. So the idea with the instructions and the prompts is it's kind of like you say it's about trying to get people to attend to their surroundings in different ways because I think that sense of attention is again part of that kind of ecological debate actually sort of paying attention both to not just to the trees and the plants but to the people and the social and political structures around you. I think Just being aware and finding different forms of attention and different ways of attending to the world Might be the solution not I'm not entirely sure but I hope they are so the book is There's a kind of narrative arc to the way they're framed which I suppose I don't want to talk too much about because people get to do the piece but one thing that is quite important with it is that it involves a return journey and I use that quite a lot in different pieces because I think you have that sort of first pass through a load of places and it's asking you to sort of pay attention to them in different ways and think about them in different ways. And then the return journey is this sort of reflection on time because you're travelling back through them and you kind of remember yourself being there and hearing something different and it becomes, I guess, self-reflexive in a way. And that's quite an important thing for me in terms of the structure that would be really hard to do if you didn't have that prompt. I do worry at the moment that maybe it's too chapter-based. That there's a fluidity maybe that it's missing that I'd like to think about a bit more. It kind of works. You know, the second section is a long return journey, so that sort of works for me.
[00:10:45.656] Kent Bye: Yeah, there was a moment when you gave the instruction of, as you take a step, you're thinking about different units of time. So it's a bit of like symbolic time. As you take a step, you're able to project out and think about longer scales of time. And at the same time, you have the audio that starts to then go into a cycle and cut in and out. And I've been thinking a lot about time and different modes of thinking about time. There's the monochronic way of thinking about it, like the Newtonian uniform time that's moving forward. And then either a little bit more relational or cyclical time or polychronic time, which is more about cycles of time that are coming back to the same point. And so as I was walking and thinking about this polychronic cycles of time and listening to the audio, I suddenly realized that There's microphones that are in some ways amplifying the sound around me and the bikes are streaming by ringing their bells and I kind of hear this modulated time as I'm thinking about time. So maybe you could talk about what you were trying to do with time and playing with time and all the ways that you're trying to modulate people's experience of time.
[00:11:47.218] Duncan Speakman: Yeah, I mean how long have you got? Not wanting to make a pun but I mean it's so a lot of my my actual research at the moment sort of compositional research and thematic is around time and there's a I guess a couple of different things which is What I'm trying to work with at the moment is the tension between that quantifiable, measurable time of a digital sound file and of a software system that's working towards my compositional aims, because I like interactive work, but as a musician, when things happen is quite important, and things I might want to say to build tension and release, you know, you need to be able to control that time. And yet the piece is trying to speak about living on multiple timelines. And I think there's something really interesting for me about the fact that you've got this lived experience in the piece. You've got 40 minutes of your life and there's a bunch of sound files which add up to 40 minutes. But then you're hearing recordings from other places which are from the past that are suggestions of geological time and of shifts that are happening. And then there's time essentially temporal processes in the sound that sort of delay some sounds and stretch them out, and I use this amplitude modulation like a tremolo as a sort of audio strobe. I was always interested in the idea that visual strobes, you sort of flash a light on off and things seem to slow down because the frame rate's decreasing. With audio, it seems to almost have a different effect. When you start sort of flickering the sound on and off, things actually seem to speed up somehow in a way, and there's this sense of acceleration that happens around you. I don't have a scientific sort of proof or reason for that, but it just sort of feels that you can change that sense of the pace of things happening around you. And you can also do it with music as well. You know, it's a really natural thing that your body starts to fall in sync with different rhythms of music. So as I have slower, more drone, dragging out pieces of music, you feel a bit slower and as the sort of pace increases. So the piece is definitely trying to manipulate you, basically, your experience of time. For those reasons that you're talking about, I want you to think about how time works and that human time is one thing, but there's microbial time, geological time, there's all these different things happening simultaneously with their own paces.
[00:14:05.922] Kent Bye: I mean, there's a lot more to that, but we can... Well, there's certainly a bit of an open philosophical question as to what exactly time is, but the... It's a nightmare.
[00:14:15.130] Duncan Speakman: Oh, you just... Once you start down that route, it's just... It's hell. Because you've got all the perceptions of time as well. You've both got the philosophical questions around how we perceive time, and then you've got the scientific questions around what is time, and then you've got the sort of relational questions around is time just... our relationship between other things, you know. But then at the same time, as was brilliantly, I think it was in Kramer's book around the time of music, there's a brilliant line that just says, and all this is great, but none of it helps you know how long to book the recording studio for. And I think that's just a reminder that, you know, as sound people working with stuff, you know, there's generative sound systems, which are really interesting. Those take away those questions of sort of time and process, and a live performance is fluid in terms of its use of time, but our sort of lived experience of music, it takes time. This piece takes 40 minutes. There's an amount of time of your life that is, however it's experienced, is somehow measurable. And that's constantly in tension with all these other different approaches to time. It's really messy, it's really...
[00:15:19.920] Kent Bye: Yeah, well, the Greeks had two words for time. They had the chronos time, which is the more objectified ways of breaking up, like, the 40 minutes, and then the kairos of the quality of the moment. And I feel like that you're kind of playing with both of those in this experience and trying to, in some ways, encourage people to dip into that kairos time flow state and encourage them to wander in the art of wandering. And I just did an interview with Samantha Gorman of Tender Claws, and she was citing some French theorists that was talking about derivés or some... Derived, yeah. Is that something you're familiar with or used as well?
[00:15:51.011] Duncan Speakman: I mean, it's an interesting one, the sort of derived, because there's a lot of other problems that come with that. There's a lot of privilege that comes with the derived as well, which is difficult to unpack. And, you know, like this idea of being able to just drift and flow through spaces, it's that sort of luxury of the sort of European white philosopher, you know, in a way. And you look at, you know, it's linked to the flaneur and like Teju Cole talks about how you can't be a black flaneur because the spaces that you move through are charged in a different way and they might not be a safe space that you can just drift through. You're constantly, I can't remember the exact quote to be honest, but it was a thing about how you've sort of got to constantly practice psychogeography and that takes a psychic toll on people. And so we quite often talk about this Derive and this Flow and forget sometimes about how that's very different for different people, you know, and that experience of just being able to walk and flow through the city. But yeah, and I don't, you know, I don't think this piece sort of unpacks that really in any way or tries to sort of deal with that. I feel like I've lost my train of thought now. What was the original question? The Kronos and Kairos thing. I mean, I think the other thing I would say is I don't think it's just me. I kind of think that all... I really have this thought that all kind of mixed reality works, you know, have to deal with that in a way. Because if they're interactive or they're dealing with the world going on, there's multiple timelines happening. That's very different, I think, to cinema or, you know, VR or something. Because in those moments, you're sort of... You're not expected to be thinking about the everyday time of your life. You're going into the time world of that. But when you're doing something that's in the everyday, that's in these uncontrolled spaces, there is a time frame that's going on around you. And the work, whatever the content is that you're making, needs to think about that. And I think a lot of works don't do that. Some do really well, but yeah.
[00:17:47.698] Kent Bye: As a musician and composer, do you play around with irregular time signatures and try to do like polyrhythmic times or try to have multiple rhythms and cycles happening at the same time?
[00:17:57.383] Duncan Speakman: Yeah, I mean a lot. Sometimes, I mean there's another piece of work that I made in collaboration with Sarah Anderson that's a sort of multiple speaker symphony for the city where you have like 40 loudspeakers and each audience member carries one and we write the music so that each speaker is a different instrument. So one might be the violin part, one might be the voice and so on and so forth. But because of various technical limitations, we could never get them to all start in sync. So we had to write the music as phase composition, sort of Steve Reich style. So that whenever they started, they would somehow meet and overlap with the other pieces. But we would use irregular times and that too to create these kind of overlapping flows. And I, you know, I think that's a really nice thing about music, isn't it? It sort of, it both uses time, but it also creates it and it shapes it and it manipulates it. Yeah, it's quite perplexing.
[00:18:48.166] Kent Bye: Well, I had a number of experiences on The Only Expanse of walking around the city and having the audio that I'm perceiving with my eyes. People often in XR talk about how the visual field dominates, but yet when you have synchrony between what you're seeing and what you're hearing, then your mind associates it. And so at the same time, I'm hearing a lot of recorded, produced sounds, but also overlaid with things that are clearly synchronous with what's happening in my environment. There's a bit of like cognitive load of trying to suss out like what the reality is of what I'm hearing and what's real-time and what's pre-recorded But I know on the way back there was somebody who was speaking Russian Coming opposite of me and their voice was sort of echoed and modulated in this sort of really watery etheric way and it just echoed in my I don't know just really profound because it was clearly This person's speaking and I could see their mouth moving But it was like this echo we and kind of the Doppler effect at the same time as he's walking by me So maybe you could talk a bit about different ways that use audio modulation for what you're trying to achieve with your narrative
[00:19:52.330] Duncan Speakman: Yeah, well I mean that section you're describing is, I mean in some ways it's a pretty sort of blunt instrument. You're hearing recordings of rising sea levels from numerous different locations. Something that is quite important for me with this piece is that there's never a moment where it's, oh now you're hearing the sea in Louisiana. You know, I don't want you to be kind of, I don't want to be making this claim that this is what the sea sounds like in Louisiana. So there's multiple places simultaneously you might be hearing it from Louisiana and Norway and the UK. So you're hearing these kind of rising sea levels and then after that it shifts to underwater recordings from those same locations and then uses filters and echo to process the microphones picking up the sound around you so they have that sort of underwater quality. So it's funny, it's sort of, it is quite a blunt instrument. It's like, here's the sound of a respirator and it makes everything sound like it's underwater after you've just been listening to rising sea levels. But it kind of works and it's sort of, I sort of like how it works. And the most interesting, not the most interesting, but I guess a lot of responses that have come from that section are people talk about that as quite often the saddest section of the piece because they feel the most distant from everyone else around them not just sonically but also I think because everyone's so muted and distant and they talk about it as no one else seems to care like everyone's just carrying on with their daily lives whereas you're walking along hearing what might be happening to all these people in the future and a lot of people have sort of responded to that in quite a kind of quite an emotional way I guess which has been quite interesting.
[00:21:32.069] Kent Bye: Well, it seems like that with these immersive technologies, we're pulling from film theory and music theory and video game design and embodiment and theater. And so I'm just curious, since you're going and getting your PhD, what you're drawing inspiration from, from a theoretical perspective, to create your experiences.
[00:21:51.815] Duncan Speakman: So, okay, so I'll try and do this simply rather than trying to crunch a PhD into a soundbite. But there's a couple of things happening because there's just the work itself which is part of the research and just the doing of it and finding stuff out through doing it. But two things that have come up that have been really interesting for me are time geography, which came out of Sweden in the 60s, 70s? I hope I got that right. I should know this stuff by now. Torsten Hagenstrand, he was looking at social mobility and so he was sort of looking at how you measure bus timetables, essentially, like where do the buses stop? How do people get to the buses? And then this shifted into work, like where is your work? How long does it take to get to your work? And he was really trying to sort of bring together time and space. And what was really interesting about it for me, and in terms of this kind of work, and I think a lot of, like, I guess the pitch of my research at the moment is that it's a really useful set of tools for analyzing this work, because it has a bunch of really useful things like tasks, like tasks take time, and you can sort of only do one thing at a time, so you can't be in the train station and back at this venue at the same moment, and it takes an amount of time to get from one to the other. So if you're designing a sort of audio walk, You literally have to take that into consideration that it takes this amount of time and so you might need content. In his work he was looking at how parents, where the creche is and like how far away the creche was from where they work and were the opening times of the creche versus when their work starts. But I think these kind of constraints are really important to think about. Authority constraints, which are, you're only allowed in a building at a certain time, or a park's only open at a certain time. So in designing a walk, if you know the park closes at six o'clock, you've got to think about when that happens. So there's a bunch of other stuff in time geography, but this sense of looking at the kind of constraints that come into play when you're dealing with physical space and temporality simultaneously, which is what we're doing in these kind of works, is really useful. The other stuff I think that's come up really recently for me, which I'm really interested in in relation to this piece specifically, is Karen Jorgensen? I hope I've got her name right. So she's a gaming research designer, and she's been talking about extra-diegetic sound, which is a few different schools of thought use that term differently, so it's not a sort of catch-all. But the way she talks about it is, in a computer game environment, so in film sound you have diegetic sound, that's kind of sound in the situation that the characters can hear, and you have non-diegetic sound, which is the soundtrack or narration, the things that we as an audience hear, but the people in the film don't hear. So in a computer game, what she talks about is the way designers use music to alert you to things happening. So you're controlling a character in a computer game, and you're making them move through a forest, and then the dramatic music starts. And you know then that something bad is around the corner, so you change the way that character's moving. The character doesn't hear the music, but their activities are changed by it. And there's this really nice sense of like, oh yeah, there's something interesting here. The sound is manipulating the everyday world, even though the everyday world doesn't know it's happening. And so when I think about something like Only Expansion, where what you're seeing happening around you seems to change, your perception of it seems to change, and yet that soundtrack that you're hearing is not out there in the world. The sound of the world hasn't actually changed. that I'm starting to think about whether this notion of using music in audio walks to change your sort of everyday movement feeling, even though it's non-diegetic sound to the environment you're in, if you see what I mean. So I'm kind of interested in how that sort of thinking might apply to these kind of situations.
[00:25:41.468] Kent Bye: Yeah. And finally, what do you think the ultimate potential of immersive technologies might be and what they might be able to enable?
[00:25:52.593] Duncan Speakman: That's a big question, isn't it? I mean... I guess my interest is if you can get people to attend to things. So my take on immersion is slightly different. I feel like there's a lean towards immersive technologies as meaning cocooning things, things that cocoon you, VR, cinema, immersive theatre. And I'm interested in thinking about the idea that maybe immersive media is not about cocooning, but it's about exposing the things you're already immersed in. Only Expansion is trying to expose you to the sound world and the people around you through the way you're listening to it. Christina Kubisz's electrical walks, they expose you to the electromagnetic frequencies that are around you. I think of these as immersive works, not because you're immersed in the work itself, but the work is showing you how immersed you are in the world. So I guess, yeah, my potential future dream is that through these things, maybe we make more work that just opens us up to how connected and how wrapped up we all are together on this tiny little planet. You know, it's like, so yeah, that's kind of, that's where I'm at.
[00:27:06.268] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the immersive community?
[00:27:10.389] Duncan Speakman: Well, all of the immersive community. Come and try Only Expansion, I'd be really interested. You know, the challenge is novelty, right? Like, all these things you feel like, I've made this thing with this technology system, and then someone wants you to make something different using a new technology. I want to make a different piece using this same system, you know. And not for people to go, oh yeah, I've done an augmented audio piece. It's like saying, I've seen a film, I don't need to see any others. So let's make sure we don't constantly run towards novelty and let's keep making stuff. Once you find a form you like, keep making stuff in that form until you are the master of it. That's what I'd say. I think that feels really good. Exclamatory. I'm not trying to tell everyone what to do. But I like to try and master things before moving on.
[00:27:59.070] Kent Bye: Awesome. Great. Well, thank you so much for joining me today on the podcast. Thank you.
[00:28:01.973] Duncan Speakman: No problem. Thanks for having me.
[00:28:03.815] Kent Bye: So that was Duncan Speakman. He's a composer who created the augmented audio experience called Only Expanse. So I have a number of different takeaways about this experience is that first of all, Well, there are certain elements about the experience that Duncan didn't talk about. And so I'm going to just expand out a little bit more just because it was a part of the overall narrative structure that he was creating that I think just is helpful. Because in this book, he's asking you to walk around and to pay attention to different things. At one point, he asks you to go to a low point and then asks you to go to a high point. And then from that point, then he starts to have you walk backwards. And so as you're going from the high ground to the low ground, then you're kind of submerged underwater. So I think the larger context and point, if it wasn't clear, is that you're going from low to high and creating that contrast. And then he's doing all this audio augmentation to be able to make it sound like you're kind of walking through water. so people are walking by you and you kind of feel like in this etheric underwater submerged nature and before that he has you do this geologic time walk so for every step you're taking you're kind of stepping into the future so it's like this symbolic time so you start to operate on many different time scales you're obviously there physically and today but you're imagining the future and as you're walking around he's taking you on this soundscape. In the soundscape, he says that you're both creating and participating in it. So as you're breathing, as you make noise, as you speak, you have this kind of real-time audio modulation and he's doing a variety of different real-time audio effects. And so probably for me, one of the most impactful one was this audio strobe where he's cutting in and out and you kind of hear this. And he says that when you have like different strobe effect, you kind of imagine it makes it sound like things are kind of moving in slow motion. And so he's, already changing your perception of your time and there was definitely some moments in this experience when I'm walking around I don't quite know what is the reality of the situation because he's feeding in all sorts of sounds into me that are pre-recorded and produced and obviously not happening in the environment. And then he's adding additional sounds to me that is modulated by the environment and doing different post-processing effects. And so my brain's able to figure it out, but it was also this extra cognitive load and somewhat confusion and somewhat feeling this surreal, flipping me into this different state of being. So that was definitely interesting. And I think it was also compelling that as things are moving around you, you see the visual synchrony of that and you have an expectation of what that should sound like. And then because he's doing all this audio modulation, it completely changes your experience. So it was a very powerful experience and I like this kind of approach. And after we stopped recording, I started to ask him a little bit about his box and he kind of described to me some of the details of what he needed to do to do this real-time audio processing. And he was saying that, you know, this is not something that any of these existing devices can do. So this is definitely a unique experience that required this type of custom Bespoke built technology. So he's been looking at audio walks for a long time and there's a tradition and other things that he's looking to. So there was people like Torsten Hagenstrand's time geography. So looking at how much do processes take, how long does it take to get to work and trying to map out. different authority constraints, like do you have the ability to go into the subway at this time or is the park going to close? But also looking at other time constraints for how long it takes to get from one place to another. So as you start to think about doing these types of audio tours and audio walks, you have to map out the time geography and do some sort of guess for what the average is, you know, unless you do like GPS located. But in this case, you're just kind of walking around and he's got this 40 minute piece. Some of the other theorists that he's mentioned is Kristen Jorgensen who's thinking about extra diegetic sound so when you're in a video game and you have like the scary music start to play then that changes the way that you're moving around and so what's it mean to start to try to add this layer of extra diegetic sound on these different types of audio tours. He wants to try to consider immersion as to just being immersed in the world around us and so He points to the work of Christina Kubitsch and her electrical walks where you put on these headphones and you get to hear the electromagnetic frequencies of the space around you. And he gave a brilliant talk at the IFA DocLabs conference day where he was giving people a very different experience of time. And he talked about this piece by John Cage that didn't have a time signature. And so people who were these academics wanted to have this organ play this piece for like over 650 years or something like that. a single note we'll be playing for a year or two and this piece is going to be played over the course of like 600 years. So during his talk he was just playing these notes and giving this sense of this different scale of time. So I think that's very fascinating and he's sort of theorizing that all of these immersive works are trying to kind of flip us out of the time of our everyday life and to allow us to slip into a different mode of time. And that's something that I keep thinking of this Kronos time versus the Kairos. And, you know, that Kairos of slipping into that different mode of time for you to be really present and to pay attention to these different cycles and how the previous context, you know, matches up with the current context. And it often happens when you're on vacation or wandering around. He's somewhat skeptical of encouraging everybody to just be able to wander because not everybody's able to wander around equally in the spaces. And there's a certain amount of luxury and privilege to be able to actually do that. But. the same time he's trying to encourage you paying attention to the world around you in a new way and to be able to allow you to pay attention to different systems and societies and structures around you and I think that was the big thrust of his work is to you know do this ecological turn so to see yourself as a part of the larger ecosystem and to start to pay attention to these larger fields of dynamics so that as you're walking around If you as an individual is able to pay attention to the larger structures that may be invisible to you, then maybe these audio walks can start to bring your attention to these larger systems and structures and allow you to come away thinking a little bit more holistically and more a part of the ecosystem. And perhaps it's that ecological thinking that's going to allow us to get out of a lot of the mess that we're in today. So, that's all I have for today, and I'd like to just thank you for joining me here on the WisaVR podcast, and if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon listeners like yourself in order to continue to bring you this coverage. So if you'd like to see this project continue, trying to do this real-time oral history of the evolution of the medium, then please do become a member of the Patreon. Just $5 or $10 a month goes a long way to allowing me to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
