Darkfield Radio have released a couple of binaural audio performances that transform your home into a immersive audio stage. Their experiences are intended to be experienced with another person, each of their own mobile phones at a specific time so that it recreates the feeling of attending a performance.
I had a hard time differentiating between reality & the binaural audio recordings within Darkfield Radio’s Double as it was one of the most immersive audio experiences that I’ve had yet.
I had a chance to talk to Darkfield Radio’s creative directors David Rosenberg and Glen Neath during the Venice VR Expanded Festival, and I’d highly recommend checking out one of their performances before tuning in to hear more about their journey into immersive audio, their creative evolution from site-specific audio performances, to exploring the future of spatialized audio and blurring the lines between what’s real and what’s pre-recorded.
Be warned that their work is within the horror genre, but it’s more in the vein of cultivating some underlying tension rather than cheap jump scares. It can provide a great escape and cathartic release as Halloween in coming up, and give you the feeling of attending a performance without even leaving the comforts of your own home. You can check out the trailers for Double and Visitors down below.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
Here’s the trailer for Darkfield Radio’s Double
Here’s the trailer for Darkfield Radio’s Visitors
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So Darkfield's Double is one of the most immersive audio experiences that I've had so far, and I highly, highly recommend people go check it out. We're going to be talking about it here in this interview. And I don't want to have anything spoiled for you, because I think it's just better for you to know the least amount possible and just know that it's got some incredible specialized audio. You do have to set a specific time to go see it. You're either going to be in the United States or United Kingdom to have access to the tickets. And it's a show for two people, so you can watch it with somebody. And it's great to be able to see it co-present with somebody as well. It can work over a teleconference as well, but I think it's preferable just to be in the same space. So when we talk about virtual reality, when it comes to spatialized audio, they're at the point of doing these analog, binaural recordings where it's really difficult for you to tell the difference between what's real and what's not real. And to play with that concept within the context of a narrative, then what dark field radio is doing is really quite interesting. So before you dive into this conversation, I really recommend going to go listen to the experience, just so that you get a more immersive experience of it all. And if you're really into spatialized audio, then feel free to go ahead and listen. But I do think that you'll have a much more interesting experience if you do have your own direct experience of this piece, and then come back and listen to this interview. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with David and Glenn happened on Sunday, September 6, 2020, as a part of Venice Film Festival's Venice VR Expanded. So with that, let's go ahead and dive right in.
[00:01:47.217] David Rosenberg: So I'm David Rosenberg, and I'm one half of Darkfield, or a third of Darkfield, but one half of the creative direction.
[00:01:55.483] Glen Neath: And I'm Glenn Neath, the other half. We've been working together, myself and David, for 10 years, making shows using binaural sound and usually complete darkness. So Double is a little bit of a sort of sidestep on from that, in that, you know, we can't control the lighting state as usual.
[00:02:13.195] David Rosenberg: So we started making work for location-based experiences, and these started off in theatres, and then over the last four years or so, when we formed Darkfield, then we began touring these experiences in shipping containers around the world. And in all of the experiences that we've done to date, the audience enter an environment, put on headphones, and then very soon the lights go out. And almost all of the work is in total darkness. And I've been working with binaural sound for quite a lot before we started making work in the dark. And one of the things that we realized is that as soon as you have an image, then your brain first makes sense of the image and then attaches the sound to that. So certainly for spatialization, you're much more likely to follow an image and then expect the sounds to come from that. which is why movie theatres worked pretty well with just one speaker in the room and you kind of still thought that the sound was coming from the lips of the people on the screen. So this move to making work in the dark was to really explore what we could do with creating sound environments that the audience inhabit where they would create the pictures and the sound would absolutely lead everything. So the vast majority of the way that we make these sounds is through quite an old technology. It's through dummy head recordings, so binaural sound recordings, where you use a dummy head with microphones placed where the ear holes normally are. And that creates, when you listen back with headphones, then you are essentially listening from the position of that microphone. And one of the key things that these kinds of microphones allow for is the proximity of sound. And this is something that with real-time spatialization and other algorithmic ways of spatializing sound, we still haven't quite got there yet with creating this presence of a character as they come close to you and as they lean in and whisper in your ear, where you feel it, you sort of feel the presence of a person in the space. So as we began to develop and we use other techniques for spatializing sound as well, the binaural recordings still form a real bedrock of how we make the sounds, just in order to really create these environments that you feel as well as hear.
[00:04:43.987] Kent Bye: Yeah, I've definitely heard famous demos like the haircut demo where someone's cutting their hair. And this is also seems like very old technology in some sense. It's not like binaural audio is new when it comes to like virtual reality. Maybe you could just catch me up a little bit in terms of like the history of this type of binaural audio recordings.
[00:05:04.958] David Rosenberg: So I think the first recordings that were done, which were kind of binaural, were probably about over 80 years ago. And they would use two microphones that were in a certain configuration from each other. And then someone had the idea to put a baffle in between these two microphones that sort of created a shadow, like the shadow of a head. And then from about the 50s or so, then people began to use more formed mannequin heads. And then these gradually developed. So both the understanding of what is required of the physical structure, developed and also the microphones themselves, they just got better and better. But I think that in terms of the quality now of binaural recordings, I mean, we generally, one of the main microphones we use is a Neumann dummy head, a KU100, which it's kind of as good as it gets. Like, I mean, this is the interesting thing about immersive sound in relation to the immersive image, whereas With sound now, we're at a point where it is actually quite difficult to distinguish between real sounds and recorded sounds. Whereas still, obviously with image, it's far more complicated and that's still something that will continue to evolve for quite a great deal of time yet, I should imagine.
[00:06:26.172] Kent Bye: Maybe you could tell the story of how you started to collaborate with each other and maybe talk about some of your earlier projects that you started with exploring immersive sound.
[00:06:36.833] Glen Neath: Well, our first show was called Ring, and as David said, it was made for theatres. So we recorded in a particular space and then we toured it to various sized theatres after that. And one of the things that we found, the disparity between the space that it was recorded in and the space that it was played back into was one of the reasons why we moved on to creating shows for shipping containers, where we were able to always know the space that it was being presented in. So that's the first show, Ring. I think one of the key things that we decided, which took a while to come to this decision, actually, was that we wanted to hide the technology because at this time, you know, there'd never really been any shows using binaural sound and it was very, very new. And the great thing about binaural sound is you can go into a jungle and it feels like you're in a jungle and go wherever and it feels like you're in these particular environments. But the decision we made very early on was to keep the audience member in the room that they were sitting in. So we played with the idea of the other audience members in the space with them. And that was a sort of a key decision we made very early on, which has infused basically every project we've made since. We always play with this idea of what is really happening in the room. You know, there's a fictional space and then there's this, there's the real space that you're sitting in and that we constantly refer to that real space. You know, we ground the audience by moving a chair to your left or somebody coughing to your right. So you're never quite sure. what is real and what isn't.
[00:07:55.292] David Rosenberg: And this is a real key thing for the way that we use sound and it's about creating this doubt for the audience and the doubt as to what is real and what is imagined and in location-based experiences then we can usually create much more doubt in those experiences because the audience just aren't so familiar with the environment and so when the lights go out and you begin to hear someone walking across a table in front of you then you're not completely sure whether or not that's happening or it could be happening. And in some senses, it would sort of be easier to have a live actor doing some of these things in the space rather than try and create everything with sound and other effects. And with all of the shows, we've found other ways of augmenting the sound through other sensory effects, so vibrations, low frequency sounds, aromas, air movements. All of the same kinds of things that location-based VR will also think about using in order to augment the authenticity of what's going on. And those little details, they open up this area of doubt even further. So when we began to think about making shows for audiences at home, then the doubt that we can create is far less. And so we had to think very carefully about the narrative and how we played with the location that we asked the audience to be in, in order to still have at least some things that felt, did that actually happen? Or was I hearing it or did it actually happen in my room?
[00:09:27.662] Kent Bye: Yeah, well, you're showing this experience called Double here at the Venice Film Festival. And it's one of the performances that you have to get a ticket to, which means that you have to get a ticket, and then you download this on two apps. And then you listen to this whole experience with another person. And so we're going to eventually maybe talk a little bit more about that and then potentially get into some spoiler areas. And I'll let you kind of lead that. But also I'll give a warning to the folks at the top of the podcast. Again, I always recommend folks to actually have the experience. And it does seem like this is going to be available for folks to actually try out. So maybe you can, before we dive into any of that, talk about how this is available through an app and why you decided to go through this route of having it more like a performance rather than an app that you download.
[00:10:16.299] David Rosenberg: So, well, because our background has always been in events and shows, in live events and location-based experiences, the live event is very important to us. That's always been at the heart of everything we do. And in this current world, especially where there's limitless on-demand content, we really wanted to make this experience feel a bit more like a show so it's something that happens at a certain time and for a couple of important reasons I mean one of them is the feeling that even though you're sat alone at home well, sat with another person at home, that at that exact time, there are multiple other couples in kitchens around the country or the world who are also sat opposite each other in the same configuration of you. So there is a feeling of some kind of communal event that is happening in your space with two people, but also in a much wider sense with other people in other houses. It feels that that's important for us, this feeling of being connected with things that are also happening at the same time outside, especially after six months of being holed up in your own home, seeing the same faces again and again.
[00:11:36.935] Glen Neath: We're also very keen not to go down the downloadable, you know, putting content, our old shows online that people were able to access. We really wanted to respond to the new situation, really. So we wanted to make a show that was very specific to the times. The fact is that we've all been locked down and we really like the idea of using that as part of the material, really.
[00:11:57.287] David Rosenberg: Yeah, and I mean, there's been a lot of scrabbling around over the last few months, especially for people that normally do live work to find ways of presenting that live work in different ways. And it's been great to be able to see live theatre shows that I hadn't had an opportunity to see that were being streamed. But there's always a slight disappointment feeling that, well, this is great to be able to see it, but this was made for a theatre and it's made for people to come together and sit in an auditorium and watch this. And so there was always something slightly diminished about the experience. And so we were quite clear that we didn't just want to put out, we wanted to make something specifically for this current situation that would work, that would only work by being played in people's homes. So a few years ago, with another company, we had developed a phone app called Wiretapper, which was made for hiding performance in public space. So it was an app that allowed a synchronized playback of audio. And that was the basis from which we built the Darkfield radio app, using some of that technology and adding some other functionality to it. So it's quite fortunate, otherwise it might have taken a bit longer for us to get it together. We had the basis of this phone up already prepared. Yes, and we released it in, when did we release it in the UK? It was in mid-July. Yeah, mid-July. But we're very excited that we're now not only presenting in Venice now, but we've also just released this show in the US as well. So it's now available for everyone in the US, which is very exciting.
[00:13:41.255] Kent Bye: So for the last number of years, I've been going to a lot of these different festivals and seeing live performances. And the question that I always ask myself is like, what is the liveness of this live moment? Could this have been pre-recorded? And if it is, a live performance. And I guess there's like, as I was listening to this experience, a big first question in my mind was, what is the liveness of this live moment? And then as soon as I realized it was more highly produced and pre-recorded, then I was like, well, why did it need to be live? And how would it be different if it actually was live? And I don't know if you've thought about that, if you're able to do the same type of performance that you've done here with the microphones and this dummy head, but maybe you're able to actually live stream it out and maybe be a little bit more like a theater performance in the sense like it's maybe you got some more errors or you kind of get a feeling of like it is actually happening right now at this moment, maybe responding to what's happening in the news or, you know, just to sort of ground people in some way. I don't know if you've thought about that in terms of the liveness of that live performance, if there's even a way to do it live or what the trade-offs are between doing the more pre-reported route?
[00:14:50.700] David Rosenberg: So something that we are developing for the next phase of Darkfield Radio will be is adding live microphones to the experience and then spatializing the audio from those microphones. And with that, also adding a certain amount of speech recognition, limited speech recognition, but in order for there to be more interactivity. So that is one answer to the liveness. It's a slightly different answer but it's about then you could get four friends that are in different places and they would be able to have this feeling that they are coming together in a sound environment but they can hear themselves and their friends placed within that environment and they'll be able to hear each other speak. And then we will probably play with turning those microphones on and off at certain points and having other people that appear to be audience members that are probably not audience members. But we're still probably going to be pre-recording this as well. So I think so much of what we've done has been about something that is absolutely not live, a pre-recorded piece of audio, but then trying to do every other thing in order for that to feel for the audience like a live experience. So it's all smoke and mirrors in one sense, and I hope people don't feel cheated, but that's part of the fun of this, because it does feel like it's like something that's happening. It's happening at the moment,
[00:16:18.190] Glen Neath: Yeah, it's important to say they are very communal experiences, particularly the location-based shows that we've made. They all very much can't be done apart from with a group of other people, obviously because we're playing with the idea of the other audience in the container with you. The liveness, I think, comes from the fact that it feels real, and that makes it feel very live. You know, even though you're sitting in the dark, you're mute, you're unable to move, and you're surrounded by a recording, it still feels as if it's happening around you, and I think that's what makes it feel very live.
[00:16:47.394] David Rosenberg: But there's, so I think this year's Venice, one of the other performances is the alien, is it called Alien Extraction?
[00:16:54.580] Kent Bye: It's a meta movie, The Alien Rescue.
[00:16:56.841] David Rosenberg: Alien Rescue, yes. So in that experience, you do have live actors, and so you can talk directly to those live actors. So I think that's very interesting. And I think, and that isn't something that we've done before, but I think it is, it's something we would be interested in. You do get a lot more technical issues with it, or certainly you open yourself up to quite a lot of more technical issues. And also, it's incredibly labor intensive. We're quite interested in the scalability of our work. And as soon as you start needing teams of live actors that are present at every show, then you're talking about quite a different project. Ticket prices will be going up if that was the case. But it's something that's interesting. And still, while speech recognition is still is, I mean, it's in its infancy, I suppose, really, in the way that it can be used in a very immediate conversational context. So while that is developing, then live performance still does have a very strong utility, but it's not something that we've done ourself, or not for a long time. When we started doing the show, when we made our first show that Glenn was saying, Ring, in the theatre, that had a live performer that started off the experience together with the audience. And then that live performer, as the lights went out, was replaced by recording of himself, which he couldn't really tell. And so it was just really to add another convincing element to the people in the room that that person was still walking around and then approaching you and whispering in your ear. So obviously that couldn't have happened to everyone because they chose you, but all of the audience members felt like they were chosen by this performer. But that performer had did five minutes at the beginning of the show and then just lay on his back for the rest of it. in the dark. Best job ever. I had a kit.
[00:18:58.188] Kent Bye: Well, when I went through this experience of Double, I was definitely tricked in the sense of not knowing what was real and not real. And for me, as somebody who's been in the immersive VR space for the last six and a half years or so, I've seen a number of different projects and a lot of the stuff that tries to virtually recreate sound there's a huge difference between what you're able to achieve with the analog live recording versus what you can get from, say, a recreation within Unity or Unreal Engine with synthesized reflections and spatialization. And, you know, there's something about the bar of what you're setting is actually pretty interesting to see, like how authentic it can sound versus something that is that synthetic recreation. And what I've noticed within the VR industry and what I've heard from different neuroscientists and other folks that have looked at it is like roughly maybe 70% of our perceptual input is like completely dominated by our eyes and our visual input. And so within the immersive industry, there's been so much focus on visual input that most of the effort, and in fact, the GPUs and CPUs, the budgets that are dedicated towards that reflect that. In fact, it's probably more closer to 90 to 95% of the resources that are available are going towards generating that visual aspect because it is so dominant. But also if you look at what's been happening with the immersive industry, you've had a number of false starts when it comes to things like either the AUSIC headphones that were trying to create, you know, really high fidelity, specialized sound and ways of dynamically measuring your HRTFs and your ears to be able to customize the sound to be then have higher specialization, or you have the Bose AR frames, which as a piece of technology were great, but yet, I think in terms of what the audio experiences were, I don't think there was necessarily any really compelling narrative experiences that were really driving people to get that technology because you're already immersed in the world and it's like more of overlaying layers of audio on top of that. in my content perspective, we're still at the nascent stages of people where I think they have to have a direct experience of what's possible. And then from there, see those narrative experiences that you're creating with Darkfield, feel like bars of those experiences of like, okay, this is what's possible with this, to then maybe catalyze other technological platforms. Because you're essentially just using a phone and headphones. You don't need anything else and that for me it was amazing to have that high level of an experience without having to know what my head pose was and not even need to look around or to move my head around to just passively receive it with my head looking forward was enough rather than trying to make it feel like I was in a space with enough agency to kind of like even move my head around. But it feels like interesting just to see how, what you're creating with the narrative experience to be able to potentially in the future catalyze more technological innovation down the road.
[00:21:56.283] David Rosenberg: So we do have an R&D project, which is using, I mean, it was actually initially using the Bose gear before they were discontinued. And also the latency was a little bit of a problem with the frames anyway. So the project, it would be for another location-based experience, but for a free-roaming, in-the-dark experience. So the audience will be in a space and will navigate that space much like a free-roaming VR experience, but in the dark, but being tracked and with spatialized audio. And we would still use some binaural recordings within that. But there, there would be the utility for also having a spatialized audio in order for an audience member to be able to actually find their way around based on where sounds were located within a space. So we do want to continue with exploring how the new technology for tracking audiences and for spatialization algorithms can help us tell other stories. But these are just an R&D project so far, and we're not 100% sure whether we will get somewhere that we have made the most of that technology. We don't know yet. But we hope to. I mean, I think as soon as you have a mobile audience, then the need for real-time spatialization becomes more important.
[00:23:15.441] Kent Bye: Glenn, did you have anything you wanted to add to that?
[00:23:18.061] Glen Neath: Well, I think that we do sort of ask the audience to believe something. I think something that we, the very first thing that we did with Ring, that David suggested, that the first thing that should happen when the lights went out was everybody in the audience should move their chairs into a circle. I was a bit like, nobody's going to believe that. That's a kind of ridiculous first thing to ask people to believe. But people did. You know, I was sitting in the audience one night and a woman called out, but I can't move. I told the box office, I've hurt my leg. And the actor went over and said, you can stay where you are. And I think it was something to do with, for me, was that whatever we ask the audience to believe, it's kind of unlikely, but it's possible. So for me, there's always something about the idea that whatever we're asking the audience, it's unlikely, but it's possible. I'm not quite sure where I was thinking that at the very beginning, before David started speaking, I'm not quite sure how relevant that is.
[00:24:04.196] David Rosenberg: And it's also about, I mean we think of immersion very much about leaving gaps for an audience. Immersion is about the imagination, well that's the most important tool for immersing the audience. So we leave out the image mainly, so that's quite a big gap, but also the way that we construct narratives as well is also it's important that there is space within the narrative for the audience to fit themselves in. Sometimes people have complained in the past that our narratives are, you know, why can't we just tell a more fulfilling story, like where something happens to some people. But we can't tell those stories if we're trying to create a place for the audience within that. Otherwise, they're just sitting there listening to some other story happen in their space. But we're always trying to make the audience a protagonist, give them the sense that the story is evolving around them, and their presence is one of the key things that is a catalyst for why the story happens.
[00:25:11.754] Glen Neath: And I suppose it's keeping them in the moment so that they're constantly feeling that they're present in this story as it's going on, rather than sitting outside of the story, looking at it as you might traditionally do in a theater, a procedure march theater, where you're watching a story in front of you. So this idea of putting them in the story sometimes feels as if, well, you know, there is no traditional relationships going on. There is no traditional story that you're supposed to follow because by doing that, it feels like a different part of your brain. And I think it's quite key for our things is to keep the audience member in the moment, sort of always not sure what's going on.
[00:25:45.310] David Rosenberg: I mean, I think there's another thing about interaction. Interaction is, there's always a tension between interaction and narrative. And so even if it's as simple as the interaction of turning your head to see something as opposed to it being just presented for you, even with that, there is a tension there where you might lose some narrative for the benefit of some interaction. This is always a thing that we're juggling around and I'm sure with every VR storyteller that's exactly the same things that they're juggling the whole time is how to create a feeling of agency for an audience that they can look and do and feel active but yet not to the detriment of the story that they're trying to tell. And it's a tightrope to manage those things.
[00:26:38.279] Kent Bye: So here's part of my own experience of experiencing double. And again, if you haven't listened to this experience yet, go listen to it before you might have already spoiled a lot of stuff about your experience. But I previously have done a lot of experiences that were with two people. And sometimes I've had those experiences where we listen to the same thing and other times we hear different things. So I was sitting there with my wife and I didn't know if she was listening to the same thing or not. And there's instructions at the beginning to not open your eyes. And it was such a convincing sound that I heard that I was like, wait, is she moving around? And I peeked and I looked and I opened my eyes. And in hindsight, that sort of robbed myself of that angst and tension because the illusion at that point was broken. The plausibility of it was like, okay, there's this really good audio and I'm just not going to know what's real and what's not. But as kind of like a VR geek, I was like, wait, am I being tricked? And I was like, yes, I'm being tricked. And oh, by the way, the house of cards have fallen now because you've broken your presence. There's this tension there of like wanting to know and so really surrendering to that suspension of not knowing. And unfortunately I ruined that for myself at the very beginning. But as creators, how do you convey that? Because you're like, you want to make sure that people have the best experience, but you also want to make sure that, you know, if people have actually had trauma regarding having their boundaries transgressed in some ways, it may be, a whole other experience for some people that they actually, in order to really experience this, they may need to do that. But yet when they do open their eyes, they may ruin the rest of the experience. And so how do you navigate that tension as creators?
[00:28:27.883] David Rosenberg: Well, up to now, because our main work has been location-based experiences, we have been able to control the environment completely. So we turn the lights out, and then there is nothing that anyone can do to spoil the experience for themselves other than attempt to leave, which occasionally does happen, of course.
[00:28:51.049] Glen Neath: Just to say, we do give people ample warning of that we're going to turn the lights out. It's not something that we just inflict on people.
[00:28:56.728] David Rosenberg: Yeah. And of course, there's a whole raft of different safety measure that need to go into a performance where people are in a completely dark shipping container. So people know what they're getting into when they go in. But yet still, if people have never sat through one of our shows before, there are people that do get quite anxious. So we have had some instances where people have had to leave because they can't actually handle it. So then when we started thinking about making shows for people, we knew that we weren't going to be able to have complete control. And with me and Glenn, we had this continual argument for weeks before where I was saying, sure, they'll close their eyes. You know, they'll definitely close their eyes. And Glenn was like, no, they won't. There's no way they'll close their eyes.
[00:29:43.059] Glen Neath: It wasn't that I didn't think they wouldn't close their eyes, I was just wasn't sure they would keep them closed. So, you know, that was something we spoke about a lot. And it was about, you know, we tried to navigate that in the narrative. We didn't want it to be a sort of a general housekeeping, close your eyes and keep your eyes closed. We wanted it to be embedded within the story. And, you know, you closed your eyes in order to further the narrative, really. So we wanted it to be there for a reason. So, you know, ultimately we can't stop you opening your eyes and there's an element of trust. As you said, the experience is better if you keep them closed.
[00:30:14.377] David Rosenberg: And we're now currently working on two more shows for the Darkfield Radio platform. And the setup will be quite different in that it's still for two people at home, but at a certain point you will be separated. Basically what I'm saying, the closing the icing isn't going to be such a big deal. In that one, it won't matter so much. And then in the third one, you're going to be in bed with the duvet of your head. So that will solve. Maybe.
[00:30:43.378] Glen Neath: I might start some other troubles.
[00:30:47.779] Kent Bye: Well, it seems to me that there's certain genres that work well with this type of audio experience. Like this seemed to be a bit of a horror or thriller. Like what kind of genre would you characterize this as, as you talk about the types of narratives that you're telling?
[00:31:03.659] David Rosenberg: When we first started making work for audiences in the dark, we weren't thinking about the experiences being frightening. And we made our first two shows for theatres, and the only thing that the audience spoke about was how frightening they were. So then when we started to do the shows in the shipping containers, we basically tried to make them frightening. And the four shows that we've made in shipping containers up to now, they're all exploring some fear and anxiety in some way. So we kind of really embraced that. And I think that that is a good genre for us. And up to now, we just haven't looked at anything else. But we might do. We might still try and make something a bit nicer. But yeah, I think we've become more and more interested in what unnerves people, what puts people on edge. And we've found a lot of thematic content that ties into that, that is interesting for us. And I think ultimately, also, all of the shows are reflections of consciousness as well. And horror is just quite a good genre to explore that, whether it's ideas the black mirror type ideas of more futuristic horror or what or whether it's more gothic horror a lot of it is all about existential concerns and fear of death and meaninglessness and loneliness and and these are all things that work very well for us and it allows us to explore horror but also have other layers of meaning in the shows as well that are real that are people's real concerns as opposed to You know, there's the people who are frightened of ghosts, but really they're frightened of death. And we can talk about both.
[00:32:46.814] Kent Bye: Is there any other specialized audio experiences that you've taken a lot of inspiration from? Like maybe some landmark pieces that folks should be aware of that you've either personally experienced or that people could go out and experience themselves?
[00:33:00.569] David Rosenberg: Well, for me, my father was a neuroscientist working in hearing. And so he used to make binaural recordings in the early 80s with this guy, Hugo Zuccarelli, who was this kind of charlatan that had a false paper printed in the Lancet, a British medical journal, about some new way that he discovered of hearing, but it was all made up. my dad got totally fooled by and so did everyone else. So anyway, so I've had a lot of experiences since being a kid of binaural sound work. I think the first artist who made a piece of work with using binaural sound that I experienced was Janet Cardiff. who's a Canadian artist, and she's a fine artist. And she made a piece with Artangel, which was at the end of the 90s, which was a walk in Whitechapel in London, where she has sort of accompanied you on a walk. You've got a cassette recorder from the library, and it was all on that. And subsequently, she's made quite a lot of work, audio work with Binaural Sound. So I would definitely check out her work.
[00:34:10.080] Glen Neath: She did a great show for a cinema as well, I think, didn't she? And also the Circle of Speakers, which was the orchestral piece as well, wasn't it? Yeah.
[00:34:19.747] Kent Bye: There's a video on YouTube that I saw, there was a performer, a man that was doing some sort of live performance with binary audio as well. I don't know if you had a chance to see that or familiar with that work. It was a performance and the audience had headphones on, and then he was doing live recordings, but also putting out pre-recorded stuff. And so it was sort of like confusing to know like what, again, that's sort of what was real and what wasn't.
[00:34:42.377] David Rosenberg: That might've been Simon McBurney, The Encounter.
[00:34:46.379] Kent Bye: Yeah, I think that was it.
[00:34:47.099] David Rosenberg: Yeah. Yeah.
[00:34:49.832] Kent Bye: Did you get a chance to see that?
[00:34:50.833] David Rosenberg: Actually, that was one of the things that I saw when it was streamed during lockdown. So unfortunately, I saw it as a diminished experience, sadly. But we know the sound design of that work is Gareth Fry, his colleague and his fantastic sound design.
[00:35:09.347] Kent Bye: Well, what are some of the types of experiences that you each want to have either immersive experiences or when it comes to immersive sound?
[00:35:17.135] Glen Neath: I mean, one of the things that changed the way that I started to think about making work was David's previous company, Shunt, who I came across 20 years ago. And, you know, prior to that, I'd been watching traditional theatre shows in theatres, you know, with written scripts. I'd written a couple of plays that I've got readings that they're all called, but I've never managed to get further than that. I never managed to get anything produced in a big house. And then I came across David's company, Shunt, and they were just making these shows. in strange spaces underneath London Bridge Station, underneath a railway line in Bessemer Green. And they were just like nothing I'd ever come across before, where as an audience member, you were basically a part of the show, you know, and there were sections where the audience had to do things in order to move the show along. And that changed the way that I started to think about making work. And I also came across a guy called Anthony Hampton, who makes work using recordings and unrehearsed actors. And I did a couple of projects with him. So I've been quite lucky to work with the people that I'm influenced by.
[00:36:13.982] David Rosenberg: I mean, VR is something that we've been interested as well in over the last few years. And we've had a project that we've been developing with Nexus Studio, which is based where they're in LA and London. So they're based in both those locations. And it's still in development, trying to get funding, you know, that story. But that would be for a location-based VR experience. And what's interesting for us about working with VR is that we would still be maybe 90% sound. to try and use VR in a similar way to how we use sound in the location that the base shows, where it could be things that are happening. It's just image to augment the sound, as opposed to the other way around. So we're very keen on trying to develop this project further. We've had a bit of a hiatus because of COVID lockdown in terms of moving that forward. But that is something that we're very interested in, especially as the technology in this area is just getting better and better now and much more affordable for use for location-based work.
[00:37:22.193] Kent Bye: Great. And finally, what do you each think is the ultimate potential of immersive audio and immersive experiences and what it might be able to enable?
[00:37:33.725] David Rosenberg: I'm leaving that one for you, Glenn. Oh, great.
[00:37:36.111] Glen Neath: Fabulous. Wow. Okay. Well, technology is not my strength. I think the idea that we... Well, I think maybe that's the key thing.
[00:37:46.516] David Rosenberg: I mean, I think that with our projects, it's always been about the content first or a particular narrative or a particular configuration that we want the audience to have with each other. And then the technology has been a way to realize that. I mean, we've tried for most of the work, we have worked in that direction and not, well, there's this really cool feature, this really cool technology. What story can we tell with that? we've generally worked the other way around and we still get excited about new technology as well. Sorry, Glenn, you were actually just on a roll and then I interrupted you just as soon as you began.
[00:38:23.209] Glen Neath: I'm sorry. That's the right word. I'm very excited about this idea of being able to, you know, this project that we've got some VR money for, to try and make this idea of a virtual space that we are able to put audience members into from different places.
[00:38:37.479] David Rosenberg: This isn't VR, so this is the live microphone?
[00:38:40.461] Glen Neath: Yeah. So yes, I'm quite excited about the idea of that by creating some sort of narrative that, again, it's always about trying to make the audience feel as if they're more involved. I mean, that's kind of been the drive for everything that we do. So it's every single step we take with every little bit of new technology is always about making it feel as if they're being listened to. You know, we've just shown, later show made in Shipping Container Eulogy uses voice recognition and the audience in that are asked questions. But it's not necessarily about branch narratives. It's always about trying to make the audience member feel as if they've been heard. and consequently making them feel more immersed.
[00:39:15.217] David Rosenberg: and also making them take some moral responsibility for what might happen in a narrative. I mean, there's been countless shows where the audience are kind of blamed for not doing something, even though there was no way that they could ever really do something. And yet you still feel that you've got the right to kind of make the audience feel shit for like not having done something that they were never able to do. So that's not really fair. And I'm sure we've done those shows as well. But now with this latest show, Eulogy, the audience do have a few opportunities to either say or not say things that mean that it would be fair for us to say, hey, this happened kind of because of you. at that point. The show will end in a completely different way. It's just about finding points that, as Glenn says, the audience feel that they've been acknowledged and they've had an opportunity to say or not say something or take the blame or not take the blame for something. And that's included in their overall experience.
[00:40:21.659] Kent Bye: Great. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:40:28.587] David Rosenberg: Well, please download the Darkfield Radio app, where you have links to the website. So it's darkfield.org, and there they'll be under Darkfield Radio. If you're in the US, then you'll be able to now see how to get access to the shows. And yeah, we'll be making our next show on the platform towards the end of October. We're making it now, it'll be released towards the end of October. And then there'll be another one actually shortly following after that. It would be great if people could experience it.
[00:41:02.015] Kent Bye: Any last thoughts, Glenn?
[00:41:03.676] Glen Neath: I concur with all of that.
[00:41:06.038] Kent Bye: Awesome. Well, thank you so much both for joining me today. I had a great time in this experience. And like I said, I think it's setting the bar when it comes to the type of immersive audio and what's possible there. For me, I personally look forward to, in the future, adding more interactivity, specialization, blurring these two together. Yeah, maybe even eventually having real time audio engines, just like there's physics engines, having audio engines that do the same thing. And I feel like as we move forward, hopefully we'll see more focus on the power of audio. And I think this is a good experience for people to, to have just to see what's possible with the technology that's out there. So, yeah. But thanks again for joining me on the podcast.
[00:41:42.079] David Rosenberg: Thanks so much, Ken. It's been a blast. Thank you.
[00:41:45.301] Glen Neath: Keep your eyes closed.
[00:41:50.464] Kent Bye: Yes, I need to be more compliant. So that was David Rosenberg. He's the artistic director and one half of the creative direction of Darkfield radio, as well as Glenn Neath. He's the artistic director and the other half of the creative team. So I have a number of different takeaways about this interview is that first of all, well, this was such an amazing experience because it really tricked me. I could not tell the difference between what was recorded and what was real. And as soon as I peeked and I looked, then that illusion was broken. Then I started to. not be able to suspend my full disbelief. So originally they started with these location based experiences. They started with the ring where they had an environment and a context where the audio recordings did not happen in the same environment as the space that they had. And so they eventually went into these shipping containers where they could completely control what the sound profile was and to be able to record it and use different shipping containers to go around to different places. But they could also control the lights and the external conditions. And so This is a type of experience where if it's a location-based experience, then they can control the environment, they can add additional cues to really blur this line between what's real and what's not real. When you're at home, you have to really rely upon the audience to not open their eyes and not to peek. And, you know, for me, it was so convincing that I sort of ruined the illusion for myself, but still it was super compelling just to see the degree to which the analog version of binaural audio using this Neumann KU100 dummy head, this mannequin that has these binaural microphones within the ears to recreate the sound profile. And we talk a lot about spatialized audio in VR in the context of Unity and Unreal Engine, you know, these synthesized, spatialized audio solutions. But when it's actual audio recordings and you close your eyes and you're fully immersed in this world, then it really does become difficult to know what is happening within the context of what you're hearing and then what's happening in the context of the world around you. So this whole liveness thing, I think was interesting because I think I understand a little bit more because I said it was a live performance and it's technically all prerecorded, but I think that's part of the conceit that you are going to potentially be attending something that is live. And then you're, you're trying to figure out, okay, what is the liveness of this live? And you soon realized that it's sort of a smoke and mirrors way to psychologically trick you into believing that something is happening live and in real time. Now, when you do it with some other person, there's an ambiguity as to whether or not that other person is the person that is making those noises, because you don't know at the outset whether or not you're listening to the same audio or not. And so as you go through these experiences, then you don't know if something's been pre-recorded or something that the other person is doing with making the noises. And that liveness of live comes in not knowing what is real and what's recorded. I think that's actually quite interesting to see how they're playing with the liveness by tricking you and thinking it's live, but it's actually not. But you still have to have the live show. There is a state of mind that you get into when you're like, okay, I have to go to this show at this specific time, then you have to put into your schedule, your calendar, you sort of get this anticipation, and then you have this countdown, and then you play it, and then you get into the show. And I think that experience, I think, is actually unique. It makes it feel like, as we're all locked up in quarantine, you do have this opportunity to kind of go out and experience this performance in a theatrical setting, even though you are at your home and you're in your kitchen and whatnot. So the whole thrust of a lot of the work from Darkfield Radio is to blur the lines of what's real and what's not real and creating doubt whether or not you're experiencing something in real time or something that's imagined or pre-recorded. And that typically when we think about immersive media, you focus so much on the visuals because the image and visual processing is just so dominant with our perception. He was talking about how if we see lips moving, then we hear the audio that's correlated. Then we anchor things onto the visuals first because the speed of light is faster. And there's a lot of reasons why that's developed like that over time. But most of the virtual reality has 90 to 95% of all the. GPU and CPU processing power is going into that visual component and the audio is something that just I think is chronically underappreciated and I hope that Folks get to see experiences like this just to open up their mind as to what the narrative possibilities with this analog technology And I think that the comment that David made around horror and what appeals people to horror is that, you know, we're already in a year from hell with 2020, the pandemic, everything that's happening in the world is just so much stuff that's already this sense of overwhelm. Like why would anyone want to go into an experience that was like adding even more anxiety? And I think there's a certain level in which the horror genre is great at actually creating a container to allow you to have some level of catharsis, to really dig deep into that fear. And he says it's not just like the fear of ghosts or whatnot, but it's also like your fear of dying and touching into this existential fear of your own mortality, but also of being alone and the sense of loneliness. So there's other aspects that you can really use genres like horror to be able to explore different dimensions of your own relationship to death and mortality. So I thought that was actually quite interesting. And I think eventually they're going to start to expand what that liveness of the live is by having you be able to speak in and to provide the audience an opportunity to make an actual choice and to be faced with some sort of moral dilemma and to see what people choose to do and adding different layers of speech recognition and be able to hear what people say and then have that be fed into the narrative later on. I think that's the direction that they're going to start to try to experiment with. So the final thing is just talking about immersion and presence within these audio experiences, because with audio, you're not seeing all the sensory input. So you have to kind of imagine this world that you're listening to. And so David was talking about how the level of immersion that you get from closing your eyes and how immersion can come with your imagination that's filling in the gaps of information that you don't quite have. And so there's a certain level of you projecting yourself into the experience. When you hear sounds from the jungle, you can really be transported into the jungle. And so you're able to create this sonic landscape that's really able to be pretty transportive. And I think a lot of people, when they go into virtual experiences, the visual field does dominate so much, but we can't underestimate the impact of a really good sound design to give you that extra layer of immersion. you know, eventually, once we see what's possible with the analog version here, so using something that is recording pre-recorded sounds, and then you move into the game engines and be able to do dynamic sound generations with reflections and, you know, eventually being able to model the physics of how sound propagates in different environments and to be able to recreate that in real time with the PlayStation 5 to start to have different audio-specific hardware to be able to do these different types of spatialized audio in these immersive video game experiences. So expect that, you know, eventually we're going to take the physics engine that we have to do the visual component, but create an audio physics engine to be able to take those spaces that are created, but also to be able to replicate sound that sounds like you would hear when you're in these places. And I think the sound is going to be a lot of innovation that happens over the next five to 10 years to see what you can do already with the binaural audio recordings and how so much open-ended innovation is yet to be done when it comes to trying to recreate that type of Sonic landscapes that sound real that really transport you into these different locations in these different places And I think for anybody that's working in the space just to see what's possible I highly recommend checking out what the folks over at darkfield radio are doing there and listen to some of their experiences and Don't be like me and just keep your eyes shut So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of ER podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a lesser-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voices of er. Thanks for listening.
