I interviewed A Vocal Landscape co-directors Omid Zarei and Anne Jeppesen at Venice Immersive 2023. See more context in the rough transcript below.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at immersive storytelling, experiential design, and the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So, continuing on my series of looking at Venice Immersive 2023, this is episode 7 out of 35, and the second out of 3 of looking at the context of communication. So today's piece is called A Vocal Landscape by Eni Yippison and Omid Zarai. So this is a piece that is really trying to explore the process of authentic communication. They did a lot of experimentations for trying to capture the voice in the most authentic way. They captured a series of different long conversations and then picked one that they really liked and started to build an experience around it, trying to imagine different visualizations of different aspects of communication, but also in the context of someone who is in some sort of relationship with each other. So some sort of partner or couple or just friends who are going on these different explorations of these imaginal ideas, and you get to see how these ideas are being manifested through some sort of spatial context of surrealism and dream logic, And the center of gravity of this piece is really into this sense of environmental presence and the ways that they're using the environment to try to explore different aspects of communication. But another aspect is the mental presence of trying to trace the different ideas and concepts that are being explored, but also different ways of abstractly representing different moments of these conversations with each other. So that's what we're covering on today's episode of the Voices of Euron podcast. So this interview with Annie and Oben happened on Saturday, September 2nd, 2023 at the Venice Immersive in Venice, Italy. So with that, let's go ahead and dive right in.
[00:01:59.592] Anne Jeppesen: My name is Annie Jebbesen, and I'm one half of Superposition XR Studio, which is based in Copenhagen. I have a background in sound and music. So that's also been my way in. And on this project, I'm also mainly in charge of the sound part.
[00:02:14.754] Omid Zarei: Yeah. My name is Omid, Omid Zaray. And I'm the other half of Superposition and this project. I'm the developer and also co-director. I do extra producing and also work in cinema, film, virtual production, everything related to Game Engine. But also, I come from background of cinema. So that's really my home.
[00:02:34.122] Kent Bye: Great. I'd love to hear a bit more context as to each of your background and your journey into VR.
[00:02:39.270] Anne Jeppesen: Oh yeah, and maybe you should start because I pretty much got into VR because of you. So maybe you start.
[00:02:45.475] Omid Zarei: So I started in VR when I lived in Paris. I was in film school in Paris in 2015 and the Oculus came out, you know, the DK1 and all that stuff and then I got interested in like doing films, doing like storytelling in VR and at that time was pretty difficult to find people to work with and I figured out the best way to do it is to create a workshop to bring other like-minded people to create it with and like create or like figure out a language. So my first step to VR was I created a workshop in Helsinki in 2015 which was called VR Workshop Helsinki and we basically like funded it by state funds and university funds and invited all sort of people almost for free to come and just play around and there was like screenwriters, there was musicians, there was all sort of disciplines. to do hands-on hackathon, let's say, of VR. And that was the beginning, and then it just kept going. And then projects after projects, and then eventually, actually, again with another project for the Finnish opera, is when we started working together, bringing Anna in as the sound production and the sound brain behind the production.
[00:03:50.797] Anne Jeppesen: Yeah, and then we became a duo, basically, after that.
[00:03:54.517] Kent Bye: Yeah, I'd love to hear a little bit more of your background and your journey into this space.
[00:03:58.117] Anne Jeppesen: Well, my background is in sound storytelling, which I also consider a very immersive medium. I do sound installations and I do sound experience of different sort, like location-based sound experience. So when I heard you talking about your work, I thought it was an interesting way to expand and to create immersive worlds. with more aspects, but for me, I always work in immersive because I work in sound. So, yeah. I still work a lot of times mostly in sound. But, yeah.
[00:04:30.829] Kent Bye: And how did the VR come into play then?
[00:04:33.030] Anne Jeppesen: Well, it was... Umid asked me to join this Finnish opera development project that we had. It was a call-out they had in 2019, yeah. We created this project called Songs for Future Past and then later we ganged together and did this duo. We were trying to exactly bring the sound aspect, the sound immersion together with his cinema view on things and to create some immersive experimental work.
[00:05:02.643] Kent Bye: I remember being at Laval Virtual in 2019 and meeting someone from the Finnish opera that they were trying to blend opera and VR. Did you submit a proposal or did you actually end up working on the project that got commissioned by them? And maybe give a bit more context to that.
[00:05:17.263] Omid Zarei: Yeah, it was the Finnish opera decided to create like a future expansion of like what opera would be and all that stuff and then they had this open call for like anyone with a project and it could be installation it could be any sort of XR and we were one of the seven groups if I'm not mistaken who were like finalists to take our pitch and pitch it in front of the audience of the opera and see if we could get it funded and also like accepted by the opera to create like a next project for the opera to do something in XR So we were like the finalist group in 2019. We didn't eventually get the project. It was a Finnish group who got the project and for good because COVID happened and all that stuff and everything shut down. But that was the start of actually creating a crew, creating like an ensemble of creators in XR. We were five people back then, people from Copenhagen, from Denmark. And me and Anna, we kept working on other ideas, which eventually led to this that is here now.
[00:06:10.802] Kent Bye: Yeah, so a vocal landscape that's showing here at Venice Immersive 2023. And in terms of genre, it's hard for me to pinpoint as to where it fits in. It feels like kind of a uniquely VR piece of a couple of folks are hanging out, talking about stuff, drinking some wine, have some stories, having some spatial visualizations of that, some altered states of consciousness or liminal spaces. How do you start to describe what this project is?
[00:06:35.773] Anne Jeppesen: Well, it initiated actually with, we had a little audio recording of an actual conversation. And we went from there, we thought it had a certain vibe to it that we actually found hard to describe. I guess we still do, but we wanted to explore what is that vibe and how can we put it out in a spatial setting. But as we dived into it, we figured more and more that it's a lot about the fact that when you sit and you have a certain conversation, you try to grab out for a concept. You have two different minds trying to maybe grab at the same thing, but who knows where the other mind is actually. So it's like that funny condition of human communication that we were interested in exploring with this piece.
[00:07:20.782] Omid Zarei: because Anna has worked a lot with voice forever and she knows how to deal with that kind of thing and that was like inherently a kind of conversational like this communication and we somehow figured out pretty early on that there's a non-verbal disconnect when two people can't communicate with each other with languages I mean it's non-verbal it's It's hidden body language and stuff like that and we thought like that's really something that could work in VR. If you actually take those like wrong paths or U-turns that people take in a conversation and like keep going with that wrong path and illustrate it and like make it animated and make people follow that story. And that was okay this is really really VR thing because we wrote scripts over and over it makes some sort of sense on paper. But when we tried it in VR, we just took a lot of those filmic things out more and more and just we cut a couple of scenes out because it's like, OK, it's about really that nonverbal body language that just makes sense in VR. And then we thought, OK, maybe we have something to work with and we should just follow that clue and go with the medium, see where it takes us. And we end up here.
[00:08:23.956] Kent Bye: It feels a very unique VR piece, like I can't necessarily imagine this in a film, maybe a podcast because there's an audio element to it, but maybe you could give a bit more context for how this recording of this conversation came about. What was the context for why this was recorded and who are these people talking?
[00:08:40.972] Anne Jeppesen: You mean the actual content of the story? It's a long story, but I can try to make it short. We actually had this whole method, because we're also acting in the piece, both of us. So we had this whole method of recording hours and hours and hours of conversation. So a lot of hours, like days after days. We went out to this little cabin in the middle of the winter and just recorded hours and hours and hours of conversation. And then eventually we took parts of that and stitched it together to some kind of script, I guess. But to really keep what actually happened in the conversation, to keep it as close to the real conversation as it was. So that's where we try to really keep a real flow of conversation throughout getting anywhere with the content. So yeah, can you elaborate?
[00:09:28.141] Omid Zarei: Yeah and also like all along like with VR, trying it in VR because sure we come from film and she comes from radio and we figured out early on that we have to just throw all of that out the window because it just didn't work as a film piece or even as a conversation piece because then it would have been very long. But what we figured was like that, what makes bodily sense when you try it in VR? What makes you like viscerally feel something and go somewhere? And that was basically the compass for us to follow that. And the conversation also like, we took bits and pieces and like created a structure and then acted it out. So we acted the video or the volumetric recording on top of the conversation. Then we cut some of that out again, and then we dubbed it again, and then we cut some of that out again. So it was basically a reductive process of going from something very big and then taking what's not necessary. and what actually works in VR, and then see what we're left with. And in the end, when you see people trying it, they always pay attention to things that you didn't imagine. And it's like, OK, maybe it's always been about this negative space that we were trying to open up, and not actually we were putting in. Because I remember I showed it to someone personally, and he said, I just feel like in the beginning, I'm feeling a little bit wet at the ocean. and that's just the opening and I was like yeah it's about that you know it's about that feeling of being there intimate with people and like following whatever you follow in a conversation yeah.
[00:10:48.904] Kent Bye: So it sounds like you recorded a bunch of audio and then you then edited it down to some extent and then put volumetric video and then that was another iterative process of Editing it down and then when the final piece here Is it dubbed or is it you acting out live this script that you made based upon the original conversation?
[00:11:09.259] Anne Jeppesen: It's dubbed, but it was a long process of also dubbing it to have it come out real. What would you even call that? But we tried to really act it out from a proper emotional state, you can say. But it is in fact dubbed, but we tried to avoid having it to be something you think about when you see the piece. Yeah.
[00:11:33.365] Omid Zarei: Also you could maybe talk about the process because like the very beginning of the piece when we were here in college cinema in 2020 because this all started with college cinema vr in venice the main concept was like the hidden nuances of voice you know like i'm talking to you maybe i have a bit of hesitation maybe i've sub layers of voice and there was a lot of research we did about how to record the voice what kind of microphone where do you put the microphone and do people act it do people reenact it and research about the method of recording we use contact mics to try to record like the chest voice then we put like lavaliers on the face on the chin and then in the end Anna came up with the idea of like putting harmonica recorders
[00:12:13.200] Anne Jeppesen: Mouth harmonica holders. I'm a voice recording nerd, because I worked in this field for many years. I have this little Roland R-05 that has this incredible sound. It's not in production anymore, unfortunately. We tried all of these different, really expensive, clean-sounding lavs, but then it has this certain gritty kind of sound quality in it. And then we mount that to the harmonica holder. It's like this tiny square thing. So we mounted to this little tripod and then put that on the harmonica holder and then we had this dubbing system. So that's what we recorded all the conversations with. And that's also what we ended up dubbing with. So yeah, it's a lot of funny methods, but we went for a very specific expression where you can really feel the nuance in the voice, the crispiness, the subtle emotional messages. Because I feel like when you hear a voice speaking freely, There's so much going on in that voice. You can tell the current state of the human talking. You can tell there's some history there. You can tell, of course, age and different emotional states playing on top of each other. There's so much information, actually, in a free-speaking voice. The whole idea of this was actually also very much rooted in that thing that the vocal expression is so complex and so rich. And when you work solely with audio, that's basically the main expression, like in audio storytelling, for me at least, in audio documentary. So it was also interesting to how to expand on that in an immersive medium like VR.
[00:13:51.962] Kent Bye: So when you went to the Biennale at college in 2020, had you already recorded this conversation that's in the final piece or did you pitch it and then go off and record it with some intention out of this exploration of recording authentic conversation voices?
[00:14:07.780] Omid Zarei: So the conversation we had for Biennale was a completely different one. We were in Paris, we were sitting, it was raining, we were like in an attic, and Anna records a lot. This is what she does. And it was an intoxicated conversation, let's say. But there was a lot going on. And what Biennale heard was a minute of that conversation. It didn't have a head or tail or... But I think they maybe saw something that like, okay, this is interesting, how do you turn that to VR? And that pretty much stayed the same for us, like, so we took that conversation, we wrote script upon that, and then I think we did like five, six iterations of script, and then in the end we acted something else, after like seventh script, and then we dubbed on top of that. So it came a long way, but in the end, we just tried to keep the essence of that first conversation. the very primary feeling of trying to understand someone else in a conversational context. And the words change, maybe the story changes, but the essence stays the same.
[00:15:07.012] Kent Bye: So it sounds like there was a very early ephemeral conversation that was sort of unplanned and organic. And then a lot of post-processing of trying to craft it into something that was shorter, that was more deliberate in its crafting. Because conversations can go five hours long sometimes, but you're trying to create a narrative piece that gets to the heart of something. So even when you're going to the very early first BMI college, you had the core essence of some of what you were going to be then after that, doing lots of iterations of trying to craft after that.
[00:15:37.765] Anne Jeppesen: Yeah, absolutely. Yeah, it's all about because you have something that was very precious to us, like a certain very indescribable vibe of something. And then it's really hard to actually create that on purpose. Like, how do you go about that? We went to great extents to actually try to do that. But yeah, it's been aimed at trying to nail that vibe in VR.
[00:15:59.240] Omid Zarei: But also, there's a good word, indescribable. We thought that there's something indescribable in that conversation that only can be done in VR because you can create feelings. I think you can invent feelings. You can invent sensations that you can't really describe it to people. And we try to take that indescribable from the conversation and use the indescribable ability in VR to create sensations and merge them together, make the audience feel like a holistic feeling of the indescribable. Because otherwise, it would be a podcast. But then with VR, you could just elevate it to a feeling that it's full-body experience, and it's not just visual, and it's not just audio. So that was very much the core of it, the indescribable.
[00:16:40.634] Kent Bye: Yeah, I think seeing it, it gave me this sense of like a chill vibe, you're hanging out, something very unique to VR. I don't quite know what to make of it. It's got this kind of liminal quality where it's this in-between space. Then you have all these other VR visualization components and surrealistic, you know, giant pieces of fruit in some of the final scenes. So maybe talk about, okay, you've got the vocal exploration, then now how do you start to do these cut scenes away into doing some sort of poetic interpretation of what people may be visualizing and what you were trying to achieve there with this miscommunication or resonance with trying to have two people understand and communicate with each other.
[00:17:19.332] Anne Jeppesen: I think maybe you can talk more about the visualizations of it, but it's very intuitive, I would say. It's very much about something-something, something gritty, something with a certain tactility. It just came out, honestly. But you can elaborate, maybe.
[00:17:36.031] Omid Zarei: Well, maybe the idea, for me at least, because I was very involved with the visuals, it was like perversion. To not be redundant and show the obvious, but also like, when you talk about softness, you show metal and blades that goes in the hands. When you talk about grandeur, it's just make fun of it, be satiric. Because I think that's also what happens in a conversation if, when it goes sideways, you know, because you just go somewhere else completely. So I tried to stick with something that makes sense, but it's quite perverse, quite on the other side of things. but still makes you understand, makes you feel something. We really wanted to make people feel something, above anything. And maybe not understand the story, maybe not follow a story, but feel something. Come out smiling, come out angry, come out happy. That's about it, I guess.
[00:18:22.916] Kent Bye: Elaborate a little bit on the use of alcohol in this piece because there is like an altered state Quality to it as it progresses. It feels like they're drinking more and more wine And so was that emergent out of the original recording and maybe give a bit more context as to that element of it
[00:18:39.285] Anne Jeppesen: We never say that it's alcohol per se. I can say that there was maybe some alcohol involved in the first recording, so that's maybe some sort of intoxication involved, but that's not really too important for us. We thought of it more as like, also it's a blue bottle and blue glasses and you don't really see what it is necessarily. So to us it's also an elixir. It's like something you bring in, you share something and you like, It's also a way of showing their interaction. It's a subtle way they try to reach each other, share something and get to the same place.
[00:19:13.200] Omid Zarei: It's the juice. Because it was actually the beginning, the very early conversation came out from a Persian poetry and there's this guy called Hafiz who is one of the very best Persian poets and he's always talking about wine in his poetry but it's never actually wine. It's something. It's the juice that he drinks and then he can talk freely and he can just like let go of any sort of prejudgments, you know preconceptions. And we thought, like, you know, they increasingly are, like, getting more and more intoxicated on the juice and being more free and being more holistic. And towards the end, they kind of, like, maybe disconnect from their own, maybe dissociate a little bit from their own thing, and they look at something together. So it was the elixir or the juice in that blue bottle that they're getting buzzed on, let's say.
[00:20:01.930] Kent Bye: And what's it been like here at Venice Immersive 2023 to be able to show this piece to people? And what's been some of the reactions you've had?
[00:20:09.003] Anne Jeppesen: We had really good reactions. Only, actually. But it's surprising to hear what people make of it. We stood a little bit by our booth yesterday and a guy came out and... elaborate a little bit on he was very enthusiastic and like really taken by the visual language without actually worrying too much about what it meant and I think that was nice to hear because also we wanted to be open to people so that they have space to take from it whatever they want actually not whatever they want, but create a lot of space for people to take something from it, feel something from it, but not necessarily anything that we make out for them exactly. So we had some responses there that I'm really happy with, I would say.
[00:20:57.482] Omid Zarei: I talked to our hostess because I go there every day talk to her and she says most people coming out either smiling or smirking which I think fine either way is good because they say like it's either the giant fruit coming out or something or the music they love that old-school that kind of like razzmatazz music but I'm like that's that's that's what we wanted you know people feeling something that maybe they're not gonna sit and talk about anything specific but they felt something that made them smile or smirk or feel something
[00:21:26.237] Kent Bye: Yeah, the way I thought of it was like this sort of chill vibe, and you've got this surrealistic twist. And yeah, it's a piece that's transcending genres to some extent and kind of like a thing that's unique on its own. So yeah, I guess as we start to wrap up, I'm curious what each of you think the ultimate potential of virtual reality might be and what it might be able to enable.
[00:21:47.415] Anne Jeppesen: It's been so great being here and seeing what other people are doing. We saw Céline Damon's songs for Passerby and it completely blew our minds. And we think what they're doing is where we see a lot of potential. When you have these journeys where you're invited in softly. It's a contemplative space where you're allowed to freely put yourself in and have your imaginative powers work more freely. that it's more open-ended. A lot of times I feel like when pieces manage to do that, they manage to create space for me as a spectator to be there freely. That's what really excites me about VR because it can be very profound. For example in Celine's piece, just like meeting a horse, dying horse. What is that? I don't know, but it was a profound experience for me. I love to see more like a development of this more free language in VR. Also, Over the Rainbow was another piece. I noticed that. I really appreciate this. Yeah, there's a lot of potential there, I feel, with the more free-floating, open-ended language in VR.
[00:23:00.560] Omid Zarei: Yeah, along those lines also, I think eventually VR will invent its own language, its own voice, where we don't have to justify what this should be VR or not, because it's things that you can only feel and experience in VR, or XR let's say, in one shape or form. And I think increasingly, when you go to the next festival, you see more and more pieces that are trying to do things that are not like a continuation of another format or another medium. They're creating experiences and language and expressions that are just indescribable. Like that horse. And you just leave wondering, like, what was that that I felt? I never felt like that before. If VR does that, I think we're in good shape.
[00:23:43.886] Kent Bye: Awesome. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:23:49.229] Anne Jeppesen: Well, just that it's exciting to see that people are still like, it's still a very experimental field, and it's very exciting to be a part of. And yeah, I hope to see it grow wildly.
[00:24:02.395] Omid Zarei: I'm just gonna go to town and say that like I think there's a lot of weird time that financials and everything is very tight and stuff and there might be a push from distribution side to create things that are like has a distribution in mind already and I know a lot of festival or platforms are like pushing on that side but we can't really go there before we know what it is so it's I think creator to creator let's figure out what it is and then we'll find our audience.
[00:24:26.922] Kent Bye: Well, I very much appreciate the experimental nature of your piece here, of vocal landscape. And yeah, kind of a deep dive exploration of starting with just the voice and capturing it in these conversations. And yeah, I think there's something quite provocative there, especially as I have a chance to talk to each of you and get a bit more context of how it came about and your intentions of what you were trying to explore. It helps me to set it into a broader context of my own experience. But yeah, I appreciate you taking the time to create it and to also be here in Venice to help explain it all. So thank you.
[00:24:56.941] Omid Zarei: Likewise, great talking to you.
[00:24:59.282] Kent Bye: Thanks for listening to this interview from Fitness Immersive 2023. You can go check out the Critics' Roundtable in episode 1305 to get more breakdown in each of these different experiences. And I hope to be posting more information on my Patreon at some point. There's a lot to digest here. I'm going to be giving some presentations here over the next couple of months and tune into my Patreon at patreon.com slash Voices of VR, since there's certainly a lot of digest about the structures and patterns of immersive storytelling, some of the different emerging grammar that we're starting to develop, as well as the underlying patterns of experiential design. So that's all I have for today, and thanks for listening to the Voices of VR podcast. And again, if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
