I interviewed VR researcher Julia Scott Stevenson at IDFA DocLab 2024 about her research into embodiment and AI within the context of immersive stories as well as her work around climate change. See the transcript down below for more context on our conversation.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.458] Kent Bye: The Voices of VR podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So continuing my series of interviews at IFA Doc Lab 2024, today's episode is with a researcher named Julia Scott-Stevenson, who is from the University of Technology, Sydney, but also collaborates with MIT Open Doc Lab. So Julia is a researcher that is looking at how to use some of these different immersive and interactive techniques, looking at climate change and different ways of using some of these different speculative techniques and using generative AI to have people imagine different futures that they want, but also looking at other aspects of embodiment and immersive storytelling. specifically with her research with MIT Open Doc Lab, which she was there at IFA Doc Lab last year doing a number of different interviews with different creators from last year's selection, but also produced a whole report on this intersection between immersive storytelling, embodiment, and artificial intelligence. So we talk about some of the different big highlights that were from that report last year, some of our research and using the speculative techniques around climate change, but also talking around some of their latest research for MIT Open Doc Lab, which is looking at the future of distribution for some of these different immersive stories. At IFA DocLab this year, there was also the DocLab Summit, and I was hired on to come in and bear witness to all the different sessions. And I wrote up a pretty extensive report that should be coming out here within the next couple of weeks that is accounting for lots of other different conversations that were happening over the course of the IFA DocLab R&D Summit. So keep an eye out for that as well. So covering all that and more on today's episode of the Voices in VR podcast. Like I mentioned in some of the other conversations, my voice was progressively getting lost throughout the course of these different conversations. So you'll hear me in this conversation with a little bit of a raspy voice. So this interview with Julia happened on Tuesday, November 19th, 2024 at IFA Doc Lab in Amsterdam, Netherlands. So with that, let's go ahead and dive right in.
[00:02:08.874] Julia Scott Stevenson: I'm Julia Scott-Stevenson. And in my primary job, I'm a research fellow at the University of Technology, Sydney. So there I am conducting a research project where I'm looking at how immersive and interactive media can be used to help audiences think through climate crisis. But I'm primarily here at IDFA DocLab in my capacity with MIT Open Documentary Lab. And MIT has an ongoing research partnership with IDFA DocLab. It's been going for about six years now, and each year of the festival, they look at some element or they choose a theme and look at some element of the works. So I've been working with the director of Open DocLab, Sarah Wallison, to do research on the last couple of years of DocLab. And in 2023, we looked at the notion of embodiment and embodied interaction, and we were really interested in exploring whether there's something about embodied interaction that offers a different kind of experience for the audience.
[00:03:06.559] Kent Bye: Okay, and maybe you could give a bit more context as to your background and your journey into the space.
[00:03:12.395] Julia Scott Stevenson: Sure. So I came to this immersive field in a sort of a roundabout way. I was actually working in international development in the Pacific and I was dabbling in making short documentaries while I was there. And then I saw a PhD advertisement based back in Australia, making a documentary about volunteers. And so that kind of connected my volunteering background with my interest in documentaries. So I went and I took that on. And it was actually my PhD supervisor who said, oh have you looked at this kind of new field of interactive documentary that's kind of emerging and that could be interesting and this was in the kind of early 2010s or just before that and that was when an interactive documentary was really taking off in a big way and public broadcasters around the world were making work and there was the NFB in Canada and there was just a lot of really interesting experimentation and so I made an interactive doc for my PhD about volunteers with Australian Red Cross And then soon after that, the interactive doc field kind of tanked a little bit in that no one had really figured out what the funding model was for these works. But conveniently, there was also a pivot to VR, an immersive experience. And so then I went and took on a postdoc in UWE, University of the West of England, Bristol, where they had the IDOX research network. And that had grown out of this interactive doc field, but was also interested in this pivot towards immersive work. So there we really started looking at how documentary makers were kind of shifting into this more immersive experiential space. And I did a bunch of research around how we can use these kinds of forms to actually make pathways forwards. You know, what is the future that we want to see? That was my research question that I was interested in. What is the world that we're trying to build and how might these immersive forms help us do that?
[00:05:05.505] Kent Bye: And can you elaborate on how you split your time between what you are doing with your day job with the climate change stuff and then also your work with MIT Open Doc Lab and how they kind of fit into your overall work and how you split your time and where you're based and everything else?
[00:05:21.533] Julia Scott Stevenson: Yeah, sure. So they're both, they're really interconnected. So I'm based in Sydney. That's where I live in Australia with UTS. And I moved back there from Bristol just after COVID. So I've been there for about three or four years now again. So my primary role is doing creative practice research. So working on this research project around climate futures and doing creative practice experiments towards that. So I ran a project. My first prototype was earlier this year in May as part of Climate Action Week in Sydney, where I brought people into this space that UTS has called the Data Arena. And it's a large 360 degree screen room. So four metre high screens in this cylindrical form. And people can come into this space. There was a QR code on the screen and a question that was, what is your hoped for climate future? And some guiding questions on how you might think about writing a prompt. So people would scan the QR code and bring up a browser and then they could enter their prompt, whatever their hoped for climate future would be. And then we would use generative AI to create an image that would then appear on their phone. And if they liked it, they would submit it to the collage. And we built this growing collage across five days. And the point being not explicitly what those images showed individually, but en masse how the collection of them grew over time and how that might contribute to a feeling of being part of something and realising that there are other people who have this same desire that you do of a positive pathway forwards. So I split my time doing that kind of work, doing research around that, more traditional academic research, seeing what the state of the field is, and then feeding that into my creative practice. So I've got a shoot coming up in a couple of weeks where we're going to do some volumetric capture interviews that might form a part one to that project, to a new iteration of that project. I also teach, 25% of my time is teaching. at the university but I'm four days a week in that role so I have this extra day that I can contribute to the MIT research and in fact now this year because the MIT research is so relevant to my own work as well I can actually pull that in I've got a lot of flexibility to bring that into the work that I'm already doing So they kind of integrate quite nicely. And then in terms of working with Sarah and MIT, it's just the usual Australian context of getting up early, staying up late for the Zoom calls, making plans that way, and then finding myself on this side of the world and actually getting to see the projects.
[00:07:44.099] Kent Bye: Nice. And so just a clarifying question on this climate change project. Is it a VR piece that you're making or because you said they're like in an immersive dome and they're getting feedback. And so I can definitely get a sense of the interactivity portion, but I don't have a clear sense of the immersive context for what this project is that you're working on. Maybe you could just elaborate what you see at the end. What does that project look like and who would be watching it?
[00:08:07.915] Julia Scott Stevenson: So initially I was thinking, well, the purpose of the project is to bring people together into space so they can actually interact with each other while they're interacting with the project. So initially I was thinking that I didn't want to make a VR version, but actually now that I've been doing this research and the practice, I am kind of interested in possibly adding a VR version as well. So the current version is a kind of drop-in experience. So the one that we ran in May, it's called Collective Visions. And people could wander in and out at any point and interact with it, scan the QR code, make as many images as they wanted and leave whenever they wanted. And we're actually running that version again in December a couple of times. And that will be fairly the same. People will be able to wander in and we'll build this collective collage. But with version two, the VolCap interviews that I'm going to do will be with some climate practitioners and people that are kind of informally working in the space and so have a real connection with their own ideas of what they want to see. So then I think there will be kind of a timed entry to this space and people will come in and see these recorded interviews at human scale in front of them as a kind of an introduction to the piece and an additional form of prompting on helping them come up with what they want to see before they then conduct the interactive part. I'm interested in how a VR element might work though as well because I think this idea of seeing a person in front of you at human scale is kind of interesting and inhabiting a space with them too might be quite interesting to see. One of the reasons I also want to do these interviews is Some of the anecdotal feedback I got when people came to do collective visions was the difficulty in devising a prompt of what they want to see. Everybody already knows the apocalyptic visions of the future. They can imagine that we're actually already seeing it around the world. You know, Australia has bushfires, other places are having horrific flooding. So we know what that looks like and we have plenty of mainstream media to show us those visions as well. But when people are asked, what do you want? And having to actually articulate that in words, quite a few people commented how difficult that was. But across the spread of the exhibition, as more and more images emerged, it became easier and easier for people to kind of draw on what other people had wanted. But also by offering these interviews first with people who've been really thinking about this quite critically for a while, I'm hoping that might provide some additional prompting and some ideas.
[00:10:29.841] Kent Bye: Yeah, I wanted to have a follow-up question on that, specifically around how I was talking to Lisa Masseri, who's a Yale anthropologist, and she wrote a book called Land of the Unreal, tracking how virtual reality has developed in the context of Los Angeles. So she was in Los Angeles and getting exposed to different speculative techniques like world building within the context of these virtual spaces and Then when she went back to Yale and to listen to her other anthropological colleagues across social sciences and science and technology studies, she said that these speculative techniques have started to be referenced a lot more, even these world-building techniques. And her comment was that we can get so trapped in these rational modes of thinking that are reductive, not thinking about things relationally in a holistic context, and that they kind of reach a point where they're no longer useful, and that turning to these speculative techniques can actually kind of open up the realm of possibilities, but also to evoke this kind of creative imagination that allows us to get beyond the apocalyptic doom scenarios, in this case with climate change. But I'm curious to hear some of your thoughts on that, just because coming from the academic realm and the interfaces between some of these concepts like world building and speculative futurism that I've seen a lot within the context of these immersive stories, but how that specifically plays out in the context of what you're working in.
[00:11:46.460] Julia Scott Stevenson: Speculative futures is a really fascinating and also kind of fraught research area, I think. And there's so much there to dig into. And it's something that a lot of different disciplines are starting to do or even have been doing for a while. So I don't come from a design background, but I know that design has for a long time engaged with speculative design, design fiction. So I've got colleagues at the university that really work in that kind of space. I come from a more almost traditional social sciences field, and that's taken a lot longer to come around to this kind of speculative approach. And I think a little bit of caution is also really useful because there's a risk, right, that when we focus on looking forwards, we forget that we actually only have now. So that speculative future doesn't exist yet. It's not here. And focusing too much on that potentially takes us away from actually building it, actually doing the work of building it. But what it can offer is a sense of active hope. So if we can do this well and build these possible future ideas, so like bringing people into my collective visions project and having them feel that struggle of deciding what the future might look like, but then engaging with what other people have said, seeing the other prompts, seeing the other images and realising that they actually can think of something i'm interested in finding out and this is some of the research i'll be doing with surveys is does it feel like they have a renewed sense of agency in the face of that future the other risk though is when i characterize this project i have to be really careful that the way i describe it is about what do you want to see rather than suggesting that i think people should only have a hopeful vision of the future I really want to make sure that there's space there for people who don't have a hopeful vision of the future as well. So I was very careful not to censor any images that were submitted. If someone saw, you know, they responded to the prompt and they were asked, what's your hopeful climate future? And they thought, you know what? I don't know. I don't know what it is. All I can see is the apocalyptic version. So we did have a small smattering of images that were everything is burning. The oil executives are sitting on piles of money, you know, that kind of thing. That was some of it. And I really wanted to keep that in there. I was hopeful that it wouldn't overtake the collage and it didn't, which was really good to see, I think. But it was important that that stays there. And it's also a risk. So I was talking to an Indigenous Australian woman who I was interested in interviewing and seeing if she would be willing to be interviewed for my introductory parts and I perhaps didn't do a very good job of explaining it and maybe it did come across sounding like I wanted it to be hopeful futures and she kind of rightly schooled me a little bit and sort of said you know screw your hopeful futures like my lands are already underwater why should I be hopeful in the face of this and that's a totally valid response as well.
[00:14:34.983] Kent Bye: Yeah, it's this kind of tension between the reality of what's happening now, but also how do you dream into the future and have that sense of hope. But yeah, there's a real tension between those two that I think are really at the forefront right now. OK, so if we move back to the type of research work that you were doing with MIT Doc Lab, the last year, maybe even the year before, you were doing a series of different surveys, asking people to get feedback on a lot of the projects that I ended up seeing last year and covering and having podcast episodes. But you were also, in parallel, doing your own interviews and research into not only the projects that were here at DocLab last year, but also contextualizing it with this lens of embodiment and the role of embodiment with immersive and interactive documentaries. Maybe just set some additional context for how this came about. What was the initial questions that you were asking and what you were really trying to get out of that?
[00:15:28.217] Julia Scott Stevenson: Yeah, last year was actually the first year I was officially involved with the research. And I came on and we started having some conversations with Sarah and with Kasper Sonnen about what theme we might want to look at for that year's research. And perhaps a little bit cheekily, I was already interested in embodiment to do with my own climate projects and thinking through how an embodied approach might have impact in that sense. And so I said, well, look, I'm kind of thinking about this already and maybe embodied interaction is an interesting idea. And Sarah was also keen on that. And in explaining it to the DocLab team, and actually Caspar initially was hesitant because he said, well, maybe I don't know if many of our projects this year are actually about that kind of fully embodied interaction. So then I had to kind of pull back and say, oh, that's actually not entirely what I mean in the sense that we're not only interested in projects that bring a full body inside a virtual space. And in fact, it's actually about a more expansive definition of embodiment. This recognition that we are embodied all of the time. So we're embodied sitting here talking now, your listeners are embodied while they're listening to this, and sitting in a performance lecture, which was one of the projects last year, you're embodied. So we weren't trying to suggest that there was some kind of hierarchy of embodiment where if you're wearing a haptic backpack and you've got the wind blowing at you that that is somehow a better form of interaction but we were just wondering if once we start talking about embodiment and asking the audience to think about that embodied approach does that offer some kind of different level of understanding or different way of engaging with the projects and when i explain it that way i think yeah, Casper really got it and thought that was really interesting. And so we selected nine projects that had a really interesting spread across, you know, from one end, the really much more fully bodily immersive through to, for instance, the voice in my head project, where it's just ear pods and you're moving through the actual physical world around you. And so that's an embodied process. And then Neerit Pellad's This Conversation is Off the Record, which is simply a performance lecture so you're seated in a much more sort of traditional environment and we were looking at the embodiment of all of those kinds of things and there were three or four VR projects where you were different levels of moving through space or required to interact physically so there were some really lovely and quite different ways of examining what embodiment might mean.
[00:17:49.605] Kent Bye: Yeah, Turbulence, Jamais Vu, was probably one of the more embodied experiences. There was also another piece around sexual assault that was with your hands being embodied in the piece. And so you were involved in the process of writing up a report with a number of different key takeaways that you presented and basically published a white paper. And two days ago at the R&D Summit, you shared some of the key takeaways. So I'm wondering if you can maybe go through some of the main findings from your report.
[00:18:17.699] Julia Scott Stevenson: So the first finding we had, which I think was actually quite interesting, was a finding that makers were actually starting to build the onboarding of a project into the project itself. Whereas historically we've often seen, and maybe particularly for VR, but not only VR, You have attendants who kind of need to walk a participant through what they need to do before they put a headset on them or before they introduce them to the experience. And there's this kind of quite involved process. But now we were seeing with projects like Turbulence Jamevu or Emperor VR that that was actually built into the project itself. So there was a slow In Emperor VR, for instance, you asked to reach out and touch something and the makers gradually increased the complexity of what the users were asked to do as a way of familiarising them and waking up their bodies a little bit. And so this is a kind of a sophistication, I think, that we're starting to see with makers now. So that was a really interesting finding. We also were interested in looking at discomfort as a creative tool. So we specifically asked audiences about ideas of discomfort and the artists as well, about whether they were explicitly thinking about how to use that. And they really were. I mean, and Turbulence Again and Emperor both used discomfort quite explicitly and in careful ways that connected with the content of their stories. And audiences seemed to get that. So that was really interesting to see. So the takeaway was about there being artists productively challenging audiences as a way to new forms of understanding. And they were challenging them in physical embodied ways, which I think is really interesting to see.
[00:19:53.274] Kent Bye: Yeah, with Turbulence Jaume Vu, it's about vestibular migraines and how Benjamin Andrews has this condition where he has either depersonalization experiences or kind of like this dissociation from his body, but also like having this dizziness or nausea and using VR explicitly to recreate that type of nausea within users, which normally you would avoid deliberately trying to make your audience sick, but in some ways, It kind of worked in a way that it actually fit with the narrative of how the story was being told. So I thought that was quite interesting. And actually, they showed Turbulence Jamais Vu again at Venice and had a little bit more of a disclaimer at the front so that people, because, you know, when you're in a festival context, you go into a piece and you walk out and you maybe get nausea, emotion sickness, then... You know, having that level of disclosure in a way that people can modulate their own triggers of emotional sickness so they can shut their eyes or not move their head around so much. So, yeah, the kind of ethics of how do you use that in a way that serves the narrative, but also not do it in a way that is unduly bringing harm on the audience.
[00:21:02.193] Julia Scott Stevenson: Yeah, that's absolutely right. And I think Ben and Emma really got that balance really well. I remember doing the piece and really early on in the piece, you hear Ben's voice saying, feel your feet on the ground and you can bring yourself back to that sensation if you need to at any point. So there was this real feeling of care from the maker. So even though you knew you were about to undertake something that might be destabilising and unsettling, you felt looked after by the piece as well. So I wasn't ever afraid that it was going to destabilize me in a problematic way. And I do get sick in VR pieces, but somehow I wasn't worried about that with this one.
[00:21:38.125] Kent Bye: And what were some of the other takeaways?
[00:21:41.707] Julia Scott Stevenson: So we also were really interested to see this shift away from some of those earlier approaches with VR about wanting to wrap people in a fully virtual space and instead now makers are engaging with layered realities. and that was not specific to VR it was VR and also some of the other immersive projects that were outside of VR was that makers were understanding now that we as audience members can move backwards and forwards between different levels of reality between a virtual space between our bodies here back in the quote-unquote real world or physical world behind us and makers are leaning into that and using that whereas maybe in the past there's been a bit more of that disconnect between leaving your your physical body behind and entering a space so recognizing that is this me am I in this virtual space am I in this physical space am I somewhere in between shadow time was a project that did that really well where you look at your hands and you're seeing them in pass through from the VR, but then you get these kind of virtual hands laid over the top. And that was just this moment for me where I felt so destabilised. And that was the point, you know, like it was this whole, wow, where do I exist here? What's actually happening? And there's some really creatively and critically interesting questions in there. And it was something also that even makes sense now amongst this year's topic or theme for DocLab, which is this is not a simulation. This pushing back that there is this simulated reality, whereas in fact what the makers were doing was realising that wherever we are, that is our reality. So that was our third takeaway, was this playing with slippages and playing with realities. And the fourth takeaway was basically that audiences are coming along for this ride. Audiences did seem to really get this and they're comfortable with this kind of skipping backwards and forwards. The audience surveys that we did seem to really indicate that people are on board with this. They didn't need to know 100% where am I and who am I. They might be asking this question and it's interesting that they're asking the question, but they weren't frustrated if there wasn't a clear answer and they were recognising that that was part of... the projects generally and a bit of uncertainty about that. For instance, in Emperor VR, you're inhabiting the filmmaker's body at some point, or perspective at least at some points, and then you're inhabiting her father's perspective at some points, but then in others, you've got tasks to do as you yourself, the participant. And that complexity totally made sense within that project.
[00:24:14.304] Kent Bye: Yeah, and the last point that you're talking about in your key takeaways being that documentary as a form is a perfect venue to start to experiment with the potentials of AI. And I know that there's John Grecian's creative treatment of actuality definition of documentary, and I think that's something that Kasper Sonnen had cited to me, and I think when we talked at DocLab back in 2019, you had a number of different definitions of how you start to think about documentary. But, you know, maybe you could elaborate on the different ways that you start to understand what documentary is and what it can do and how AI starts to play into, you know, especially this year at DocLab, where there are so many more AI projects. But last year, we start to see the early beginnings of experimenting with the latest AI technologies and in a context that allowed us to reflect on what it means to be human. So maybe you could elaborate on how you start to think about what a documentary is.
[00:25:07.984] Julia Scott Stevenson: This is something that I'm really thinking about a lot at the moment. I'm actually writing a piece about this specifically and the impact of AI on documentary and how documentary is a really useful tool for engaging with this space of AI. And in fact, I hosted a panel on Saturday here at IDFA called the humans have entered the chat where we had three or four filmmakers or artists who all are doing projects in some way related to AI. And we were having this discussion on stage about why documentary really is a particularly apt art form and way into this conversation. And probably the primary reason that I think documentary is great for this, on the face of it, it doesn't seem like it would be, right? Because we have this expectation that documentary represents the real and the use of generative AI forms messes with that. But documentary as a form allows for complexity. And that is what I think we need in debates like this. So in the documentary format, there is space to straddle the line of both the positive applications and the negative and not have to come down on any one side of the argument. For instance, the opening night film for IDFA this year was about a hero, which is this generative AI documentary where they've synthesized Werner Herzog's voice and a script based on his work. And so there's these really fictionalized elements in there. But what it's doing is it's doing both the work of showing you, oh, my goodness, look at what these tools can do. This is quite incredible, while at the same time bringing in really classic documentary techniques of talking head interviews with experts who can also pass judgment on this and so it's doing that polyphonic multiple perspectives thing that documentary does so well which is bringing in so many different voices so many different points of view and opinions and letting the audience sit with that for some time and really really think about it So I think that's why documentary is so great in this space. And also documentary is a form that mainstream audiences can engage with in a way that's not scary. And they can get that complexity of argument that's beyond a headline or a fear-mongering article that's a couple of hundred words in the news or something like that. or anything that's maybe trying to scare them off with complexity. So complexity in documentary can be really useful, but there's also a form of complexity that is distancing for audiences, that is like, you're not smart enough, you don't understand the models and the tools that have gone into this, so you just have to believe the tech hype about how it's either going to kill us all or it's going to save us all. Yeah. And again, documentary can kind of skewer that a little bit and use complexity in a way that an audience can engage with and understand. And I think that's so interesting to see.
[00:27:52.222] Kent Bye: Yeah, you had mentioned at the R&D Summit that you had just seen about a hero. You're very excited to talk about it with other people. Have you had a chance to talk about it to other people? And what was it about that piece that you wanted to explore more or really kind of interrogate or talk about?
[00:28:07.125] Julia Scott Stevenson: Ah, yeah, so I ran into the filmmaker yesterday, actually, Piotr Winiwicz, and he was like, yes, we're going to talk about it, but I'm trying to catch up with him today or tomorrow, so I haven't sat down and had the conversation. I have had a few short conversations with a couple of other people and had some really varying opinions, which is great, you know? Like, that's what I want to hear. I don't want to hear the one soundbite that is passing judgement on this film. I want to sit down for three hours over a coffee and really, like, tear it apart. I guess I was fascinated by this film, I think... Their opening scene is just so brilliantly done where what they do is they take Herzog's lawyer in what seems to be a real live video interview. And okay, you sort of, it's a talking head interview. I understand this setup. And then he seems real. And then he says something like, I'm now going to say words that are actually not real. And then it shifts to kind of a deep fake version of him, which they've flagged for you. You know that it's coming. But there's nothing else different about it. Nothing changes. And that's the point that it kind of blows your mind a little bit and you're like, oh my goodness, oh yeah, I really don't know how to trust what I'm seeing. And then it moves into these kind of fictional scripts that has been generated by the AI based on Herzog's films. And it's got that still kind of uncanny sheen that generative AI does. but it's good like it's really it's really quite effective and real to some extent but not like you know like working in this space and seeing a lot of ai you can pick it as generated but maybe a mainstream audience might not people that aren't engaging with this so much so i was sitting there watching it for the first 10 minutes just being like oh i think i hate this no maybe it's amazing i'm not really sure and i was kind of skipping backwards and forwards And then the next point that made me really think twice as well was they then moved to interviews with real people. But at this point, you don't know what's real anymore because there's been so much messing with what's going on. And actually, the first interview, I didn't know it's with a philosopher whose name I didn't recognize and know. So at the time, I just thought it was another generated person. But they said the first real thing that had been said, like I actually started to pay attention to them. And I was like, oh, wow, the script's actually generating something a bit more meaningful this time. But then once I saw the second interview and it's Stephanie Dinkins and I know who she is, she's a creative technologist. And I'm like, oh, that guy must have been real. He was actually a human using his human brain to generate something that made sense. So, yeah. So, I mean, I could just go on for hours. But there's a lot of layers there that are really interesting.
[00:30:38.905] Kent Bye: Yeah, the beginning title card is basically priming you to say this is a film that's really pushing the limits of what's real and what's not real in that they have some conceits where they will show a script and the text of the script and they'll have people read that script so you get the sense of like, okay, they're reading the script. This is a part that's a scripted part. And then they have these interviews, and sometimes the interviews are on the TV screen. And then you see them, but sometimes they have the audio from an interview embedded into a diegetic element of a scene, of the radio playing. But then it'll switch into what was an interview, then into something generated. At the end credits, I don't know if you noticed, but they had some disclaimers saying, OK, from these people, this was what was generated. They said Werner Herzog was all generated. But then I was like, oh, wait, what about if that was all generated, then Herzog's giving his consent in the very first scene. And then so I was asking the filmmakers, well, was that real or not? And so even after watching it, there was all these things of trying to figure out what was real. But because I got early access to it, I had a chance to watch it twice. So I watched it again, but with that key code of listening for, OK, I know this person, this phrase is going to be generated. And I was able to identify and clock Okay, this was from an actual interview and now we're flipping into something that was generated. But again, it was very subtle and I didn't pick it up the first time and it wasn't until the end where it was disclosed what was generated to really understand. So I think there's some interesting conceits for how they're communicating in the visual language of the film of kind of like cluing you in. But sometimes... They will start an interview with a poet, for example, and then he's coming in and speaking, but then they say, oh, actually, this whole interview is generated. They show the script and what's being read. And in a sense where they actually filmed everything with real human actors, but it was kind of like Sunspring in 2016, published by Ars Technica, that was, again, training on a corpus of science fiction and then generating a script and having the real human actors act everything out. And that's kind of like what they did in this piece. So, yeah, I had a chance to have an in-depth conversation with both Mads and Pieter to unpack some of the intents that they had with the piece. But, yeah, I think it's super fascinating because, like you said, documentary is a perfect form for AI and to explore the potentials of AI, but to have that be the opening night film at a documentary film festival... is in some ways saying, okay, here is what is to come here. And I know that MIT Open Doc Lab did a whole series of what are the ethics in terms of using generative techniques within documentaries because there's implications for the corruption of archival material and having stuff that people may think are real but isn't real. And so I think this is a type of a conversation that's going to only get even more complicated and nuanced as we move forward. And so I feel like About a Hero is a sort of initial take of how to have the types of media literacy, visual language to kind of disclose in a way, but also purposely leave different things in an ambiguous way that you're left questioning as to what's real and what isn't.
[00:33:39.248] Julia Scott Stevenson: I think that's absolutely right. I think by placing this film as the opening night one, And I think this film in particular, as you mentioned, they made a whole host of creative and ethical decisions around how much to flag, what to put in the credits at the end. So it seems to me like they were fairly intentional about doing that. And I think... Then hosting that film as the one that is kind of the first one to do it on this scale is really important. And it sends that message that this wasn't a look, we're going to try and use the tools to just mess with your reality. We're doing it for these very specific reasons. And we're showing you a little bit behind under the hood. But we're also making some creative decisions that are up to us as artists that we're not going to reveal. So I do want to go and quiz Piotr on a bunch of things, but also I understand it's his prerogative to then say, I'm going to keep some of this as a creative decision. So, yeah, so I think actually I was reading recently the Archival Producers Alliance has released a set of guidelines. And I think we'll see more and more of that kind of thing emerging where there are these suggested ways of how you might flag work like this without necessarily always just having to have a big sign at the front that says warning AI ahead. But how do you build that in in a way that respects the audience but respects the creative integrity of the process and the film itself as well?
[00:35:03.925] Kent Bye: Yeah, the MIT Open Doc Lab webinar that was around ethical uses of AI was featuring the Archival Producers Alliance in their paper that they had just recently published that had all sorts of guidelines. I know Netflix has made some creative decisions that are on the less ethical side in terms of reconstructing different scenes and messing with archival material. But yeah, at least having normative standards from the industry to follow to try to do all this stuff in an ethical way. I did want to ask briefly around the research that you're doing this year. I know it's at the very early stages, but also happens to overlap with some of the larger discussions that I think are happening within the broader XR industry, which is the challenges around distribution. At the Venice Immersive this year, there was a think tank that was bringing together lots of different stakeholders from producers and content distributors to have a whole two-hour discussion that I was a part of bearing witness to and writing up the report that hopefully is going to be coming out here within the next month or so. But I'd love to hear some of your thoughts on how this came about as an issue to really dig into and investigate and what your approach is to start to investigate some of the different challenges of distribution for these immersive pieces.
[00:36:14.451] Julia Scott Stevenson: So this is quite a shift tonally from last year's research. So we were looking at embodiment and AI. So that was very, you know, internal to the projects. But this year, as you say, we're now kind of zooming out, I guess you could say, and looking at distribution across the industry as a whole. And the reason that we're doing this particular subject is because, well, on the one hand, it's just so needed. The distribution is almost non-existent or like consistent pipelines for distribution just don't really exist well in this industry. and director of Open Doc Lab, Sarah Wollison, this is something that's just really close to her heart. And she is really, really keen to see something built that is just so much more intentional and forward-looking and open and independent and supportive of artists. And she really thinks there's a moment here at which we kind of have the possibility to build that a little bit. And so we did like a very rough field scan of the reports that exist. And there is one that came out of that Venice conversation. And there are a few bits and pieces that have been done by different organisations in different regions. And not always, but quite a few of them sort of have these findings that are like, oh, okay, the artists need to kind of come and meet the market a little bit more. And while that's an interesting thing to note, We also want to say, well, hang on, how about doing it the other way around? Like, how about building the market to actually come and meet the artists a little more rather than just trying to make artists fit whatever this sort of market process is? So there's really an opportunity to say, well, what might this industry look like if we really kind of acknowledged what those different pipelines might be? figured out ways that you could support artists to understand where they fit in this landscape. So understanding if they have a project that is going to tour and it is going to be a blockbuster at festivals, or maybe they are like a little indie production and there should be some kind of a set of pipelines for them to recognize where they can take this work without it having to be this kind of one and done. And then they're exhausted and then they don't want to come back again because it's just been too hard.
[00:38:23.772] Kent Bye: So what are the methods that you're doing in order to survey what's happening and then provide some sort of feedback or direction or what's the output that you're aiming for?
[00:38:33.960] Julia Scott Stevenson: So there's two phases. One is specifically this partnership with IDFA, which is planned out between now and next year's festival. So we launched that at the summit on Sunday and that includes a field scan. So looking at those existing reports in a lot more depth, seeing what the gaps are, seeing what's already been done and what's been found and trying not to repeat that. Then there's also a set of in-depth interviews with players in the field to have that real insight into what's been happening and what could be happening. Also some case studies of projects that have either done this well or done something really interesting in how they get themselves to audiences. So those are kind of some of the initial outputs for now that will be happening over the coming year. But the other thing that Seria really wants to do is build an actual ongoing coalition around this. So an independent XR distribution coalition And that's something she's drumming up support for at the moment. And we have a number of partners who are interested in participating. And this is looking over the next kind of two to three years in actually building something that could be really interesting for, you know, not just putting out a report and then moving on, but actually trying to make this a kind of an evergreen, ongoing, solid project. solid thing that we can engage with and so what that means in terms of outputs that will actually be part of the research is finding out what people want to see. Last year's research was a report and I think it's a really interesting and useful one but there are so many reports and people struggle with reports so maybe the output is a toolkit or maybe it is an online platform or maybe it's just an ongoing discussion board where people find this information or maybe it's all of those things or maybe it's something we haven't even figured out yet.
[00:40:13.049] Kent Bye: Great. And finally, what do you think the ultimate potential of interactive and immersive storytelling might be, and what it might be able to enable?
[00:40:23.877] Julia Scott Stevenson: Maybe I'll answer that by saying I was having a conversation with someone last night who said that the ultimate potential of AI in the immersive space is the kind of personalising of narratives. And I'm a little bit wary of that in the sense that I worry about adding to filter bubbles and kind of contributing to this idea of just giving people more of what they want. But I do think there's also something in this. I think... There's a really interesting future in creating immersive media projects that are collective, so multiple audiences together. I think that is a really interesting part of the future. I think people participating in a group and being able to interact with other real humans around them, but in a way that is responsive to them. I think responsive storytelling that is expansive rather than following kind of filter bubbles. I think that is going to be a really interesting next stage.
[00:41:18.595] Kent Bye: Great. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:41:24.217] Julia Scott Stevenson: I think maybe the final thing is part of the research for the distribution will be finding where audiences are and how audiences might better engage with immersive space. So I'm just really interested to talk to more people about how much they engage with immersive, whether they understand it as a field, and what are the ways in they find. So that's something I'll be keeping an eye on, I think, and talking to people about how do they access this work.
[00:41:49.304] Kent Bye: Nice. Well, Julia, thanks so much for joining me today on the podcast to talk a little bit more about all the different projects that you're working on. And yeah, just quite fascinating, the range of different stuff that you're looking at. But also, there's just a lot of overlap between some of the stuff that you've been doing there at OpenDocLab and asking these questions and doing interviews with different creators and looking at stuff and the stuff that I'm doing, which is just capturing more general oral history interviews and trying to document what's happening in the field. But also, yeah, just looking forward to the issues of distribution is something that I've been thinking a lot about talking to people as well, having some conversations with different stakeholders who are at the forefront of this issue and some of the things that they're experiencing. And yeah, it's really hopeful to hear that MIT is really taking this on as an institution to help look at and survey and potentially even help to cultivate and build something that's going to provide alternative pipelines for distribution since it's certainly needed. So yeah, thanks again for joining me here on the podcast to help break it all down.
[00:42:47.601] Julia Scott Stevenson: Thank you, Kent. So great to chat.
[00:42:49.687] Kent Bye: Thanks again for listening to the Voices of VR podcast. And I really would encourage you to consider supporting the work that I'm doing here at the Voices of VR. It's been over a decade now and I've published over 1500 interviews and all of them are freely available on the voicesofvr.com website with transcripts available. This is just a huge repository of oral history, and I'd love to continue to expand out and continue to cover what's happening in the industry, but I've also got over a thousand interviews in my backlog as well. So lots of stuff to dig into in terms of the historical development of the medium of virtual and augmented reality and these different structures and forms of immersive storytelling. So please do consider becoming a member at patreon.com slash voices of VR. Thanks for listening.
