In 2019, Lance Weiler premiered Where There’s Smoke at Tribeca Immersive, which has since had four major iterations focusing on different innovations in immersive storytelling. It’s a very personal story where Weiler interviews his father about his life and passion of taking amateur photos of fires while his father was dying from stage four colon cancer. I had a chance to speak about his 17-year journey with this project after his world premiere in 2019, and then the next episode is an update after continually iterating on this project four and a half years later. It’s a piece that asks us to reflect upon our lives, our mortality, the most meaningful objects in our lives and how they often reflect our memories or identities in a unique way. Where There’s Smoke has had elements of a community grieving ritual that has evolved over the years, but the core seeds of that are also contained within the original installation that showed in New York City for Tribeca Immersive in 2019 where this conversation takes place.
This is a listener-supported podcast through the Voices of VR Patreon.
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye and welcome to the Voices of VR Podcast. It's a podcast that looks at the future of spatial computing. You can support the podcast at patreon.com. So I'm going to be doing a bit of a mini series of looking at three different artists and interviews that I've done with them a number of years ago and interviews that I did with them this year in 2023. over the course of doing the voices of vr podcast i've gone to many different events of the years and have recorded hundreds of podcasts that have yet to be published and so i've got this huge archive and backlog of different interviews and so this year i did a number of different interviews with artists that i have previously interviewed but and published and so i figured it'd be a good time at the end of the year to go back and look at some of those earlier interviews and then the update conversations with them. So I'll be talking with Lance Wyler with a piece called Where There is Smoke, where there's been like at least four major iterations of this project. I have Tung-Yin Cho, who did a trilogy of looking at gay saunas in Taiwan. And so I talked to him about the first and third in a series called In the Mist and then Traversing the Mist. And then Lauren Lee McCarthy did a piece called Surrogate at Sundance in 2022, and then came to IFA DocLab in Amsterdam in 2023 with Kyle McDonald with the piece called Voice in My Head. And so over the next six episodes, I'll be going through both the first and second of each of these episodes, starting with Lance Weiler, who did a piece called Where There's Smoke, which was a piece about his father, who was like an amateur photographer, and he loved to take photos of different fires. And there was always this question as to whether or not his father might have actually started some of those fires. So it's a piece that is exploring these questions, but also is an interactive experience that he's done at least four major iterations of. I saw the first, second, and fourth iteration. And so this is the very first iteration that was showing at Tribeca, which was an installation, an interactive component. It had like escape room components, and there was a whole art installation with his father. the second iteration was at if a doc lab where it was like a online grieving ritual and then the third version was taking lessons from that online version and then making more of a physical installation with a lot of IOT technologies and then just recently Lance came to Portland for the Portland Art Museum Center for untold tomorrow where he's been doing a prototyping session with where there's smoke where he's doing a little bit of a live remixing of this piece and That has different generative components and interactive components where he's turning different aspects of the theater going experience into an interactive immersive experience So that's what we're coming on today's episode. Otherwise severe podcast. So this interview with Lance happened on Thursday April 25th 2019 at the Tribeca immersive festival in New York City, New York so with that let's go ahead and
[00:02:55.814] Lance Weiler: Dive right in. My name is Lance Weiler. I'm a storyteller. I've been kind of exploring new forms and functions of storytelling. I've been doing it for quite some time, probably over 20 years. I am also a professor in practice at Columbia University in the film and theater department and the founding director of the Columbia University School of the Arts Digital Storytelling Lab.
[00:03:19.358] Kent Bye: So yeah, maybe you could describe to me this project that you've created that you're premiering here at Tribeca Film Festival.
[00:03:26.582] Lance Weiler: Sure. The project is, it's called Where There's Smoke. It's definitely the most personal and vulnerable work I've ever made. It's been, I'd say probably 17 years or more in the making. I think it was because I had to be emotionally ready to make it and it's a project I collaborated on with my father. He passed away last July from stage 4 colon cancer and we collaborated on it together. It's an exploration into memory and loss and end of life. And it's set within this frame of, I grew up in a firefighting culture. My dad was a volunteer firefighter for over 20 years and an amateur fire scene photographer. And so after he passed, I started to find thousands and thousands of fire slides. When I was growing up, I grew up in that culture and, you know, like the soundtrack of my life, at least at a young age, was always scanners, fire, police scanners. when something would come through I would go off with my dad from a young age probably about seven upwards and we would race to go to where the fires were and we would photograph them and then later in life there were two devastating fires that kind of shook our family and for over 30 years I always wondered if there was any connection between the fires and my father and so I initially started the piece to kind of explore that but then it became something else as I was going into it and I and kind of going through all the archival elements of it and working and collaborating with my dad.
[00:05:03.577] Kent Bye: So 17 years is quite a long time for any project. Maybe you could talk about that moment when you decided that something needed to be done with this material and to be able to tell this story of not only your life and your relationship with your father, but a part of your own upbringing of going to these fires. But what was the catalyst? What were the questions? What was the thing that you're really trying to unpack here?
[00:05:29.842] Lance Weiler: I think a lot of it came from, I'd say the inciting incident was when my father was getting a diagnosis on how much time he had left to live and I went with my wife because my mom was too overwhelmed and she couldn't go and My wife and I went and we had this very, I guess you could say, unempathetic interaction with a doctor that became the inciting incident for the piece. I remember, even though the doctor later apologized, I remember going through that and saying, there has to be a different way. There has to be something different. I guess it was this thing where it was the realization that My dad didn't have much time left, and I felt, if I was ever going to make this project, I had to find, I guess, the courage to make it. And there was an element of, through interviews, I felt like maybe I could get a deeper sense of who my father was. And so, we set forward and did a series of interviews together over the last year of his life, and collaborated on the project, determining what kind of slides that we would, you know, show in it, what the set would look like, stories that we would discuss, and it was really quite powerful. But I think the 17 years was really, it was something that just, I think when I reflect on it, it was like, I don't know if I was ready to tell it, right? And I don't know if I would have ever been ready to tell it, but I think the fact that he became sick and, like, I felt that clock, you know?
[00:07:08.167] Kent Bye: Well, throughout the course of the experience, there's like at least three different phases, maybe four different phases that there's different rooms that we're going through. And I've had a chance to see both your Frankenstein AI at Sundance and then the next iteration of the Frankenstein AI that was at the IDFA doc lab. And so I see this evolution of these group facilitations of before you dive in, you are paired up with an individual, you have a series of different things that you're asking each other. And in some ways, because you've gone through the experience of fire of your home and having to be in that moment and to potentially grab something maybe you could talk a bit about How you're gonna translate an experience you've had and to create this imaginal experience for people to imagine if they were in that situation What would they do and then how are you using the memories of the participants as a part of the storytelling process?
[00:08:06.443] Lance Weiler: Yeah, I think that what's fascinating to me is this idea of something becoming embodied in some way. And I think when I was thinking about it, it's something very powerful and visceral about a fire. And I thought it would be interesting to kind of start with a visualization exercise where somebody was asked to kind of imagine that they were in a space that they had lived in, that they live in currently or had lived in in the past. and that they could only take one thing that they were emotionally connected with and they could save that. Everybody else got out safely. But they were able to save something before fire consumed everything that was there. I've been fascinated by this idea of objects and how objects hold memories. going through the experience with my father I felt like his body was fading but these objects remained and the memories remained but they became subjective over time and they shifted and they changed and they were almost like elastic. And I thought it was very interesting to just explore that and what would it be like to have kind of an onboarding element where somebody could come into something and step away from the hustle and bustle of the festival and come into a moment where they had a chance to connect with a stranger, another human being, and they had a chance to kind of reflect on something from their own life. And I think sometimes it's interesting because people just randomly choose something and then they'll realize through the course of the experience that That object that they felt that they were either a lot of people purposely choose some things but sometimes people choose something and they're not really sure why and in doing so Later, it comes full circle, you know and they realize oh I guess that was the perfect object to choose and the level of stories that come out of it are really quite amazing and it almost becomes like this really wonderful collection of emotional artifacts and And I think that there was something within the work, I feel like, even though things become more and more immersive, I think that there are certain control issues that exist within stories within the immersive space, where somebody wants to craft it so fully that every step of the way they're telling you what the story is. And it's not to say, I mean this is a very crafted experience, but it leaves room for those who are participating in it to bring something to it too. And I think that in the end, I wanted to make work that was emotionally resonant. and I think immersive work, at least in what I've been experimenting with in terms of, as you mentioned, Frankenstein AI, or a piece like Where There's Smoke. I'm fascinated at that blur of a line between something that is crafted in terms of the authorship of what the story is, and then the opportunity to leave space for others to bring their stories into that environment. And so I see that as an evolution within the work. And I think it's the desire to bring human connection into things that have become... I think that there was a phase where the technology is exciting. This piece is really about calm technology. It's about how can I try to make the technology more and more invisible within the experience. And I think an aspect of that was trying to hold a space that allowed people to tell their stories within it too, and to allow them to do it within the context of this idea of things that can be ethereal, I guess. So I think it's kind of meditative in that sense.
[00:11:45.312] Kent Bye: Yeah, I had the experience of having the question asked to me of if my house was burning down, then what object would I take? And I think the first mental thought was, oh, I'd grab my computer. But then I projected myself into like the following days and then like really checked in into my own gut intuition into like, what would I really regret of not getting? and I decided that I would grab my cat and then when I I thought about it in terms of oh I think this is what the answer is for why I chose that because it's a living being you know that's the ethical and moral thing to do or whatever but the fact that I was asked like what is my emotional attachment to this object five times over and over again started to unpack all the deeper layers of the history and the memories and the arc of my own emotional life of how much my cat has meant to me 16 years old and I've had him through so many different phases of my life and so that to me was just interesting to go from having an idea in my mind to then really checking in to like, is that really the thing that I would want? No, I would choose this instead. And then have another phase of actually unpacking it to the point where I'm like discovering all these things that are surprising, but are also the perfect thing that everybody else in the room had picked objects that had all this deep sentimental value in terms of what they represented to the memories of their own life. It's like these objects that we project onto them, these stories of our history, And that we want to preserve that history because without that we lose a part of ourselves. And so that was something that I found fascinating was the different phases of that unlocking. But just to iterate on a singular question over and over again and just keep unpacking it and going deeper and deeper.
[00:13:32.100] Lance Weiler: Yeah, it's an empathetic listening exercise, and I think what it does is it's intended to usually maybe around the first two are relatively easy, and then when you hit the third it gets a little more difficult, and then it will go deeper. But I think it's fascinating to look at the different types of collections of objects that people have chose to save, you know, and when you kind of walk in there and you start to You hear the stories that they're telling and you see like the commonality around certain objects and then you see these other objects that are really kind of these very, if you looked at it, you'd be like, oh, that's interesting. I wonder why they chose to save that and then it just has such deep emotional meaning to them. And so I think within the piece, there's this desire to try to, how do you balance those things? I think it's kind of almost like the exploration that I was personally going through in trying to understand who my father was, maybe trying to understand a bit of who I am. and then also trying to make sense of certain things within that, which is kind of interesting because it becomes this, it's like a memoir, you know, in a certain respect, and it's allowing, you know, we're kind of sitting here in this gallery space, you know, and it's filled with these photos that my father shot, and he never considered himself to be a photographer or creative, right, and they're breathtaking photos. But what's interesting when you kind of look and you say wow all of these were buried away in the closet Nobody had ever seen them before there's so much that's packed away in that that's kind of fascinating And and I guess I wanted to try to make something that was like really personal I wanted to challenge myself in a way that I hadn't before and I wanted to see if I could make something, in a way it's a grieving project, but I think it's also a way to try to look and say, okay, out of that, how can I make art from it? And is there a way that maybe this art could be used in interesting, you know, like what could come from it that maybe could create some type of an empathetic healing space for others, you know? And so I've had conversations with some of the folks at the Narrative Medicine Program at Columbia University, Through the digital storytelling lab, we often collaborate with them. And so we've been talking about interesting ways that narrative can be used and emergent technology can be used to create that empathetic kind of healing space. So I think when I look at it, being here at Tribeca Immersive, sitting on Canal Street, and having something that's kind of a standalone 1,400 square foot installation with multiple spaces within it, using the Internet of Things and comm technology, it's fascinating to kind of look and say, well, maybe in the future, What is it like to have installations like this, and what if they were up and down canal? Because we're sitting here in a space that's owned by Wall Play, or leased by Wall Play, and they have an on-canal initiative, which is like kind of a pop-up arts district. 1.8 million people walk up and down the street all the time, and what they're working to do is create space that allows anybody off the street to walk in and interact with emerging technology, to be able to experience art, and maybe that will spark a whole new generation. Thinking about that combined with what's the purpose of the work, combined with the accessibility of the work, I think is pretty exciting. And I guess I keep trying to figure out how can I make work that's more emotionally resonant. You know, that's something I'm trying to challenge myself with. Because I came out of doing a lot of genre-based stuff, and I love genre-based stuff, and there's hints of that within this piece. You know, because I grew up on, like, B-movies and film noir. That was something that I got introduced to by my dad, but yeah, I think when I think about it, it is the opportunity to put it in a space and allow people to move through it is interesting.
[00:17:39.717] Kent Bye: Well, and then the second phase of the experience, which I'd say is really the heart of the unpacking your own personal story in this project that you've been working on with your father for the last 17 years. And I see it as this blending of traditional storytelling in the sense of, you know, there's parts that I could imagine that you could just sit and watch a 20 to 40 minute video in a movie theater and watch it and you'd have an experience of that, but yet you're creating a whole context of people where they're engaging with it in a different way, they're exploring around, they're playing with objects, there's these puzzle elements that they're playing with objects and moving them around with this ambient detection. There's lots of underlying technology that is essentially this interface to be able to have these cut scenes to watch these vignettes of these stories of you, your mother, your father, the history around fire and just this whole going back through your own memories and drawing upon these oral history interviews on top of just you telling your own personal story of how this is all related to you. So I'm just curious to hear about your process of taking something that could have just been a very linear passive experience, but all the different things that you were trying to bring into this experience to make it an immersive experience.
[00:18:59.711] Lance Weiler: Yeah, well, I think that in the core of it, there's a level of investigation that's happening with it. Like, I often say that this is kind of almost like a documentary that's not linear. It's generative. It's an immersive theater experience without performers. And it's an escape room that really kind of has no escape, right? And I think what the piece is really kind of wrestling with is form. You know, there's an element of that when I think about it. I wanted to try to create something or the attempt was to try to design something that had an aesthetic, you know, and that aesthetic hopefully would evoke some type of an emotion and could I get it to be something that felt like how I felt when I was going through and trying to understand all this, you know, like it felt like at times it was a puzzle. It felt like at times it had an arc of when somebody's terminal, it has like kind of this arc where you're like, There must be an answer. There has to be a cure for this. And then you start to realize that all there is is questions. And then eventually you have to kind of let go. And so I was like, could I explore that a bit? What would that be like in terms of a mechanic or an interaction? How could I do something that would allow the order in which things play to continually change, which then would change your own interpretation of what they were, which was very akin to what it was like when I was interviewing my dad over and over again. And we were kind of, you know, like when I say over and over again, over the course of that last year, the multiple interviews that we would do, we'd revisit certain topics or he'd want to talk about something again from maybe a different perspective. And so, That notion that you might see what does that change in terms of your own perception of something when you start with? Something that's kind of about this mysterious element within the story and then you start to understand like the background of that character and And then you realize there was diagnosis and there was cancer. Well, what if it started with cancer? And then it became something about this crazy, suspicious fire. And then it became something about some childhood story. And how would those orders change your own interpretation of what that story was? Wanted to kind of explore that and see what that was like and to try to build a space The challenge here was like could I make something that has no performers in it? you know and can I allow people to actually go through it and start in a paired way with somebody and then become a team of four and what's that dynamic like when you're kind of moving through this burned-out space and you're trying to find these things and you're Smelling smoke and you're laying them down and all of a sudden something's awakening Right and to your point, it's like an interface with something. It was kind of like an interface with my father So I thought that that was an interesting challenge to kind of base a piece around that and I guess I was also really interested in How can I do it with really no? There's so much that's focused on virtual reality or augmented reality and screens, you know, like how could I take it to the most basic level? You know, it's got a slide projector in it, you know, and it's got these audio kind of stories and a nostalgia that's kind of going as the slide projectors going and so I thought that that was an interesting creative spark for it, you know, and then it started to through a lot of paper prototyping and an amazing team of collaborators just started kind of unpacking it and experimenting with it and trying it. You know, we ran prototypes early on at Lincoln Center that were all paper. We did a private preview of it at the Future of Storytelling, you know, where we built one room and we let people go through that. I'm fascinated by this idea of what's work that can continue to be iterated on, you know, as opposed to like a final finished piece. What's it like to kind of workshop something in immersive form? What does that look like? Because I think a lot of the work has tended to be so focused on completion that it doesn't give enough time to really reflect on what it is. and doesn't give enough time to really kind of poke at it and say, okay, well, what if we change the way that people interact with this, or how they come into it, or what is it like when they're leaving, how do we help them land, or what's the mechanic in the center of it, or whatever that looks like, right? You know, just the ability to kind of explore those questions, and that to me is a very exciting aspect of it.
[00:23:35.273] Kent Bye: Well, one of the things that I've seen over the course of the last four years at Sundance, I first went in 2016 and then 17, 18 and 19. And so seeing all the experiences there and seeing how when I first went, they would just have a place that you would see the experience and that was it. But more and more, there's dedicated installation spaces where you're actually walking into an entire context that is in some ways trying to serve as this liminal space or this Onboarding Magic Circle where you're starting to get a sense of this world that you're about to go into and then eventually you go into the fully veiled VR experience, but you've been prepped with a certain amount of world building that's happened before you actually get to that point of that experience or that media object and And I feel like there's something similar here with what you're doing in terms of you have these group exercises that we do, exploring memories and objects that we're connected to. But then you open up this box with a bunch of clues of very personal artifacts from your dad's life that we are allowed to sift through for like 90 seconds. And then we are walking into a dark room, there's smell of burnt wood, it's confusing, and we have to kind of go through the mental friction of finding these objects and solving a little puzzle, and then we get access to these deeper emotional stories. But I see that the whole room, the whole context, that whole experience is creating a certain context that is leading us into those deeper stories. And so I'm just curious how you were architecting and designing those elements of that experience in order to have the audience listen to these slideshows and the audio recordings that you had recorded, but have the audience go through all these other things before they get to that point and what type of experience you were trying to create from that.
[00:25:22.380] Lance Weiler: Yeah, I mean, I think in terms of it coming into a burned out space, I think it's this You know, hopefully what's happening in the first part is you're being primed and you're thinking about a space of your own and you're going through this visualization exercise. So as you make your way into the actual space in the back of your mind or consciously, you're thinking like, wow, this could be a space that what if this was a space I lived in? You know, like there's a moment of that, which I thought was interesting. And I guess in some ways, you know, I've talked with the team and we've talked about how the table that we use is really kind of a metaphor in some ways for not only the interaction that I have with my father, but you know something that's metaphorical for the complexities of like the health care system and challenges and things of that nature. So in terms of the tasks that you're doing, you know, there was this thing when my dad was sick that I found myself running through these almost like the tasks that I had to complete to keep him alive were preventing him from dying with dignity. There was something that was interesting about that mechanic and playing with that mechanic in there. Some of this is obviously very layered in terms of my own thought process around architecting. I think over time it will get closer and closer to that or it will evolve into something that maybe I haven't even fully considered yet, but that was kind of some of the idea, you know, like what would it be like if I actually handle these objects and I'm trying to figure out what the connection is between these body parts and as it goes further I start to realize that his body is literally deteriorating, you know, so there's quite a bit of metaphor within there. So I think just kind of trying to experiment with this idea of unlocking, I guess in certain ways, and unlocking of memory. And, you know, in some cases you might not get any stories about the fire and people have thought that the burned out space is metaphorical for cancer, you know, and they take a different meaning from it. So that's kind of interesting too, you know, to think about like how that space is priming in some way or creating some type of a like a response from someone but the context of it can change you know like the context might be like oh he grew up in a firefighting household oh wait he actually had house fires this is his house fire or wait maybe it's a metaphor for cancer and then at the end you come to realize like oh that's what that was So trying to leave room for discovery within it in some way, and trying to find that balance between passive and some degree of interactivity within it. But it's somewhere packed in there. I think it's with this kind of work, it emerges over time.
[00:28:12.198] Kent Bye: Yeah, one of the interviews that I just aired on my podcast with Graham Sack was talking about dream logic and he specifically was looking at Freud's interpretation of dreams and looking at how there's these very personal symbols that were in these people's dreams and that he was Really interested in those personal symbols and not creating these more general archetypal dreams that anybody could connect to but these are very specific to someone's life and I I feel like there's a bit of the language of Immersive experiences is with this kind of dream logic metaphors and symbols that are this spectrum between personal metaphor where you have a very specific meaning for what they mean for you and Versus things that are gonna be like these universal archetypes that are gonna be universally translatable that it's just clear I think fire within itself is a pretty universal archetype in terms of the burning and the destruction and the the spontaneity and excitement of the live moment that is Happening, you know, you talked about how as a kid you were going to these Fires and watching them when you were from like 7 to 12 years old and that there's a certain amount of just excitement there but there's also a big part of the destruction of that fire and You know having the materiality of our lives be burned down to a core where we could lose everything in fires but there seems to be like this spectrum of the individual personal symbol and the universal symbols and I'm just I Curious if part of the art making process of experiential design and immersive storytelling is finding that balance between what you find personally meaningful in your own symbols and hoping that it's clear enough so that people see it that they get what the story is without having a lot of explanation of what this is all about.
[00:29:59.540] Lance Weiler: Yeah, well, I think you're always trying to find some degree of balance and I think this piece is really playing with this idea where it's letting people in and they're activating their own memories within it. So, what's the line between the ambiguity, you know, and allowing people to interpret it in ways that work for them, but then also, you know, hopefully work for the whole piece. And that's an interesting kind of tension, I think, within it, you know. And I think where fire is universal, you know, it's like there's an element of, you know, it's a purging or a cleansing kind of thing at the same time. It's incredibly destructive, but then once it's all destroyed, what rises from those ashes and what could that look like or what world comes after that, which is really kind of fascinating. I think in terms of the interpretation, I'm trying to, with this piece, just explore that notion of how difficult memories can be, you know, and what you perceive them to be or what you think they are. You know, if you return to something, you hold something, you smell something, you know, and then when you start to talk to, for instance, in this case, I talked to my siblings and their interpretation of these various stories. They're like, wait, I didn't remember that. What did Dad say? you know, like, where was I? You know, this is how I remember it. And then there's all these different interpretations of it, which are really kind of fascinating. And so in this, I wanted to hopefully maybe weave that into what it was in a sense. And what is it like to create that space where I'm giving a story in order to get a story at the same time? And I think with that, you need things that could be metaphors that maybe are universal. but then also incredibly personal. So somebody walks out, and they think, oh, that's what that meant. But somebody else, they're like, no, that was very distinctly intended to be this. And so that's a balancing act. Because I think a lot of the times, the digital work or the immersive work is overwrought with a concern that somebody is going to break it, that it's over-designed. So how do you take it to the edge of where Somebody can interpret it. I mean, that's what's so wonderful about cinema. It's this form that allows you to leave and it can activate all kinds of memories for you. The sound, the image, so forth and so on. Yeah, I guess when I think about it, I didn't want to fully be prescriptive over what those things were, but they are our universal parts, but then it's ambiguous, you know, so sometimes people kind of take their own interpretation out of it. And when you make this kind of work, it comes back and you start to hear these things where you're like, wow, I didn't think about that. That was always the case when I made film or when I make film. You know, you're like, okay, no, I did it this way. And then somebody's like, I saw this other thing entirely. And that's where it's actually really exciting, I think.
[00:32:56.404] Kent Bye: Well, you'd mentioned that there was like nine different vignettes and that when you go through the main heart of the experience, you see like four of them and you have this three by three grid that is slowly going from one color to another color to indicate progress through the different stories that are being unlocked. What is the logic behind what stories get told? Is it completely random? Is it based upon your interactions you're having with the table? And what is the reasoning for why not just Show the same three or four why have so many there's some storytellers that would say well if you're not telling a linear story Then maybe you know, do you know what you're trying to say? You have this randomness there that either you're trusting that people are going to receive the exact portions of the story that they need or there could be Lack of cohesion that maybe they don't receive the things that they need because they actually May have gotten a lot more out of it with those other ones and how how as a creator Do you make sense of those types of trade-offs of doing something that's generative, but you're only gonna see a portion of the total content
[00:34:03.113] Lance Weiler: I think there's a power in the ability for it to, it's kind of like life, right? You know, it serves it up in all different orders and you hope and you're trying to make sense of it. You're trying to apply logic to it and you're like, okay, here's the beginning, middle and end. But I think when I was going through that experience with my father, it was such a roller coaster, where it was like, okay, he's in remission. It's going to be all right. And then it would come back again. And then all of a sudden, I'd be driving him to the hospital, and I would think, this is the last time I'm going to see him. And then he would go in and he was having kidney failure, and then he'd come out on the other side like a cat with nine lives, and all of a sudden his kidneys seemed like it was better than when he went in. And it was like, oh my God, I can't even make sense of this. It's not following a progression that is linear. It's jumping all over the place and at times it's kind of maddening because you're trying to make sense of it. You want there to be a logic to it. And so I felt like that was a really powerful way that I found myself going through that. Not only in terms of the memory and how the memory is unlocked and what my father's telling me, the element of what it's like to deal with somebody who's terminal and so I think that That aspect and it comes in wave grief comes in waves like all of a sudden you'll just be hit you'll smell something you'll see something or you'll hear something and it just comes at you and Right? And so I think I was really kind of fascinated by the nonlinear aspect and then it is kind of generative in the sense that it might be different for you than it is for another group of people and it's kind of dictated off of the combinations that you put down on the table and just kind of exploring that I think was really kind of fascinating. But a lot of it was really like how can I capture or try to make sense of this element of death? And I think I've been trying to understand that. There were so many very cinematic things to it, like how I found my father, the exchange I had with my mother after he passed away, and what she thought she saw. And he died on her birthday. So there were all these things that were really very, to your point before, it almost felt like a dream. It felt like it was moving from this to another thing to another thing. And so I guess I wanted to play with that. Because that, I think, is really interesting in terms of experimenting. So I think it was in that vein of really going through that experience and then saying, oh, what could be an interesting mechanic that would capture that?
[00:36:42.530] Kent Bye: Hmm. Yeah. And the table was very interesting in terms of, you know, you would pick up objects and there'd be a light that would change colors. And we had to solve a puzzle to find the objects to put them in the right places. And then we're moving them around. What is happening under the hood there in terms of what's being detected and what kind of technology did you bring together? Because it seems like it's using all these Internet of Things type of sensors, but the underlying principle of what's happening there as an interface to be able to interface with immersive media.
[00:37:12.667] Lance Weiler: Well, I can't say enough about the collaborators that I have, because I think it's just phenomenal in terms of what they bring to it. And I think the table becomes like a metaphor, and it's making use of IoT-based technology. It's using load cells so it knows whatever is put on it. So it knows, oh, that object is there on panel four. Oh, no, they moved it to panel two. Or they moved it to panel one. And that feeds into a state machine that determines, oh, OK, that combination happened. Or even when you walk in and you find some of the objects and you pick them up, the room knows. And it recognizes and it changes state. And I'm fascinated by that. This idea of ubiquitous things being able to power an experience. Like all of a sudden, what's that combination of IoT with AI? And what if we don't have to deal with these screens that are always like, look at me, look at me, look at me. And now all of a sudden it's in the real world, but it's not like I have to use glasses to see it. you know, it's seamless. And so I think playing with IoT within it and exploring that is a real center of it. You know, for a while I've been fascinated, I think back when we talked the first time, it was around Sherlock Holmes and the Internet of Things, you know, and that was in 2015. So I've been very interested in this idea of calm technology combined with machine learning or AI. Future forms of this, when we think about what an empathetic healing space could look like, there's a desire to weave AI into it. There's a desire to use it as a transformative learning experience or a simulation in some way, and being able to have data that you can collect along the way. So in the core of this, we're kind of just scratching at the surface of some of those things and creating a mechanic that is basically based on load cells and state machines, so it knows, okay, this is the combination that was triggered, and then it fires different audio or video files based on that. But the same thing is like a conditional statement. If this, then that. We could change environmental things. We could change lighting. We could do all kinds of really interesting experimentation with it.
[00:39:30.535] Kent Bye: In the third room after we go through all the story elements and go through those four little vignettes and then go into another room where there's a table and the pieces of paper where we wrote down the objects that we would take from a fire are there and we sit down in a circle and we have a little thing that we read through and instructions are to tell the story of why we chose this object which was interesting because in the first phase we had recorded the object and then we're asked five times, like, what is your emotional attachment to this object? And we're able to deep down iterate into why they're important. And then we have this whole experience of your story. And then we go back to our story, our memories. And now we're have the opportunity to just tell the story from scratch without needing to put it into any structure. We're just free form, able to synthesize all of the things that we discovered Through that first phase and to really articulate to this group of strangers what are the important parts of this object and what it means to us and then an opportunity to kind of leave it behind or to take it with us and to see the matrix of all the other objects that people had Thought about and so I'm just curious to hear a little bit about that phase and what you were trying to achieve with that
[00:40:48.390] Lance Weiler: Well, I think when it comes to the clothes, I thought it would be really interesting. You know, you started in a paired activity and then you became a team of four and it's almost like you have this chance to dig down into why you would select that object. And I thought it would be really interesting to kind of have it returned to kind of almost a social kind of thing that was four people. So there's somebody there that has heard the story, but then there are two people who have not, you know, and then you're kind of revisiting it. Maybe through a different lens as you sit there, you know after having gone through that experience in the second part You know where you're interacting with the table and the IOT elements And I thought it would be nice to kind of just bring it back to like a shared moment You know and the music's kind of playing and now you're hearing these very succinct kind of stories as to why these things are important and sometimes or what I've heard as feedback is somebody said well, you know I kind of grabbed this object in the first part and I was like Oh, is this really the thing and then all of a sudden they get to the end and they're like, oh, that's why That's why I grabbed that object. I didn't even realize that there was that depth to it, you know And so I think that there is a nice thing in that closing moment where you bring it back around to a shared story everybody's sharing but it's like a collective story and then it kind of ends with like a beat where people are hanging those things up and you realize that everything on the walls are these emotional artifacts and then you kind of end with that letter that's talking about my collaboration with my dad around the project and it just felt like a nice way to Land the project, you know like to allow somebody it's like I'm always trying to think about like how do you bring them into something and how can you carry them through what it is and then allow them to settle back down and then make their way out of what an experience is. And I felt like there was a nice shared moment there, you know, especially I could tell when I could just hear it and I could hear the stories and I could hear that with the music that Peter English did this really amazing score for the work. And that part, that final piece is just so wonderful as you hear the stories from everyone, you know, and so I really, it just felt right.
[00:43:03.213] Kent Bye: Hmm. Yeah. And after we closed out and hung up our cards up on the wall, then there's a light that comes on. We see that, Oh, now there's another place that we can go into. And there's this big, long black corridor with a big, bright light at the end with lots of slides from your father that we walk down and see the slides. And we walk into this art exhibition, which you said that a lot of these photos and no one has ever seen before. And that this is kind of like the first art showing of a lot of the work that your dad Had done from his life of going out as a amateur photographer and listening to the scanner and taking all these photos of fires And that's the room we're sitting in right now It's just all these photos of fires some of which that you were at and but just beautiful photos and so You happen to be here to be the docent and to tell the additional stories or answer any questions. You're not always going to be able to be in this room when people experience this, but maybe you could talk about the final exit as people are coming out of this experience and into this room of the art of your father.
[00:44:06.931] Lance Weiler: Yeah, I was really interested in because a lot of these images I found after he died and so people hadn't seen I hadn't seen and I thought what if I could give him the art show he never had and By coming down the dark long hallway and then just having a bank of lights, but it's consists of all these slides I just felt like it would be nice as a way to kind of move out of that towards the light You know like a passage And I felt like just having space that you could see the work presented in a way, it just blows my mind, the quality and the artistic nature of these things, and to think, wow, they might not have ever been seen. I think that there was an element when my dad, he had gotten contacted to go and show some fire slides that he had. And he was very excited about it. And this was towards the end of his life. And there was a fire company that asked him to come to an event that they had. By that time he was pretty weak, but he decided he was going to go because he made the commitment. And he went and he showed up there and started projecting this stuff. And, you know, there was a disconnect. You know, there were a lot of younger people that were there and some of the fires that he was talking about, they couldn't relate to. And so I remember him coming back and kind of just being He was glad that he went and did it, but I could tell he felt kind of defeated about it. It was like, I showed my work, but I don't know if they fully got it, or it was just a different time. And so I thought about that, and I thought, wow, what if he could have a different context for it? Like, here we are. It's a little gallery show in New York. You know what I mean? Which is really wonderful for somebody that was very much, I consider him to be an outside artist. Because when you look at it, there's amazing composition to them, you know, like look at that You know, I'm pointing to a photo that has a fire in the attic. It's glowing like really hot red and there's Firefighters with white helmets and there's a perfect triangulation, you know There was just a natural composition that he brought to the work and it's just so funny to me or I guess in some ways ironic that he never considered himself really to be artistic because all these shots are incredibly artistic and So it just felt like a nice way to close it out and to have it move from you start with your own story, you move into a space that's my father and I's story, you move into a space where you share it, and then it ends in the space that's his. And I just thought it would be nice to do that.
[00:46:43.360] Kent Bye: Yeah, and I'm just curious to hear about your process of doing a piece that is this depth of emotional vulnerability, but also grief and the different phases of grief that comes in waves. And you feel like this level of artistic expression through these immersive storytelling, immersive experiential design types of experiences is allowing you to get in touch with your own depth of emotional presence and grieving process of the loss of your father.
[00:47:13.287] Lance Weiler: Yeah, well, that's a really, that's a hard question. You know, like I think, I think it's an exploration. You know, like I think it's something that kind of just comes, like it's interesting each time that we kind of run through it and I hear it and maybe I hear it in a different way or I think maybe I'm still searching for certain things around it and trying to find meaning in it and there's quite a bit that's kind of packed into it. So I think each time it's like a little bit of a a layer kind of maybe pulls back, or I think it does, and then it leads to a whole bunch of others. So, I think when I think about it from a grief perspective, I guess I've always been somebody who creates things. I think I process the world through making work. And so I think I'm processing the loss of my father by making work where I'm kind of collaborating with him, I guess, in a weird way. And maybe it's trying to tell a story that allows him to live on in some way, I guess. I think that there's probably quite a bit tucked into that that I haven't gotten to yet, but I think there's something powerful about being able to make art and to just be able to explore that space. And for me, as I had said at the top, it was like I wanted to explore that emotional space. I wanted to make something that would allow me to not only pay homage to my father, but to also try to understand more deeply a sense of identity, I guess, or just the ability to say, how can I make something that really has truth in it? you know, cuts through the bullshit and just makes something that has truth. And I think that I'm trying to do that, you know, just exploring it. And if it's something that goes forward and is able to help some other people with it, that's great. But I think it's very much processing it all. and trying to figure it out. I'm excited my mom's going to come and see it, which will be really kind of amazing. Because she hasn't really talked about any of it at all. Not the project, but the loss of her husband or my father. Just really doesn't talk about it. So that's going to be really powerful when she comes. She hadn't really mentioned it until just this now opening. She started to become a little more inquisitive about it. Started to ask if she could come and see it, you know So I was just waiting for her to be ready.
[00:49:46.291] Kent Bye: So I think that that'll be really powerful And and finally, what do you think the ultimate potential of Immersive storytelling is and what am I people to enable?
[00:50:02.761] Lance Weiler: Well, hopefully maybe in some way it can lead to a greater sense of human connection, you know, I think Right now and this is something that we explored in Frankenstein AI you know this idea of isolation and connection we have these devices and we feel as though we're more connected, but are we really and What is it like to really kind of sit down and truly connect to another human being? you know, let alone a stranger. And so I'm hopeful that maybe this piece over time will not only help me and my family to process the grief of what we're going through, but maybe it can make its way into, you know, becoming some type of a healing space or using immersive technology as a way to bring people together to tackle complex problems and to allow us to maybe reconnect with what it truly means to be human. So I don't know maybe I hope those aren't grandiose ideas of what immersive storytelling can do but I think it's in its infancy and There's ultimate potential it can go so many different directions, but I think that there's a great degree of responsibility You know and because it's embodied storytelling and so I think with that you need to take care and you need to be ethically responsible So I I don't know I think in some ways Storytelling can an immersive storytelling can maybe help to tackle some of those complex challenges that we face Yeah, thank you so much for joining me today on the podcast, so thank you Yeah, I feel it's like my I think my third time I'm a it's a trifecta. So thank you.
[00:51:42.496] Kent Bye: Thank you for having me back So that was Lance Weiler. He's the director of where there's smoke which premiered at Tribeca immersive in April of 2019 So I've a number of different takeaways about this interview is that first of all? Well, there's a lot of different components that are in this piece that have shown up in the other iterations. In this piece, it was much more of an escape room mechanic with a group of four people that were going through and were asked these different questions in terms of what object would we save if we were in a fire. And those same types of questions are actually repeated in other versions and iterations of this piece, especially with the latest version that I saw at PAMCUT, which is the Portland Art Museum's Center for Untold Tomorrow, back in November 17th, 2023. And I'll be diving into that version into the next episode. I did actually have a chance to talk with Lance after the Ifadoc Lab version. However, I can't find a recording of that as it was done under the auspices of the Ifadoc lab, kind of a live session that I just didn't get the recording of that, unfortunately. But in that version, it was a real interactive version that was using a lot of the online components of Miro, and there's elements of that Miro board that he's using in the next version. So in the next interview, Lance is reflecting back on this initial version. And one of the comments that he said was that there are a lot of the different interactive components that were in some ways metaphorically reflecting the way that he was still at the very early processes of grieving. And so it was a lot of that intellectualization of that process. But there's also a lot of different core components that I think have repeated throughout the course of this piece, throughout many different iterations that he's had, including all the different testimony that he has from his father and these deeper questions around not only trying to figure out what is important in our lives and our identity, but the story that he has with his own relationship with his father. We'll be diving into more of that in the next episode, which is much more how to translate this into more of a cinematic experience, but yet do it in a generative way that is able to be live and dynamic, but also involve the audience in this collective grieving ritual to explore and really reflect upon life and our mortality and our grieving processes. So lots of really fascinating seeds of this project that started with the version that showed at Tribeca Immersive and Lance is someone who is constantly iterating and prototyping and so we'll hear in the next episode more of the journey of where he's taken this project over the last four plus years. So, that's all I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.