StoryFile is an interactive storytelling platform that allows you to interact with someone as they tell their story. A comprehensive oral history could be as many as 1000 questions and 20 hours of content, and StoryFile’s combination of a conversational interface with natural language processing to categorize the content provides new interactive ways of accessing these oral histores
I talked with the StoryFile founder Stephen Smith about his Ph.D. on how how video has changed Holocaust testimony, the evolution of this tech platform, his involvement with capturing volumetric testimony of Holocaust survivors in The Last Goodbye, and how oral history helps to ground the broader historical context through the lens of an individual’s character, stories, and experiences.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. So continuing on my series of looking at the VR for Good movement, today's episode, start to move into the fourth part of these different interviews. And I'm starting to look at some of the projects that have different architectural or technological innovations. And so what are the ways in that they're actually producing it that actually creates new opportunities to be able to tell stories. And so Today's episode's with Stephen Smith, and so he's a founder and board member of Storyfile. So I first saw the technology for Storyfile at the USCICT, that's the University of Southern California Institute for Creative Technologies. They had an open house during the 2017 IEEE VR academic VR conference. So Storyfile, you're able to capture an oral history of somebody. So you'd be able to do these whole like long oral history interviews, and then you have this conversational interface where you're able to ask this subject who's been recorded a question and then through artificial intelligence, natural language processing, they're able to figure out what your question was and to be able to match up this massive archive and to be able to deliver you the answer. So trying to mimic this conversational interface with an oral history. So in the future, maybe we'll start to get into virtual beings and artificial agents and Have artificial general intelligence to be able to have these dynamic interactions with characters but for right now just starting with the more documentary side to be able to see how you can capture someone's actual life and To provide a portal for you to be able to actually, you know, have a conversation. So story file has created different technology They've created a whole app where you can either see somebody that's been captured either in volumetric spatial experience of some of these interviews or if it's just 2d and so you're just having these conversational interfaces and so they were demoing some of their latest iterations at the Impact Reality Summit in Seattle. And they were part of the pitch process that was there as well. And I figured I'd be able to catch up with one of the founders and board members, Stephen Smith, to be able to unpack the technology and his whole journey into this whole immersive space. So, that's what we'll be covering on today's episode of the Wastes of VR podcast. So, this interview with Stephen happened on Friday, January 10th, 2020 at the Impact Reality Summit in Seattle, Washington. So, with that, let's go ahead and dive right in.
[00:02:26.151] Stephen Smith: My name is Stephen Smith. I'm a founder and board member of Storyfile, which is an interactive storytelling platform which allows you to converse with people who tell their own story, be that about some area of their life or some area of expertise. And that combines both regular 2D capture but also volumetric capture for use in XR.
[00:02:49.915] Kent Bye: So maybe you could give me a bit more context as to your background and your journey into immersive technologies.
[00:02:55.447] Stephen Smith: So I wrote a PhD in England 25 years ago on new media and testimony. I was interested at the time about the democratization of video and how that would change the way in which history was documented. And I looked specifically at how Holocaust survivors were changing or developing their narrative when video was available for them to tell their story. And what's obviously happened in the meantime is that the availability of video has become ubiquitous and fully democratized to the extent that we all have these devices that are on us at any given moment of the day. But what I'm really now interested in is, so what do the new XR technologies allow us to do in terms of telling human stories? So that could be a VR project, like I was involved in one called The Last Goodbye, where we took a Holocaust survivor to the death camp of Majdanek in Poland. And this individual, Pinchas Guter, would take teachers there and school students there, but he told me he was going for the last time. so it was directed by Gabor Aurora and Ari Pallertz. We went with Pinchas Guter to that site and then he told those same stories in their same places so that that experience of visiting the camp with him would then live on past his own lifetime. In a similar way, Dimensions in Testimony, which was a project which I was involved in producing, where we interviewed Holocaust survivors volumetrically, we would capture them at the Institute of Creative Technologies at USC, film them in 360 with up to 116 cameras, where the volumetric capture obviously, unlike a 360 camera looking out, was cameras looking into the subject. We would capture then up to a week's worth of interview content with the subject. to allow then a virtual conversation to be able to take place. They would answer questions, the questions get databased and then using natural language processing you engage with the interview, which you can then display either on a flat screen or on a holographic Pepper's Ghost or in VR or AR.
[00:04:57.949] Kent Bye: Yeah, well, let's maybe take a step back with this process of testimony and bearing witness to testimony. There's a certain element of truth and reconciliation that happens of listening to someone tell their story. So what is it about that that is so powerful in terms of not only healing for the person to be able to articulate their story, but also for people to bear witness to that story and to understand the deeper context of the past and the history?
[00:05:23.123] Stephen Smith: Well, I think that one of the aspects of, if one uses the word bearing witness, is for the benefit of the witnessor. Somebody who's been through a traumatic event, one of the ways of healing is to be able to talk about that event. Sometimes that's extremely private and it needs to be done in a context which is not public at all. But there is also context in which sharing one's story with a wider community actually can be beneficial to the user in terms of coming to terms with their past, enabling or feeling the sense of empowerment in being able to perhaps prevent others from experiencing the same thing, so that you're participating and contributing. Some of it's about passing on the legacy to make sure that future generations know. Another part of it is about education and informing others about what happened in a given period of time. So an example like the Holocaust is really powerful, obviously, because there you have individuals who've gone through the most traumatic experience you could possibly imagine, who later in their lives have come to terms with that experience enough that maybe earlier would have been difficult, but now can start to articulate for a variety of different audiences their experiences. This has application very broadly and universally, where an organisation like Storyfile comes in, where actually all of us have different experiences in our lives that might be relevant at a family level, for example, where just sharing family stories is meaningful. My father, for example, who now has Alzheimer's, when he was younger, had TB as a child, and then both of his parents died. So he was orphaned and had TB. Now, this is something he never talked about. He's now 83 years old and with the onset of Alzheimer's I used the Storyfile application and interviewed my father. In fact my daughter interviewed my father with me there. And so I heard his story for the very first time because he felt that at this moment in his life he could share in a way that would be meaningful for his granddaughter and something important to leave for the future. So I think it has very wide applications.
[00:07:27.049] Kent Bye: So let's talk a bit about the structure of these oral history interviews because in order to capture the story file you have a whole range of questions that you have because you're essentially asking people to ask open-ended questions with an astronaut or with a people who are Holocaust survivors. So then you think about that context and you're like, oh, this is what people would maybe want to listen to stories about that. In this demo that you have here, you have sample questions that were very helpful because it's like, okay, well, I know that we're going to answer, that'll help be a little bit of a guidepost. But you have this range of approaches where you can do just open-ended inquiry, where you can ask whatever you want. You can either have an answer to that or not, or have other ways of guiding people through the larger arc of the story. So I'm just curious how you approach that.
[00:08:12.544] Stephen Smith: So the way we approach it is, first of all, we get to know the subject very well. We interview them. If they've already done an oral history interview, we will listen to that. If they've written a book, we'll read that. If there's a documentary about them, everything that we can find out about them, we research deeply. And then we go to them again and say, OK, we've discovered these things about you. Can you please tell us a little bit more and clarify X, Y, Z? So what you end up with is a deep knowledge of that individual and his or her life arc. From that we create a script, and that script consists of several elements. One element is what we call subject-specific content. That's where we've taken the very specific things that they have told us or that we've learned about them and we ask specific questions about them. So let's just take, for example, Lawson Sakai. He's a 97-year-old Japanese-American veteran of World War II, and we were interviewing him for the Japanese-American National American Museum. So, we didn't know anything about Lawson to start with and there was not that much published about him. But we were able to, by interviewing him, get to the bottom of his story and find out which division he was in and where he was and so forth. So now when you approach that story file of Lawson, which you'll be able to do in the Japanese American Museum shortly, you can come at it very open-ended, but basically there'll be some form of bio which will give you a way in. Each of the individuals gives a short version of their story. So in five minutes, give us a summary of your story. So then what you can get is the context and the main sort of story arc. That usually then raises questions in people's minds. Oh, you weren't in a detention center, why not? For example, if he says he wasn't in a detention center. What happened to you? How did you get into the army? Why weren't you in a detention center? Whatever. So, then what happens is, as the interview goes on, we also then ask them generic questions, simple things like, where were you born, and where were you born, that would apply to everybody. And then we ask them a series of off-topic questions. For example, if I say to them, you know, who is the President of the United States? Well, in five years time, Lawson's not going to know the answer to that, and Lawson, I hope, will still be alive, but may not be. And so therefore he's not going to be able to give the answer to that. So he'll say, I don't have an answer to that question, or I don't know, or one of the several topics. Because the system has to be able to determine whether it has an answer to a question that's being asked, or it doesn't. And it has to be intelligent enough to know when it doesn't, otherwise it's going to give a false answer. So that's part of what the interview process allows for, is to be able to take these different branching approaches. The final thing we do is, if a question is asked, and he gives an answer, but it leads to a next question. We then do what we call a pick-up question. So let's just say, in fact, Lawson did answer the question about whether he was in a detention centre, and he said, no, he wasn't, but he went to visit one. So the question would be, well, when did you go to visit one? Why did you go to visit one? What happened when you got there? So those pickups are really important because in a natural conversation, you would naturally want to know, well, if you weren't sent to one, but you went to one, why were you there? So that's how we try to perceive what the pickup question might be.
[00:11:22.750] Kent Bye: Yeah, in this demo that's here at the Impact Reality Summit, I saw that as I'm asking the question, I see a real-time transcription of what I'm saying, which I found helpful because I could know immediately as to whether or not I was getting what I was actually saying be properly translated. And then sometimes it would be close enough to a question that I was being prompted by to actually activate the answer. But other times, it was like it wasn't exactly right. But this seems to be one of the biggest evolving aspects of the natural language processing and the parsing human speaking and being able to actually translate that into this database that you've collected. So maybe you could just talk a bit about what you've had to do with the natural language processing and AI in this piece.
[00:12:03.369] Stephen Smith: I think for me as a non-technical person, the main discovery was that artificial intelligence is very artificial and not so intelligent. and that it does require a lot of programming and a lot of thinking through as a program team. The brilliance of this is, of course, what's happening is the machine learning is improving the system as more and more data goes into it. But it has to know what the subject matter is in the first instance to be able to be successful in finding it. So the biggest challenge there is when we do the interview, what happens is we have a question And that question becomes the file name, effectively, of the answer. But the real data is in the answer, it's not in the question. And we don't know what questions the people are going to ask. But what we end up with is, let's just say, 500 clips, all of which are transcribed and have content in them, data in them, that relate to a range of subjects that may or may not relate to the original question. So, for a straightforward question like, what is your name? We know the answer to that will be found in the original file name, what is your name? Because I'm going to say Stephen Smith is my name, so that's an easy match. What's more difficult is if I said, what did your parents call you? I might say, well, my name's Stephen Smith, but there's an interesting story behind Stephen. because it relates to a relative, or it relates to somebody that they admired, or a film star at the time. So now the question's got a bunch of different things in it, which then the natural language processor needs to be able to work out, well, is that a relevant answer to the question, what is your name? So, what we have to do, particularly with the interviews that have got a lot of content, where you don't have that one-to-one correlation between question and answer, is that we have to train the answers. So we have to think about what are the various permutations of questions that might be asked that would get to that answer. So, for example, the question to, what is your name, or what did your parents call you, might have a reference to somebody I was named after. But we may not have perceived the question, were you named after anybody? The system has to be intelligent enough to know if I ask the question, were you named after anybody? It can find that in that data set. Sometimes it will. Sometimes it needs some training. So what happens is, in a live situation, we're collecting every single interaction between the Storyfile users, wherever they are in the world, and the system. And our database is checking on a real-time basis whether or not it gave a correct answer. And if not, it's flagging it and letting us know, you may need to take a look at this. And then that training then will apply to every other question of the same type. So it is developing.
[00:14:52.494] Kent Bye: And so on average, how many hours for each interview, once you capture it, how big is it and how long is it?
[00:15:00.248] Stephen Smith: Well, so a basic, what we call a what's your story questionnaire is like 25 questions, maybe 30 questions. And you can, in those sort of 30 questions, get a basic bio down and you can answer that in about an hour and have a very interesting, dynamic, personal profile somewhere between 25 and 35 questions, as long as you know what those questions are. So what we do there is we give a hint on the app as to what questions you might want to ask, because you're not going to be able to be fully wide ranging about that person's life. They're fairly big buckets, you know, tell me about your childhood and so on. Once you get to, say, something like a five-day interview, generally speaking we're interviewing people about 200 questions a day, it might go as many as 250, so you're going to be getting north of a thousand questions. And then what you do, you get down to some really fine detail. So you're able to then sort of dig into the depths, like with Lawson's interview about what it meant to be in that division and issues around race at the time and how it felt to be a Japanese-American in a Japanese unit. Did that give you a sense of pride or did you feel excluded? Things that you wouldn't normally get to ask an individual, you're able to get to those really sort of fine points, which is great because then when the interviewer comes along, the user comes along, They can be very wide-ranging and you can get a very natural conversation.
[00:16:23.111] Kent Bye: And how long does it take to answer a thousand questions? How much content does that end up being?
[00:16:28.129] Stephen Smith: you'll be around 18 to 20 hours of content at that point. So they're very in-depth oral histories. So if you think about this, this could be conceived as being a sort of a, you know, a simple process for interviewing an individual about their lives. What you actually end up with is this sort of mega oral history because you can probe every single corner from multiple directions and really get to the bottom of somebody's life. It's amazing.
[00:16:58.291] Kent Bye: I've sort of taken on what I do with the voices of VR as like a real-time oral history, but it's less of a biography of an individual and more of a prospography of an entire community. So it's like 1,500 interviews and hundreds of hours of content, but looking at through the lens of the topic of virtual reality and also the creators that are making it. But I'm not doing 20 hours worth of interviews with each individual, but it's sort of more of a community. But it was at Tribeca last year, there was a piece that was using these types of oral histories and starting to translate them into spatial stories. And so I feel like there's something about podcasting, the rise of podcasting, and the need to really dive deep into this that I feel like that's going to be maybe a resurgence in oral history as a medium because it hasn't been really good ways of consuming that. And it feels like this is a way to take new recordings of oral history, but potentially go into archives of existing oral histories and start to translate them into this format. So that becomes more of a conversational interface that people can have access to it.
[00:17:58.384] Stephen Smith: Yeah, I think it's going to be very possible to take existing oral histories and parse out from the original history segments that answer particular questions, so it can still be voice activated and more interactive. What's going to be great about the forthcoming version of the Storyfile app, which will come out in about May of this year, is that you're going to be able to go and take your own oral histories directly on the app. and there'll be pre-curated library of questions so that if you're going to go and talk to your grandpa about the Korean War there'll be a Korean War script so you actually know then how to navigate that particular period of history as an oral historian but you're going to be using your phone application to interview grandpa and then get all of those interviews down, those segments down, so that in future his great-great-grandchildren can then come and then ask him questions about the Korean War, for example, if you're using that script. So it's going to be an interesting crowdsourcing oral history application and it'll be available very soon.
[00:19:02.071] Kent Bye: So while you were doing your PhD on testimony, did you dive into studying oral histories and like the theory of oral history?
[00:19:10.035] Stephen Smith: Yeah, very much so, actually. And what I did, I looked at, specifically I looked at the way in which testimony of Holocaust survivors had developed over time. So, in fact, they were filmed as early as 1946. There was an oral history project by a guy called David Boda, who took a wire audio recording device from the Institute of Technology in Illinois, and went and interviewed Holocaust survivors in the DP camps, in the displaced persons camps, while they were still refugees. And that was sort of a remarkable example of taking testimony in real time, because effectively they were still living this experience at that point. through to how the arc of narrative has changed over time in relationship to technology. And there's no doubt that the ubiquity of technology and the accessibility of recording devices of all kinds, starting with audio devices and then right through now to the 360 cameras that we can all get our hands on, has changed the way in which narrative itself is emerging within our society. Because I think there is this sort of understanding that social media has meant everyone's got 15 minutes or 15 seconds of fame and we all think that we're famous and so forth. But I think there is a deeper need to be able to tell one's life story and there's a greater awareness that the trajectories of our lives do have meaning not only for ourselves but for others. And so I think oral history has developed more from a perspective of tell me about the event to tell me about what that event means to you and to the rest of us. So they've become sort of the interlocutors between their own history and the world in which we live. I think that's a really interesting development over the years.
[00:21:00.243] Kent Bye: Well, you also mentioned that you've been collaborating with the Shoah Foundation and doing other projects using 360 video and oral history. Maybe you could go into a little bit more details as to those other oral history projects that you're doing with VR.
[00:21:12.764] Stephen Smith: Yeah, well, one of the things I'm really interested in is the relationship between VR and location. You know, the history of our lives unfolds in places and, you know, we've become very mobile as a species and we start our lives in one place and we end in another and, you know, it's sort of a very much a kind of a trajectory of places and people and movement. When histories like, for example, the Holocaust occur, we tend to see them retrospectively, through the lens of history, with the benefit of hindsight. But for the person that's going through it, that history unfolds in very local places. They don't have the benefit of history. They are living through a very confusing, very frightening, anxiety-raising, horrific experience. And those places have very significant meaning in those people's lives. So one of the projects I'm involved in with the Show Foundation is taking people who survived the Holocaust back to the places that they were in, their homes, the schools they went to, the ghetto that they were in, the concentration camp they were in, the site where they were liberated, and actually filming them in each of those locations. Not as a VR project, like not as a produced film piece, but actually as VR oral history. So we have a 360 camera on the tripod, we also have a 4K camera, a flat camera on the tripod, and we do photogrammetry of the same environment. So we're capturing the individual and capturing the environment. And then we ask the individual just to talk about that place. And then what they'll do, they'll tell stories. This is my house and I used to go down here for my school. And now you're seeing them immersed in the environment in which those events happened. And then we collect them. In some cases, I think we've been to 25 locations with one individual in three countries. following the whole trajectory of their story. And then we're placing that whole interview sequence into an archive, so that effectively what you have is this 360 volumetric photogrammetry archive of their life. The idea being then is that you can then render that into a variety of different platforms, depending on what the need is. So it can be used in a VR context, or it can be used in an AR app on a phone, so that you can go to that street, stand there, and then use the... Because we're collecting depth of the character as well, we can then cut the character out from the background, and now what you've got is an AR version of the individual standing on the location, because we also geolocate the film. So those are sort of new developments in oral history that I'm working on, which I think will have implications, you know, for not just horrific experiences like the Holocaust, but for all kinds of human history that we'd like to be able to pursue more.
[00:24:00.603] Kent Bye: Well, I imagine by looking at The Last Goodbye and with Pincus, who's been able to go to these locations, but there seems to be something about memory when we're in a place. Like, when we're in that place, we tune in to our lives and we're able to be much more rich about the stories that we tell because there's almost like they're... the whole environment, the architecture of the space just invokes a whole flood of memory. So, imagine it's really powerful to go to those locations and you're able to get a whole other new level of depth of the story. I don't know if you can sort of validate that or characterize what is actually happening there, but it seems to be a strong connection between location and memory.
[00:24:38.666] Stephen Smith: So, for The Last Goodbye, I was producing The Last Goodbye and Gabor Arora and Ari Pallitz were directing it. And so one of the questions we had between me as producer and them as directors is sort of where does the boundary lie between the production of the film and the individual as a living embodiment of the history? And how do you not encroach upon his experience as an individual in the place while you're making a film of him in that place? And what was very interesting was, you know, we filmed him against a green screen inside this barracks at the very place where he was parted from his father. Now, I was with him 10 years previously when he went there for the first time in his life, subsequent to the experience. So I knew what it was like for him to go through the emotional experience of rediscovering that place and what it meant to stand there. And my concern as a producer was, well, will all the green screen and the stereo cameras and the lights and the microphone booms and everything else take away from the emotion of being there? Actually, it didn't. because all of that paraphernalia, all that sort of equipment sort of just melted away in the moment of him walking into that space and standing in front of the camera. The green screens behind him, you can't see that. All he just looks, he's looking past the camera in front of him to the barracks where he lost his father and all those emotions just came flooding back exactly as they had done. So as the producer of that, and as an oral historian, that made me feel comfortable that we were not putting him in a place where he had to perform for a camera, effectively, something that was deeply embedded in his soul. And it seemed to me that actually it was very possible for him to be present with those emotions, notwithstanding the fact he was being filmed at the same time. But there is a fine line and I think what I'm cautious about as an oral historian is in our need to and want to represent the people's lives well. As producers or as directors, creative directors, we want to get the best out of their so-called performance. We want those emotional moments, we want that connectivity So, actually, I think the best way to do that is not to over-direct, but actually to keep it as simple as possible, as cut-back as possible, so that they can relive that experience. And that's a hard balance to strike, because you've still got to have your equipment, you know.
[00:27:07.710] Kent Bye: So, for you, what type of immersive experiences do you want to have?
[00:27:14.618] Stephen Smith: I think I want the immersive experiences that connect me more closely to the lives of other people. I was just watching Daughters of Chibok right now and I think that's a VR piece that So it does, I would say it's oral history done really well insofar as it brings you into the life and the home of the women who've been affected and you have no idea that these women are the women whose children had been taken away by Boko Haram until about seven or eight minutes into the piece. And by that time you're sort of in their home and in their kitchen and on their farm and you're feeling them as a human being. So then when you realize that they've been so drastically impacted by this horrific terrorist event, then you have the level of empathy with them because you feel like you know them already. So for me, I like the kind of immersive experience that can take me into the lives of real people and open up the meaning of their life in a way that I maybe hadn't understood before. And I think VR has enabled us to do that. I also think that the range of technologies that fit into the broad category of XR, whether it's VR or it's AR and so forth, I think they're becoming more blended now as the technologies progress and our way of thinking about how to tell the stories is developing with these new technologies. I think that they offer similar and different ways into the lives of people around us and that's what I'm most interested in.
[00:28:45.527] Kent Bye: And so for you, what are some of the either biggest open questions you're trying to answer or open problems you're trying to solve?
[00:28:54.695] Stephen Smith: I think how do we harness technology to tell the really best stories that we can? I think there is a danger that technological advances are getting in the way of our ability to tell good stories, so that we're more focused on the tech, because we have to improve it. So we're pushing that, and we want to push it, and we want it to be better. But I think just be in the story, and then allow the technology to deliver that for us, rather than being worried about the technology and then stuffing our story, trying to pretzel our story around the technology. And I think that's a challenge for us right now. I think some people are being highly successful with these new media. And I think some of us are struggling a little bit, that we're sort of doing it for the sake of doing it. I think we should never do it for the sake of what the technology can do for us, but rather do it to ensure that we are communicating better and better with our audiences and harnessing technology to support us with that.
[00:29:58.471] Kent Bye: Great. And finally, what do you think the ultimate potential of immersive technologies and immersive storytelling might be and what it might be able to enable?
[00:30:08.609] Stephen Smith: no question, to bring us closer together as people. If I was hoping for something, for a story for our platform, it would be that more people get to know each other as we are, to see that we're not so different from each other, to really develop empathy, not just for terrible situations in the world, but for just great learning from other human beings, because basically all of our knowledge and experience and wisdom that we have on this planet comes from other people. It's not coming from the ocean, and it's not coming from trees, and it's not coming that we can learn a lot from them, don't get me wrong. It's coming from people who care about those things, or people that have explored the planets, or whatever it might be. Just learn to love other people and what they've got to offer, and be curious about their lives. If we can put our level of curiosity up, there's so much to learn from each other. That's what I would like to see happen.
[00:31:01.591] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the immersive community?
[00:31:06.787] Stephen Smith: I think I would say is that, you know, I come from this from a totally non-technical background. I'm a theologian, for goodness sake. So, you know, I started my career learning Hebrew and Greek and whatever, you know, and here I am. very much a part of the immersive community. And that's because I've been on this journey of discovery myself that's not been led by technology, but it's been really led by how do I tell meaningful stories in a way that can connect other people. Keep doing that. I know most of this community is doing that. And I am always inspired by the amazing projects I see, including at events like Impact Reality. Just amazing, amazing projects. Keep telling those stories, because it's the best way to connect to each other.
[00:31:52.486] Kent Bye: Awesome. Great. Well, thanks for all the work that you're doing here with your project, Storyfile. And yeah, just thanks for taking the time to chat with me today. So thank you. Thank you so much. Thank you. So that was Stephen Smith. He's a founder and board member of Storyfile, which is an interactive storytelling platform that allows you to converse with somebody as they tell their story. So I have a number of different takeaways about this interview is that first of all, well, I think there's huge potential for where this can go in the future. When I went to the doc lab, I talked to Kai Messenberg of Arte, and he said that he was looking to Miche, who was a French media theorist, and he said that a medium is the distribution of content and the content itself. So this is in some ways a new distribution platform. So you have oral histories, long interviews. It sounds like they do as long as like a thousand interviews. That's like 20 hours of different content. So you imagine that's like a super deep dive into some individual character. And if you really want to explore and have an open-ended conversation, can they start to mimic what it's like to have a conversation with people about the biggest parts of their lives? And for people who are astronauts or who are survivors of the Holocaust, there's these big moments in their life that obviously people have a range of different contexts and questions that they would want to ask them around. So it sounds like Stephen got his PhD, started originally looking at how survivors of the Holocaust were using these new communication mediums of like film and video to be able to start to capture testimony. And, you know, the evolution of how do you capture testimony in these new emerging mediums? Testimony VR is another great project that starts to go into using the whole spatial affordances of virtual reality to be able to create a context for you to listen to the testimony of survivors of sexual assault. And The Last Goodbye was another piece that I interviewed Gaba Herrera back at Tribeca 2017. That was in episode 573 where we start to go into Pincus, one of the people that was featured both in the USCICT original versions of Storyfile, but also going back into this concentration camp and going around and recounting different memories, you know, actually in the places and capturing the testimony as they're in those places. And it sounds like Steven's actually doing that a lot with people from the Shoah Foundation and other survivors and kind of tracing the evolution of their lives through different locations of where they went from one place to another. And then as they went into the concentration camps and then beyond. So trying to really capture this rich volumetric aspect of people's stories. And so I think that's a huge impact because, you know, when you're in locations and places, it starts to invoke very specific memories and emotions that you may not be able to, to fully invoke. If you aren't actually there, maybe even virtual reality in the future, we'd be able to do photogrammetry and be able to take people back there virtually. But I think there's still something about actually being there that allows you to kind of tune into different aspects of your life and your story, because you, you have to actually get there. And so you're, you're traveling through all those other aspects of the context to get to these different points. So to kind of break out to what's this mean for the future of conversational interfaces? Well, I think it's going to be like the foundation for where virtual beings and virtual interaction goes in the future. You know, if we start to look at the spectrum of storytelling between the authored stories versus the generative stories, where the generative stories is completely manufactured, you start to then, rather than writing a whole script and the whole story, you start to then, you know, if you're talking about virtual beings, create more of personalities that can then construct different aspects of a story in history. But in this case, you're actually using somebody's actual life. And you're trying to create this access to be able to navigate 20 hours of content, because there's not a lot of people that want to watch all that content straight through being able to engage and to see what you're actually interested in. You're able to maybe go off into these different branches. And so just this idea that you could start to operate at this scale, to be able to capture oral histories and really be comprehensive and to be able to use it on the app and to be able to give access to these different stories. And I think Steven said this very interesting things, which is that, you know, it's. It's not just that the people's story doesn't matter, but it's actually like these stories are actually allowing us to connect to the larger context of history. And just by having this access point to one individual's story, you then get a broader context for other things that were happening that were more historically relevant. Events and so that's been a lot of my approach with the voices of VR podcast because you know I'm talking to all these individuals But they're all in a moment in time of a reflection of what's happening in the larger culture And so by talking to people about what life was like or different experiences they had they start to reflect what the culture was at that time and so it's a way of recording that history and So I think this is really exciting to see where this goes in the future. And especially as you, you move into a much more robust conversational interfaces, they said that they would ask a question. They would like translate the words of that question into the file name. And then that's one aspect of getting access to the content, but there's so many other ways that you need to tag that content with the metadata and the meaning and to try to find these other patterns. And so. You can do some parts of natural language processing, but I think there's going to be potentially some role for coming up with different taxonomy systems and ways to try to cluster different aspects of content together, especially as you start to move across different people's lives and to look at all these variety of different contexts and try to get full comprehensive picture of someone's character. So as I talk about the four different major aspects of experience and story, I see it as like the quality of experience, the context of experience, the character of an experience and the story of an experience and so how it changes over time. But the character is something that 20 hour interview, you're able to do this deep dive into an individual's character. You see how they act, they see what they value, they see what their needs are. So As we think about the future of immersive storytelling, I think having new models for understanding character and the depth of character and how to explore that either through metaphoric projections of someone's life and when they're in specific contexts of their life, different aspects of their character may be evoked. And so through these oral histories, they're telling the story of these different situations and how they acted. And you're trying to determine, you know, different aspects that are kind of like the fundamental nature of their character, but also the deeper context of the historical context. So anyway, I think this is super exciting to see where this ends up going. And I was happy to see their latest demo and to see that the, I think they're going to be launching their app here sometime soon in 2020, maybe as soon as may, but keep an eye out for story file and looking forward to see how they continue to push forward. What's possible with oral histories, making it easy for people to capture their own oral histories, adding more volumetric capture and seeing how you can start to use other aspects of augmented reality. and yeah to be able to imagine it creating this whole spatialized oral histories and then in the future be able to kind of time travel and actually go back in time and to actually have a volumetric experience of these stories so i think there's a lot of potential here for where this can go so that's all that i have for today and i just wanted to thank you for listening to the Voices of VR podcast, and if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a list of supporters podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voices of VR. Thanks for listening.
