ReachYou is an AR story that receives transmissions from the future in order to connect to the tenderness of the now. It’s a slow media project aimed to create a portal into a story world that allows the audience to access emotion states of deep reflection and contemplation on topics like grief, what’s worth preserving on the human record, and the full spectrum of what it means to be human.
I had a chance to sit down with husband-and-wife team Jonah Goldsaito (creative technologist and visual artist) and Katrina Goldsaito (writer and performance artist) to talk about their journey and process of creating ReachYou. The ReachYou AR app is available to download from ReachYou.Space where the creators are planning to push out more transmissions in the future. I’d recommend experiencing their first transmission, and then tuning into our conversation to get more context for how this project came and where it might be going in the future.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast, a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. If you enjoy the podcast, please consider supporting it at patreon.com slash voicesofvr. So continuing on my coverage of looking at some of the different immersive stories that are featured at Tribeca Immersive 2022, today's episode is about Reach Now, which is an augmented reality transmission from the future that is trying to use these concepts of slow media to help us ground into what's now. And it's trying to get you to tap into different aspects of your grief and to share different parts of your human experience and to create a record of the human experience. A really interesting piece by a husband and wife team that is exploring the structures and forms of storytelling of AR and trying to get us into a real contemplative, meditative, and reflective state about what it means to be human. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Katrina and Jonah happened on Sunday, June 12th, 2022. So with that, let's go ahead and dive right in.
[00:01:19.587] Jonah Goldsaito: Hi, my name is Jonah Goldsaito. I'm a technologist and visual artist, and we are building Reach You, which is a transmission from the future to help us in the tenderness of now.
[00:01:37.892] Katrina Goldsaito: Hi, I'm Katrina Gold-Saito and I'm the co-creator of Reach You. I come from a writing and performance art background, so what I bring to the project is sort of the storytelling elements and figuring out how to take those other mediums and translate them into augmented reality.
[00:01:57.815] Kent Bye: Maybe you could each give a bit more context as to your background and your journey into augmented reality and immersive storytelling.
[00:02:04.457] Jonah Goldsaito: Well, I found the computer and creating things on the computer to be an absolutely transformative experience when I was quite young and back at the beginning of the internet. And just found it amazing that suddenly there were no gatekeepers and we could make whatever we wanted. And I, as a kid, had the same tools of expression as anyone else. And it's just been a long road of me making things, startups, and a lot of creative tools, creation tools for creatives in various media. And that did eventually lead me to work at Google for a while, where I got to see how that is done. in a large-scale organization, like, highly distributed across people with different expertise, because I'd grown up always having to learn every part of each component, or at least a little part of each component of the creation process myself, so that I learned to code, I had to do my own branding, I had to do my own outreach, I had to do all the communication, describing my ideas in various media to people, so that that could actually manifest things. And it wasn't until there that I got to rely on other people for the things that they were really good at. We were there for four years with our family. I don't think we mentioned, Katrien and I are not just making this piece together, but we're also life partners. We've created children together. We've created a pretty amazing life together.
[00:03:28.834] Katrina Goldsaito: So I guess what I've always been really interested in is going into the depths of our human experience. And so through performance art, through the books that I write, I'm really interested in figuring out how we connect at a basic human level. What is our shared humanity? What is the way that, even as strangers, we can have deep and meaningful connection? Augmented reality and this project Rechu, it's been this new medium to explore how we connect with each other and how through our sort of deep contemplation, through the ways that we think about our human experience and express that to each other, we can be more connected. And augmented reality has been totally incredible for me to be able to understand that we can both be centered and connected in our own reality and we can also put this layer of beauty and this layer of wonder onto the reality that we're experiencing. And that to me as a total, like a naive, like I'm totally new to this technology, that's been incredible to see the way that we can push storytelling further through the technology. But I'm new to it. And I think too about people who experience the project who are also new to it. And there's a door that opens when there's technology with which you're unfamiliar. There's more of that opportunity to ask questions that you might not think about in another medium. So that's been really exciting to me. And yeah.
[00:05:22.482] Kent Bye: Yeah, and we've seen the virtual reality and then the augmented reality since you know I've been doing the voices of VR podcast since 2014 and then oculus kickstarters and August of 2012 and so as you think about your own journey into virtual reality and specifically working on augmented reality I'd love to hear about your own journey about some turning points that made you decide to get into making augmented reality pieces and
[00:05:45.480] Jonah Goldsaito: Well, a big attraction to me of augmented reality is that you don't have to rebuild the whole world. And with some types of stories, you do want to rebuild the whole world. You want a world in which gravity doesn't exist, and that's hard to do here. you want to totally recreate from a molecular level. But what fascinates me is how can we use the richness of this real world and what's already built around us, these very complex systems and the very complex things that our bodies naturally do, some of which we don't even fully know about, that what's clear, researchers are always finding out about new ways that our body is relating to the world through signals that we just didn't know about 10 years ago, 100 years ago, as we've, over the course of history that humans have tried to abstract ideas and human ideas to pass on through some other medium, whether writing on, you know, papyrus or whatever, all the way to now. but in augmented reality we can utilize and hopefully ground people even more in the beauty of what exists now and and not cloud it but like bring them more to wait look at that like that thing in front of you that you really see that your brain is constantly dumping out all of that information that doesn't matter for your current survival and normal patterns in modern life like the way the light is reflecting off that metal in front of me right now is absolutely gorgeous And can we actually bring attention to that in the way that like meditation does, that these technologies built over all the time and utilize the technology, our internal technology that has been developed far longer than we've been conscious of it. Can we utilize all that? And it's actually the reason why when we start our experience here, we say, we're going to start with two things. First, we're going to check your internal technology and then we progress to the external technology. So we start with a very simple thing of just having them check their pulse to bring them into now. Because that internal technology, we shouldn't just discount that. And it's going to be a long time before something like virtual reality actually is tapping into a haptic suit. It's great. It's really cool. But the vibrations that I'm actually feeling, especially, let's say, for someone who's blind, who has one of those main senses that virtual reality relies on, they're feeling the vibrations from the ground. And that's a vital way for how they're reading the world. And the vibrations coming through the air that a suit would dampen. And then all the signals you get from, you know, you chew something and the feeling between your teeth and then the transfers of information that are going through your internal system. Like, we're just at the beginning ourselves of figuring out, and this is really just the first experiment, and how we can actually rely more on those technologies that we already have.
[00:08:30.517] Kent Bye: Was there a specific experience that you had, like a turning point where you realized, ah, this is what the potential of this augmented reality medium might be?
[00:08:38.295] Jonah Goldsaito: I don't know. Do you have an answer to that? For me, it was an evolution. It's just been evolving as we've built this out. What do you think?
[00:08:45.338] Katrina Goldsaito: I mean, I go back to the storytelling part of it, that there's something so magical for the people who have experienced it, most of whom are not in this world, most of whom are just people who are like, how do I place a beacon? Wait, let me figure out how this works on my phone. It's bringing them back. I think there's some Star Wars references, that when this hologram launches, in their space telling them this is a message from the future. There is a part of people that go, oh my gosh, like how else would a future person manifest except for through these phones that are ubiquitous that we have on us all the time. It would just sort of launch into your living room and you can only listen to it once as part of the project as well. You would only be able to hear that message once, respond to that message, And then that's it. And I guess that to me felt like an aha moment, where we were like, this is what augmented reality can do. It can launch people into a space where a little part of them feels like it's true, that this is a transmission from the future in a way that you can't do in a book, you know. I mean, and that's just such a simple thing that Jonah was talking about, about gravity. If we're evoking a space future, if we're evoking somewhere where we have these pyramids that can hover in air, that can open like lotus flowers and emit voices of people all over the world, That is not possible in a performance piece. You'll see the wires. You'll see the mechanics of it. So I guess I think through story. I think this narrative is only possible within this medium, if that makes sense.
[00:10:30.418] Kent Bye: It sounds like that is less about you seeing a piece and being inspired, but seeing that the new technology was available and that this was a new opportunity to maybe explore the stories that you wanted to tell. And so what was the moment, the origin point or the catalyst for ReachNow where you decided to move forward with this project? Like maybe give a bit of a backstory for this project and how it began.
[00:10:51.291] Jonah Goldsaito: So when I was at Google and our family was in San Francisco and we had lived that adventure for a while We decided we wanted to see instead of just continuing down that path because we were starting to develop patterns that we thought let's jump out of them, let's try ourselves in other patterns, let's see ourselves in other contexts. So we took our two little kids, at that point a four-year-old, almost four-year-old, and a six-month-old, and Katrina's two parents, septuagenarians, and we developed a little framework where every three months we would move to another country, in a little town somewhere, pick that place, go there, and just see how our lives would play out, how they would manifest, what we would want to make. And it was a pretty incredible experience. And after Morocco, living in a little medina on the water and in the mountains, and then in New Zealand living in vans just out and surrounded by green, we started to formulate some ideas that were the framework of this world. And then one day we're driving in the car with our kids asleep in the back, we thought, what if we turn this into a live performance? And the next night we're at an amazing dinner of five artists, and one of them, our friend, said, I'm going to Kazakhstan for this art, media, and tech business festival. in a month. Do you have anything you'd want to do and pitch? I bet they'd be open to hearing it. We put together ideas into an official proposal the next day, sent it along to the curator, and then we were in and we had a month to script this thing right and learn the performance. I wrote a piece of software that assisted it that had Katrina with pre-recordings, individualized in WebGL filters that would modify them based on input from the audience. that I was controlling from my back pocket so we could mediate between the interactions, the questions we asked these people who are at a very important time historically where they were, that day, having their first change of power since the fall of the Soviet Union. These were young people who were trying to take hold of their agency and seeing the possibility of what they could do. And there were people crying. It was beautiful. It was a beautiful experience. And then we thought, well, this is great. Are we going on tour? Like, wait, is this... Katrina has an acting background of many, many years. It was new to me, but I wanted to get to AR. Like, I really wanted to find a way to build within it. And we decided to move to Colombia as our next stop after Kazakhstan and the Kyrgyz Republic. And that's where we focused on building this. And then the pandemic hit and it turned from that group experience where people were working together to get to the same place into a very personal one that they could do in the safety of their own homes during quarantine, but which also reinforced their connection to humanity in a way that felt safe at that time. So that's a big part of the evolution of how we got here.
[00:13:51.745] Kent Bye: I'd love to hear your side of that story in terms of like starting to build out this world and like what was the moment that that world congealed and what that world is about.
[00:14:00.730] Katrina Goldsaito: I mean there are these ways that I am convinced that time is not linear because there's so much that happened that feels like if I were to rewrite it I would have written it in a different order because the thing that brought us to this is we had traveled all over the world We were trying to understand, okay, what is it that we want to make? And this idea that we had about a space future, where we asked people what were the things that, you know, we tried to create a human record that was a takeoff on Andrean and Carl Sagan's golden record that would collect these sensory experiences of humans on Earth right now. When we came up with this idea, We thought it would be fun. We were like, oh, wouldn't it be fun to make a project where we convince people that the earth is no longer? And they're responsible for capturing the most important things in the world. And this was before COVID. This was a part of our daily imagination and our daily dystopia. We had already kind of started thinking about this, and so the transition from the performance where we had to say, listen, here are all the reasons that our earth is gone, you know, climate change, fascism, racism, we had to list all of the reasons. When this became an AR experience, which we started right before COVID and then, you know, moving through the pandemic, built this piece, all we had to say was, I'm contacting you from the future. We're in this space pod in the infinity of space. And suddenly every person globally can touch in with what that destruction was, has their own imagination around it. in that imagining, that imaginary space, the place that we're able to take people to, hopefully. I think that it exists within this time that we're held in right now. And I think that the possibility of it, it feels like a happy accident that we came into doing this in augmented reality. I mean that Jonah had those skills and we're really built it in quite a bubble. You know, we were like, we don't know anything like this that's happening. We weren't tapped in with the festival. We weren't tapped in with any other makers. We just sort of had these skills that we had gathered and we were in a very isolated place in Colombia, just making it, just spending all of our time building this world and talking to people. I mean, so many of the people who, like the significant, like 150 people who did it during that first year of the pandemic, during that first year of making, I would cry every time I received people's recollections because every single one is totally unique. There's no doubling. There's no like, people use unique language to talk about what is the grief they're experiencing right now. And every one of them I felt. And I think that that was also the idea behind the human record, is that in hearing each other, so much of it is an exercise in listening. How do we listen well, you know? And I think that that sense of community that was created through the piece I don't know how else we could have done it. It feels like this narrowing where it's what happened, but I can't imagine how else it could have happened, if that makes sense.
[00:17:39.957] Kent Bye: Yeah, well, as you're having people go through this experience, there's an arc where they're entering in, they have a transmission from the future, we're asked to record different aspects of our human experience and contribute to this human record. And then at the end, there's like ways to kind of listen to other people's contributions. And so I'd love to hear if those recordings of those contributions there at the end was something that had happened to the process as you were creating this. and also if you just maybe want to comment on how this is a single transmission but that you may want to have like future transmissions and if you plan on now that this is launched into the world will that whatever the recording is going to be recorded into this database and shared amongst this community or if this is something more of a process where you've done some tight curation of specific things that you have captured in your own quality control and stuff that you want to share in a more editorial fashion or if it's going to be more of an open organic evolving process that has to have some sort of like moderation. So just love to hear a little bit more insight as to the stuff that we heard in this first transmission and as you move forward and have continued episodes what the plan is for it.
[00:18:49.359] Jonah Goldsaito: So yeah, that's really interesting. We've developed this, you know, it's a piece of art, but part of the practice of making it was I just couldn't help it. I've built so many things, like the process of putting something out, seeing how people use it, seeing if it's helping them get to where They need to be like all of my UX practice is just I can't help but have that Manifest in the creation process. So when people were first answering this the testers that were doing it I mean really they were friends at first and then friends of friends and not in big numbers when they gave Their responses. It's a lot to ask someone that kind of a question. What is a grief that you're feeling right now? just totally cold, you have to get them into the right space to do that. They have to trust you. And part of trust is giving people something they already know about, but we're introducing a whole new world, and I'm trying very, very carefully to not put in things that make it feel like a normal app. We never talk about it as an app. We never talk about our participants, our archivists, as users. We try as hard as we can not to have normal UI and buttons and all of the patterns that like if you want an app to be successful you would put in but we're not trying to do any of that. So how do you create a feeling of comfort? And one thing that we found in that evolution of helping people get to the place where they're able to be vulnerable and feel good in that and ultimately is creating a beautiful human record was instead of having their first vocalization be an answer to a question that's really really vulnerable to them, we ask them to hum. And we say that that is a key to opening up this interdimensional channel. And really it is a way for them to open up, I mean, just by vocalizing for a moment in a way that is that anyone or nearly anyone can do. All of a sudden, that really unlocked something. Like, the equivalent might be some kind of a tutorial, but calling it a tutorial and making it look like a tutorial, again, breaks this world. But it allowed us to really tap into something that, again, the internal technology, I mean, I don't know if it sounds cheesy or cheeky, but... There's a reason that's part of so many practices around the world, why people come together and hum and sing and vocalize. So we get their vocal cords activated first. I mean, maybe it's the vibrations in the body. I don't know. But that was an important step. When we found that out and we tried that, it was amazing. It unlocked something. And as Katrina said, I mean, they're really powerful. And when we're listening and we do have a level of moderation, but not in an editorial context, Only in that we don't want hate speech. We don't want things that once people are vulnerable in this space, and then they listen to others, if they listen to something that makes them recoil, like, oh my god, why was I vulnerable in this space where people are being evil to each other? That's the only thing we're making sure not to put in right now. We do listen to them for that reason.
[00:21:47.191] Katrina Goldsaito: I think that the intention is that it's for everyone and that any voices are welcome with the exception of racist or hate speech like that's where our line is for that and it's just it wouldn't be answering the question but I think that in terms of what people's griefs are or what people's gratitudes are. Those are all welcome. And I think the idea that we have one potential use is that there would be lots of librarians who are the transmission from the future, that there would be librarians from any different country, community, anywhere where they could think about and reflect on what is the question that they would want to impart in the human record? What is the thing that they think either their community or the world at large should be thinking of? And that we would build a whole series of human records that would have different objects and there would be like rooms upon rooms upon rooms that people could explore. And of course, again, that's the beauty of this kind of technology that we can include. You know, the golden record could only fit what was on a record. The human record can fit the whole world on it. So I think the future of it is being able to give it to lots of different people in their communities. and to have as many voices as possible, because I think there's something intimate about the voice, too, that it is disconnected from the sort of flash assumptions we make about each other based on the way that we look, the way that we present in the world. There's something intimate about the voice that it's also non-hierarchical. All of the pyramids, you choose which ones you experience, and they can be taken in in a really different way. And I think that that ethos would continue. I think we'd continue with that kind of questioning.
[00:23:43.711] Jonah Goldsaito: Yeah, I think we need to take a moment, I think ourselves, because this is only the first transmission. And everyone who's doing this today and everywhere that we launched, we launched globally as well. This is just the first transmission and we need to explore ourselves in this kind of new medium, what we want to source and what we want to start filling this with, what other questions we want to ask people, like what would we want to keep for the future? And once we understand that a little bit better, I think it would be wonderful to invite other librarians to take the role of what Katrina has done and ask their communities, as she was saying, like, what for that community is important of this time to preserve for that community? And they could speak directly to them and think, really, no, they're the only ones who would know for that community what is important to preserve.
[00:24:34.137] Katrina Goldsaito: Which is so wild because I was just thinking about, we were talking about Kent as ethnographer, but you've now interviewed 1,700 people. And do you always ask the same question? Like, you have that one question that's really the thing that you're trying to understand about this world. And just imagining being surrounded by all of those answers and being able to, you know, play them next to each other and hear the overlaps of them. Like it's such a beautiful thing Yeah, the very first conference.
[00:25:08.265] Kent Bye: I started covering in 2014 eventually I the final question I would ask people is the ultimate potential of the medium and then having asked well over a thousand people that question in my episode 1000 I featured the 120 best answers in a three-hour mapping of the cartography of the human context under which that these immersive technologies are going to play a part and trying to Look at both the positive and negative but also just what are the future potentials for where we're going? So yeah, it's been a meditation on that and chasing my own curiosity about We don't know where the technology is going, but we do know what the consistency of the human experience is, and if we can really map out the universal archetypal dimensions of the human experience, then you can start to then understand how the technology will start to play at the baseline of what it means to be human. and as we expand what it means to be human as we expand our sensory input so yeah just that process for me has been each person has their own spotlight into what they're focusing in on and I think there is a part of that artistic impulse where you just kind of follow your intuitions and through those intuitions then you stumble upon incremental little innovations that like in this experience like you were just saying how a lot of human-computer interaction and user interface design has a lot of aesthetics of like, this is the user interface, these are all the buttons, and it's kind of noisy in the sense of all the different things that you want to do in any moment, like just think about any 2D app or the file system and all the different actions that you want to do, but In a lot of ways, we're moving away from that abstraction of the buttons and moving into more of embodied interactions where it's just like maybe a single button to progress to the next step or like a triangle to record, but it's not like a record button, it's a triangle. And so, you know, it's not immediately clear what that is, but you, in the context of the story and the experience, we're shown what that means by actually having you push that triangle and record and so we're mirroring that so it seems to be like this pivot into the experiential interfaces and that there's a number of different triangles around and rather than clicking on them you just walk up to them and so there's a way in which that your body is becoming a controller within the context of these augmented reality environments so then it creates this spatialized embodied cognition for understanding how to interface with these different pieces of information, but it's more about not having a lot of things to click on, but this is a phone that you're holding that becomes a portal into a world. And the experience that I had that I thought was particularly interesting in the Reach U is that You know, a lot of times we have our phones and we may be like, hey, here's a moment you want to record. And so it kind of felt like, hey, there's a really interesting moment that I'm like recording, but yet I can only see into this world through this portal. And that there's all these modulations of the effects and it suspends my disbelief into understanding that there is this imaginal world that I'm looking into, but that it feels plausible and believable, especially in the context here at Tribeca with this amazing set that you've built. to be able to be transported into this little transmission pod receiving unit that's receiving this transmission from the future. But that I just come back to this transition from the 2D human-computer interaction into the 3D user interface and the more experiential ways of approaching this. And I feel like, in a lot of ways, with your background in user experience design, simplifying that and creating an experience that is really streamlined so it's intuitive for people to use, but it allows them to be really immersed into this other world that you've created.
[00:28:30.502] Katrina Goldsaito: And I think that Jonah was so thoughtful. I mean, together we were so thoughtful about what our relationship with our phones are. And in looking at the tenets of humane technology, how do we make something that doesn't use addictive engagement metrics, that is not taking and selling people's data, that has no profit motive, which we have to figure that one out, but that is free for everyone. How do we make something where people can exactly like you said that their phone becomes a portal that it is no longer this like irritating thing that is like doom scrolling and constant notifications. Can we slow it down? Can we change their relationship to their phone for just this little amount of time? And you mentioned, too, the physical environment, which I wanted to just shout out to Natalia Zubko, who did the beautiful pyramids and who was sort of in charge of how that beautiful space looks, and Chrissy Liu, who's the social practice artist who made all of those. Tetrahedrons out of paper. Her project is called Lattice Work. And then while I'm at it, our amazing performance director, Christina Valladares, who helped with the performance within the app, and Bo Kenyon, who is the composer, who did the beautiful soundscape that is part of the app. You know, it is a whole team of people who make this embodied experience but I think the thoughtfulness with which We've never wanted people to feel like you know There are things that we are unavoidable that Apple pops up these additional Dialogues that you have to go through which are a little annoying but the consent to be able to like can you use the camera?
[00:30:07.999] Kent Bye: Can you use all this? Yeah, just making
[00:30:10.141] Jonah Goldsaito: totally understandable, right? There should be consent, there should be more consent in our world, right? So I just want to go back to, we were talking about the phone as a portal and what the phone normally does, and it was designed for very specific things, but then they also, a long time ago luckily, well, along in a certain sense of the word, allowed those APIs to not just be used by themselves, but allow developers, when the flash from the camera turned into a flashlight based on some external developer saying, well, what if we just left the flash from the camera on? And then, you know, eventually Apple integrated that. But by leaving those APIs open, they allowed so much more potential than they could have done just on their own. Now, when we look at those APIs, we kept on thinking, how can we use these in ways to ground people more now, to surprise them into being here in a way that, you know, an amazing hike like awakens you and opens your eyes a little bit more or, you know, obviously. various types of chemicals might do. But in that moment, you know, in a normal film, the film is just playing through. But we say we're reaching out to you here now at the beginning of the Great Unwinding. If this is working, it should be reaching you now. And it shows the time. It kind of displays in our own unique way, like the time that it is right now. And we've had a lot of people say, oh, my God, wait, no, that is right now. And they realize, oh, wait, is this prerecorded? There's a moment, just a second of thinking, this is different. And then when she then reaches, she says, this is the closest I can come to touching you right now. And she reaches out, and it both warps what's happening on the screen, but we also use the haptic feedback right as her finger comes out. So we're trying to use what already exists there in ways that help our narrative and deepen you into this world, not just the world of Richu, but the world of now, because they should be interlocked and married in a way that, hopefully, and not use the pieces of the APIs that would pull you out of that. I mean, I wish we could ask people for consent to go into do not disturb mode automatically, for example. But yeah, all of those considerations, not trying to pull them back in daily, like there's a reason why you don't go around to people's voices and their unfurling pyramids and hit like so that they get a like back to pull them back in or some message that they have to reopen the app to reopen. Yeah, all of those decisions, they're difficult to make in some context because it's what makes your typical app successful. We had to go against so many UX best practices, and I say best practice in air quotes because they're best to attain certain, I guess, capitalist goals, but not necessarily best if you're trying to create good, humane technology that supports us living healthy lives. And that was a really big part of the project is making this in a world where that's not necessarily what's supported and praised, applauded. But we're hoping that there's a way in with Tribeca here and through this amazingly thoughtful podcast that we're creating right now that you've put so much time into. that we could get this out there in a way where it's no, it's not necessarily going to be backed by a big company or who knows, you know, we want to get as many voices around the world who don't have the ability to come to a festival like this. And hopefully soon we'll be able to push this to Android, too, because many of the countries we've lived in, that is clearly the dominant platform that people can afford.
[00:33:41.590] Kent Bye: You had any other thoughts you wanted to finish from before?
[00:33:44.997] Katrina Goldsaito: One thing we thought was funny in the App Store context was that we had to choose what category it's in. And so I think we ended up choosing health and well-being because there wasn't like an art experience bucket that we could choose. But in a way, I think we started feeling like there is something about this experience that can fit in a health and well-being. It's not an exercise video, but there is a hope that there's a slowness that comes about, that there's a moment of centering, a moment of contemplation that isn't usually available on our phones, which are things that are staying with us. we need to negotiate these relationships so that they're healthier. And it seemed an interesting way to do it, to do it through the phone, rather than just feeling like, turn it off, just turn your phone off, which is also, turn your phones off, that's also a good thing. But are there ways that we can change that relationship? And that's what we're exploring.
[00:34:52.710] Kent Bye: Yeah, one of the first things that you told me, Jonah, before I did the experience is that you were trying to do this as an experiment or exercise in slow media. And I've heard like a radio documentary of some of the slow media experiments that were happening somewhere in Europe or Norwegian areas where they would have like a train ride for like 36 hours straight broadcast on TV and these weird things that you wouldn't think would work in the medium of television, but ended up creating a whole shared moment amongst like the entire nation. But it's the idea that the normal narrative structure is not the hero's journey with the building and releasing of the tension, but it's more about being present with whatever is arising in that moment and maybe finding how you're related to that context of that moment. And so I'd love to hear your understanding of slow media and why you're calling this piece of ReachNow slow media.
[00:35:42.876] Katrina Goldsaito: I just wanted to say that there was an amazing one that was going sheep to wool that I watched that was, I don't know if it was a day, it was very, very long. And it is so soothing, you know, you see like the sheep being shorn, you see the wool getting spun, the whole process from sheep to sweater actually. So then there are like these women who are, and it's a competition, but it's like the slowest competition in the world to go from sheep to sweater. So I just, that sort of movement of Slow media television is really evocative to me and I think that as we develop a relationship with people who are doing this we can like expand it more and more because I want even more space within it I think. I'll let Jonah.
[00:36:24.370] Jonah Goldsaito: I think it's funny that slow media is kind of a new thing and I think it's beautiful what folks have been doing just playing in that space because it's so counter to what's happening with everything just speeding up naturally and exponentially. And I look at what happened in the pandemic with people, you know, even in cities buying little fire pits, looking at a fire together as humans. I mean, that's some slow media that's been happening since the beginning of our human experience, right? And there's something so magical there. And there's something deep that it patches into. So yeah, like the idea that it's new, right? It's not new. We're bringing it to these media that are new, these forms of media that are new. which is great because the natural impulse is to bring in the exciting and to stimulate all of our senses all the time. But I'm hoping that more people start to think about how to bring into this immersive and interactive space ways to make us more present in here and now. And when we see people leaving the Tribeca, because when we were testing this living in Colombia, we were just seeing what people recorded coming back. We weren't in person with them at all. And it's really interesting for the first time, really, to see people doing it in person. And when folks emerge from this little beautiful space, seeming changed in some ways and being really thankful. Oh my God, who was that woman who came out? She was a doctor who worked at Rikers and a VR creator. and totally inspiring on many levels. And she said, I don't know how, but this pulled forth a moment and story for me in my grief that I've never talked about or thought about before now. Like I'd never, I, it just came out. And when I mentioned that to someone else, they said that it did the same thing for them. And is there ways that more creators can do things that we've never thought of, that we can keep evolving, that we can all learn from each other? to help us be more here and now and I think like part of the narrative here is that if we're all more present in this moment and noticing what's here and seeing each other as humans and not as just lots of transactional relationships, I think we have a much better shot. of surviving and coming together and not going down some dark path of all the dystopias that people have been imagining in various forms of media, you know, going back through film and books and everything. I mean, I think if we are here and now more often and doing what so many practices we're trying to do with meditation forever, If we can do that through technology, great. So yeah, that's where we are. And we're just at the beginning. And I don't want to have too much hubris. We're just touching on this. We're novices. We're painting with paint for the first time. This is a new medium for us.
[00:39:25.109] Kent Bye: It reminds me of the Buddhist practices of the meditation, the practice, but the community and the Sangha is a big part of that. In some ways you create this distributed ability for people to come together and share their stories and use this phrase of the tenderness of now, which I think is a really provocative reflection of what you're talking about, of being present in this moment right now. And is there any contemplative practices that have provided a foundation or inspiration? I'd love to hear a little bit more about that mindfulness and that tenderness of the now and what that means for you.
[00:39:56.883] Katrina Goldsaito: Yes. So I grew up as a child. My dad is Japanese. He's from Tokyo. And he would drag us out of bed in the early mornings before the sun came up and we would go outside and we would sit. And his instructions were so simple. They were just, listen. Just notice the sounds around you. Notice the birds, the cars going by, whatever the sounds are, just note them. And we would do this practice. And he would then ask us at the end, like, how much time has passed? And depending on our internal states, we'd be like, oh, that was so long. That was the worst. Or like, oh, it was really fast. But it would always be the same time. And so we were playing with time in this really interesting way and playing with listening. And he comes from a Zen Buddhist tradition. Those traditions have always been really important to me in my daily life. There are a lot of contemplative art practices that my dad also introduced us to. Ikebana, which is the contemplative flower arranging. Shoro, which is the traditional contemplative calligraphy. And those things are absolutely foundational to all of the work I make to The Sound of Silence, which is the children's book that's about ma, which is the aesthetic concept of the space between sounds. So all of this comes from a really deep place for me. Within all of those traditions, within Buddhist traditions, the inner space, that inner technology that Jonah was talking about, is so vast. And it's something that we're skittish about, I think. It's a bit frightening to go deeply inside and see what's there to discover. And I think that there's chaos that we have to be with when we're going internally in a really deep way. And the hope is that if we can all get friendly with all of that, friendly with that chaos, friendly with all of the different sensations and emotions that arise within us, that there might be a possibility for more peaceful interactions, more kindness between each other. So in this small way, what we're really asking people through this augmented reality experience, we're asking them to go inside and just see what's in there. Check it out. See what arises from that.
[00:42:50.777] Jonah Goldsaito: Oh, wow. I don't have a meditation practice. I really should. We've just started sitting together. Katrina's bringing me in. But growing up, I would say that, and I don't know if it was because my vision wasn't great, But I started looking at the world differently and painting with amorphous shapes of color to form things that were representational. And in looking carefully and looking through the slits of my fingers and seeing how the light bent around the edges, and all of a sudden something wasn't where it was, it would shift over by the light being bent. Or like closing my eyelids and in between the lashes, refracting the light in a way that I thought, wow, like, Am I changing the world? What's actually happening? You know, as a little child, I would just play with those colors. I would play with them and then want to represent them with what I was making on a piece of paper in markings. It slowed me down. It wasn't necessarily always great when I was in a classroom and doing that, and my mind was trailing off thinking about the nature of color and light instead of the lesson at hand. But I think that is deeply a part of who I am and then throughout my life also in terms of not just the internal but in terms of engaging with other people. I'm the one to be dancing at a bus stop and then someone walks up and I will talk to them no matter what. and just trying to be there with them and find out where they are and who they are and why they are and hopefully you know get past some of those okay i'm just surrounded there there's some people around me like let me just dig into my whatever i'm thinking but being present with other humans and understanding them as other selves Those are kind of my natural ways of doing it. I'm sure there are ways, like deep ways, that I could have pulled from my background. I grew up and I come from Jewish culture on both sides. And there are ways in which you pray for so many things every day that are normal things that you would normally walk by, like right before you eat, right before you drink, when you see something beautiful, the first time you see something in a year. There's all these prayers that you're supposed to say that I think are meant to pull you back out of the normal flow of life stop for a moment and appreciate the amazing fact that this is all created here and existing in this moment. And I have some elements of that certainly. But yeah, I'm hoping that that I have the wherewithal within, again, this accelerating age to continue to like appreciate and recognize what's around me here instead of just letting my mind float back to the past and what I should have done or could have done or what I'm proud to have done or what I will be able to do and what I hope won't happen or will happen. Yeah, I'm hoping that through this practice I'm actually listening to what we say, practicing what we preach, and being here. And I'm so thankful to be here with you now doing this.
[00:46:03.403] Kent Bye: Yeah, and as you start to move forward, you had mentioned that you're going to have future transmissions. You also mentioned that you, from your user experience design, have a lot of iterative practice, and so getting the feedback. And so do you have a sense of an overall journey that you want to take people on? Or is it going to be kind of like each step and transmission that you have? And is there kind of a cadence that you have for as people move forward, if you have a sense of this journey that you want to take people on?
[00:46:28.142] Katrina Goldsaito: I was so inspired by how you noticed the non-linear quality of it and as you asked that question I was thinking, I mean I don't have an answer really for you, but I was thinking Oh, I really want to lean into it not being linear. I think that there's something interesting about that space. We've come up with so many different arcs of the whole experience. We've recorded other episodes. I mean, I think that each one is just about sitting with a different question. Because really, in the end, our aim is towards the human record. Everything is in service of how do we give people a moment to contemplate something that's important to our humanity, that needs to be shared, that can then be listened to. I mean, that's really what our aim, where we're aiming towards. And however the story serves, that end goal, I think, is where we're looking. Yeah, I mean, there's something exciting, too, within the process of working together, that as I'm writing a story, I'll be like, oh, hey, Jonah, could we trace where people walk? Like, is that something we can do with the technology? And Jonah will be like, oh, OK, let me figure that out. And so there's a lot within our process where the storytelling comes both through the writing of it and through what can we do with the technology. What are the ways that we can wake people up in this moment, have them notice something different? So I imagine it will continue to evolve in that way. And we have a lot of ideas, story ideas, about that character and what we find out later on about her and her journey. And we'll see. We'll see how it evolves.
[00:48:22.440] Jonah Goldsaito: Yeah, we went to a talk, I believe it was two days ago, on dystopian fiction. It was an amazing talk at Tribeca. God, I wish I could remember any of the panelists' names. They were all brilliant. But I'm sure it's recorded, so anyone listening who wants can find it. But they were talking a lot about creating realities, futures, where we actually want to go. And how when LeBron James shoots the shot, he doesn't envision it not going in and then them losing and being destroyed so that he can have a hero journey and come back in some other... He's envisioning making that shot. in our natural tendency, for who knows what reason, well, there are many reasons, I guess, of creating dystopian realities in fiction that when you create a lot of controversy, there's a lot for your hero to overcome, right? But that when we keep generating those, are we actually envisioning our shot missing? And then a lot of research shows that by envisioning a shot missing, you will miss it more likely. And if you envision it going in, you will make it. how do we actually envision a future we want to be in? And one panelist said, just by the act of trying to envision that positive future and stepping out of that dystopian loop, not all things have to be a Hansel and Gretel myth of like, hey kids, don't go into somebody's house because you might get eaten. Like that is in a lot of cultures. And you know, those work. Like you can tell someone not to do something, but it's also good to model good behavior. And like, just by stepping out of that pattern of telling those cautionary tales, okay, we'll get rid of those. Now we have a whole new palette. It challenges you just to make better art just by stepping out of somebody else's paths that have already been walked. Because they've been walked so many times. They're well tried. So yeah, I think that's part of our, I think we're re-inspired now to envision as these transmissions come out, how do we get to a positive place and a positive model? And I think a lot of it is coming from us and our internal vision and a lot of it is reactive to or learning from how this sits with people and what manifests from what's happening right now at the festival and, you know, around as people do it. But like Katrina said, we've recorded a bunch of transmissions, but we've made it kind of easy to make them. They're all, one thing that Katrina has said, I don't know where you got it from, or if you made it, but that all of her writing, as she's writing books, she writes and writes and writes. Maybe they don't all turn into a finished piece, but it's compost. It's fertile compost for the next thing you're making, to grow the next idea. And what we keep making, yeah, we'll see. And certainly as the technology continues to evolve, like I would love for there to be realistic shadows in this thing, and actually tracking and estimating the light better. All of that tech will feed back in, and once you're not holding up your phone and having that really small field of view, once there's actually an affordable headset, that will all just give us a bigger, larger, more beautiful, richer, convincing palette to work with. But yeah, we're just taking steps and hopefully next month we'll have a wonderful transmission too. And people will always start their journey at the beginning. I think that's what we're still thinking, is they'll always start and move through this linearly. But yeah, who knows where it's going. I mean, we're excited to see where it goes too.
[00:51:58.302] Kent Bye: And finally, what do you each think is the ultimate potential of augmented reality, immersive storytelling in this type of slow media, and what it might be able to enable?
[00:52:09.875] Jonah Goldsaito: Wow, the ultimate goal? The ultimate goal, I guess, is to help us be... I mean, we've been building tools as humans for a long time. It's not that long ago that we started to realize that we aren't the only species making tools, that we're not the only species, that other species have other gifts that we don't have, like really amazing abilities to sense things that we can't sense on bands that we don't sense. But in that long progression of tools to help us be able to, you know, the radar that some other animal has, as we progress along, it's all about hopefully allowing us to just be our better selves, right? That's from a tool standpoint, and not just supporting like military tools and tools that help us be more productive, which are great. but tools that help us just be better selves and from a storytelling perspective to inspire us to be better selves. Especially when in our daily lives we might be presented only with options that are not who we really want to be. You know, people are born into a certain little slice of human existence that they only see certain adults doing certain jobs and they only see certain ways to live and a way to manifest the world and they see that this is the version of success, this is the way to be a good person. And like maybe that for one person that includes they want to be a doctor and for another person it includes they want to make sure the stock market is doing like there's so many ways that people are told that they can be successful but with story it allows them to see just a vast number other like the such a plethora of Potentials of themselves and they say oh my god I don't have to be that like my teacher maybe said that that's what I should be My mom said that that's what I should be but look at this whole other world and I mean that's what happened for me when I first saw what my stepbrother made this nonsensical, crazy website for this college class he was taking and he saw how fascinated I was with him being able to put out his voice and he gave me this HTML Bible that was this thick tome of a book and I just poured over it so that I could start to make things in a space that at that time had no real path to success and like the roles that I've had in my career didn't actually have, they didn't exist, they weren't terms through storytelling, we can present people with possibilities of themselves that they're not necessarily being presented with. So, I mean, to me, that's the ultimate goal.
[00:54:56.686] Katrina Goldsaito: Yeah. I mean, it does feel like the ultimate potential is just as great as what our ultimate potential as humans is. What is that ultimate potential? When I try to answer that, I really do feel this like tenderness, like I'm almost afraid to voice it, because I feel like our ultimate potential as humans is to be able to experience the fullness of what it is to be human. which includes and has to include the knowledge that it will all be gone. That we end. That the world ends. And that within that, there is the potential to really see this world that we're in and to really see each other. and to be in and feel that connection to each other. These tools have the potential to broadcast that. But it is equal, it is really just equal to what we do with our own human potential. The infinity in ourselves and the infinity in these tools they're the same, you know? And what's exciting to me is that there is this freshness of discovery. It's a new way to remind ourselves of what it is to be human.
[00:56:48.550] Kent Bye: Beautiful. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:56:55.558] Jonah Goldsaito: I think there's a real potential here to be so generous and especially as this technology becomes more and more ubiquitous, inexpensive, to be generous in giving these experiences to people and helping people where they are, trying to figure out how to be really inclusive of all kinds of people. Especially when a lot of natural tendencies within the current system that at least we immediately are living in right now is to... How do we figure out how to recreate scarcity via blockchain and NFTs? There's a tendency in that direction of supporting this other world that's like... Hopefully we can support an economy and people can feed each other, but like... Why are we using digital tools that kind of naturally allow for everyone to have and be plentiful? And we're like trying so hard and pouring so much money and actually destroying the environment with like a proof of work system to create scarcity in a world that doesn't actually need scarcity. But like focusing more on creating beautiful things for people here and now. I mean, I guess I keep, I don't mean to be a broken record. And also, yeah, thank you, Katrina, that was so beautiful. That last thing you said, I really appreciate it.
[00:58:19.212] Katrina Goldsaito: Do you want to get married? Yes, let's get married. I mean, I'm just really grateful to have this time with you. I think Having the opportunity to think about it, to be with you when you've heard so many of these stories and can reflect so beautifully what it is that we're trying to make is really a gift. It's a gift for us to have this time with you. Thank you.
[00:58:52.390] Kent Bye: Awesome. Well, Jonah, Kuchina, thank you so much for all this journey that you've gone on to reach you and to sit down with me and share your journey and your story. Because, yeah, there's something about the onboarding and the experience and offboarding. I think that this experience is really well thought out and trying to really cultivate that sense of mindful reflection. And, yeah, you're kind of creating this distributed sangha of people as they're able to kind of share their thoughts and you're able to create that sense of community as a practice of people to try to create this movement of slow media with an augmented reality. Maybe this is the first piece that I've seen like that that's trying to accomplish that mindful presence within your own self and your own body into the tenderness of the now of the moment, but also within the context of a larger communal context. Anyway, a lot of really interesting potentials as you move forward, and I look forward to seeing future transmissions and where this all goes in the future. So thank you so much for joining me today.
[00:59:46.655] Jonah Goldsaito: Oh my god, thank you so so much. And that, wow, a distributed sangha. That's like, that's such a cool way to verbalize it.
[00:59:55.623] Katrina Goldsaito: Yeah, thank you. Really.
[00:59:58.905] Kent Bye: So that was Jonah Goldsaito, a technologist and visual artist who's working on ReachNow, transmissions for the future to help us in the tenderness of now, as well as Katrina Goldsaito, the co-creator of ReachNow, who works in the mediums of writing and performance art and taking the storytelling elements and translating these other mediums into augmented reality. So I have a number of different takeaways about this interview is that first of all really I really love the vibe of this piece because as I was talking to Jonah getting on board into this piece look at my pulse and trying to slow down and this concept of slow media which if you haven't heard of it is Taking stuff that unfolds in these processes over long periods of time, like a train ride across a country or sheep to sweater was one that Katrina was talking about where they show the many long processes of shearing a sheep and just turning it into a sweater. But just that idea that everything's happening so fast in the world And what does it mean to use the media to help us slow down to reflect and to? See the relational dynamics of what is happening in the moment and in our body and get these Transmissions from the future that is actually helping us to reflect on our human experience especially our grief which I think has probably come up a lot in the context of kovat and and just dealing with the isolation, separation, and death that we've been all experiencing and some degrees of separation over the last number of years since the onset of the pandemic. So this is a project that started before the pandemic, but is really timely for tapping into ways of using these immersive technologies to be embedded into the context, wherever we're at at our home or wherever, but to create this portal into another realm. And just the way that using the augmented reality technologies, you're holding the phone, you're looking into this portal into another space, using all the filters and portal metaphors, but also glitch art and flips you into this other context that you're actually observing and witnessing something, but paying attention and able to get these messages that they're trying to transmit of this deep contemplative practice of really noticing the present moment and helping to cultivate us into the tenderness of now. And there's going to be different transmissions that you could get over time. It's an app that you can download and experience it and highly recommend checking it out. Really like the vibe of both Jonah and Katrina and using all their different skills and practices and collaboration with each other as a husband and wife team to be able to communicate in this way and to explore the futures and potentials of this immersive medium to really great what I see this distributed sangha because there's ways that you can hear messages from other people capturing different aspects of the human experience. And then as you do that, you're able to reflect upon these different nuggets about what you would want to share about the essence of your experience. And as you do that, then you can hear other people and connect to their experiences and also to reflect upon your own experiences in that process. So that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a supported podcast, and I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you could become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
