I interviewed Alexis Zerafa, Lafiya Watson Ramirez, Meredith Binnette, Heather Dunaway Smith, and Nella at the Snap Lens Fest about the Snap Spectacles and their hackathon project about dreams. See more context in the rough transcript below.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.458] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So in the next three episodes, we're going to be diving into some of the Lensathon hackathon teams that were able to participate within the Lensathon team. So at LensFest, they bring all these AR developers together, and then they basically gave everyone free hardware, the Snap Spectacles, and then they wanted to have them do a hackathon over the course of like 10 hours, over the course of two days, in order to try to build something as best they could. From what I heard from a lot of different teams, there was a lot of bugs that were launched within Lens Studio that were not there before. It was basically one of the more unstable bugs. I think they were trying to keep a lot of things secret, and then they basically didn't have time to do enough of a thorough QA to prevent all the attendees from essentially becoming their QA to be able to help find and reveal some of these different bugs. So there was a lot of aspirations for the different features that weren't always quite available. I think there were some core ML bugs that were there. There was no one that really did any social connected lenses types of experiences. So I'm sure that they'll get all that sorted out over time and that there'll be much more of a stable platform, but ended up a number of different teams had a little bit more minimalist implementations for what they were really aspiring towards. But I didn't have a chance to sit down with the team called the dream team where they were basically taking a dream and animating it and having the aspirations in the future to be able to essentially like record your own dream and have that animated as a part of like this kind of dream journal that you could share with other people as well. So that's where we're coming on today's episode of the Voices of AI podcast. So this interview with the Dream Team Lensathon team, which included both Alexis, Lafea, Meredith, Heather, and Nala, happened on Thursday, September 19th, 2024. So with that, let's go ahead and dive right in.
[00:02:02.401] Alexis Zerafa: Hi, I'm Alexis Serafa. I'm a creative technologist and an artist in the realm of spatial computing.
[00:02:09.445] Lafiya Watson Ramirez: I'm Lafea Watson-Ramirez. I'm an AR creator and multidisciplinary artist.
[00:02:14.252] Meredith Binnette: I'm Meredith Bennett. I'm a creative director at Black Math, and we're a small creative studio in Boston, and we do all kinds of stuff.
[00:02:21.738] Heather Dunaway Smith: Hi, I'm Heather Denway-Smith. I'm an immersive artist, designer, director, and developer.
[00:02:26.843] Nella: My name is Nella. I'm an art director, and I teach thousands of people all over the world how to make AR.
[00:02:33.008] Kent Bye: Great. Maybe each of you could give a bit more context as to your background and all the disciplines you're pulling together in your practice and sort of your journey into the space.
[00:02:42.847] Alexis Zerafa: Yeah, so my journey into the space was a long one. I actually studied front-end engineering and digital art in school, which really came together through AR, but at the time, it was a little more rudimentary. So I've stuck with it, and it's so exciting to see how everything has grown and evolved. So now, you know, spatial computing, creative direction, thinking in 3D instead of 2D, that's really the road that we're walking down.
[00:03:04.409] Lafiya Watson Ramirez: My background and foundation was actually photography but then I started getting into web design and development and always in my art practice really liked like mixed media projects and so my entry was really Flash and then Flash died and kind of in a roundabout way found my way to Unity and AR and kind of really love the space now.
[00:03:23.965] Meredith Binnette: My background is in animation and fine art. So I got a really traditional art school education. But then I started to see that animation really involves a ton of computing and a lot of math and a lot of physics and all that kind of stuff. And I just started to get really fascinated by what goes on under the hood. And then that led me to AR.
[00:03:42.742] Heather Dunaway Smith: Yeah, I started out as like a playwright and songwriter. So I sort of started in story and then I got sort of fascinated with computers and this new type of story that you could tell. So I got a degree in interactive design, started doing games and apps and stuff like that. And then that gradually just sort of led to augmented reality, which was this sort of new medium that felt really exciting. So I dug in.
[00:04:03.837] Nella: Yeah, I have a similar background. I am a TV producer. I work in film and I really feel like the way that the digital age is changing. is going to be a lot more immersive. And so funnily enough, it was only when I met Lefea, which is three seats down from me in this conversation right now, was I introduced to the world of AR and I found it was a much more powerful tool for storytelling. Whereas before people would engage through likes and comments, now you can actually be immersed in the story yourself. And that's how I got excited about it.
[00:04:36.718] Kent Bye: Great. Well, we're here at the Lensathon, which is the second day of a hackathon that you all just participated under the name of the Dream Team, making a piece about dreams. But maybe if we take a step back and each of you could talk a bit about your journey into working with Snap and AR. And if you were a part of what generation of the Spectacles? I know the fourth generation was the other one that had... actual AR capabilities. And now we have the fifth generation that you all just were making an experience on. But yeah, maybe just give a bit more context for your entry point into working with the Snap ecosystem.
[00:05:07.782] Alexis Zerafa: Yeah, so to be honest, I can't even remember where I came across Lens Studio. It was so long ago, but I started in 2017 or 2018 when I was in the beta phase, and I was still a student. And I loved the little dog filter, and I was like, I need to do this myself. So that was really the entry point. I dropped off for a little bit because I got busy with college, but now I'm a full-fledged partner. I came back around, and I love working with the team. I think my first introduction to wearables was that fourth generation of Spectacles, and even before that, the iterations that were just glasses and training meals for people getting into it. It's really exciting to see how they advance in this space and the commitment to AR as a whole.
[00:05:43.044] Lafiya Watson Ramirez: So I participated in a residency in 2020. Snap had a couple of different residencies and one was for games. I think one was for art. So I was in the game residency and that's how I got to meet a lot of creators. And through that, thankfully I got to become part of their like official lens creator program and got the chance to use their fourth generation spectacles. And so I made two games for that. And thankfully I'm happy to have the opportunity to work with the new generation as well.
[00:06:09.688] Meredith Binnette: I also got started pretty early in the game with Lens Studio when it was in the very, very early baby stages of the software. And at that point, it wasn't really like you didn't have to be good at it. You just had to be one of the first people doing it. And then you kind of got noticed and brought into this cohort of developers. And since then, over the years, we've all just become buddies. And we see each other every few months at these events. And it's really, really great to pull a community together. And there's all this shared knowledge between everybody. But yeah, with wearables, it's a really, really interesting new thing. And I played with the fourth-gen spectacles a little bit, but what really, really excites me is the new hand tracking, because that just opens up so many ways that you can interact with things. And I think it's using your body as the interface is what really excites me.
[00:06:56.963] Heather Dunaway Smith: Yeah, I'm also super fascinated by the ability to leverage context in, like, new ways, just sort of pivoting off of that. Yeah, so I was an AR art resident at Adobe, and then Snap heard about me and then commissioned me to make some pieces for the spectacles. And I was just kind of hooked, because it sort of, like, unlocked this sort of, like, truly immersive experience, or the beginnings of it, at the very least. And again, I'm just sort of interested in this, like, evolution of storytelling. So I just started experimenting and trying things, and this is the perfect sort of playground to do that, so...
[00:07:26.434] Nella: Yeah, so coming from a film background, I was actually brought into a project by one of the senior Snap team members to help a bunch of devs visualize narratives that they were trying to play out and help them create lenses. It went really well. I shocked myself and I was totally brought on another project. I'm not sure if you know Club Unity, which is a mental health project that they do here at Snapchat. And I helped a bunch of celebrities tell their stories using AR lenses. So that was like three years ago and then two years ago and last year. And I got to meet a lot of people on the Snapchat team from that. And they started asking me to host events. And since then, I've been hosting a lot of lens fest extenders myself. I've done 15 in 15 different cities and trained almost 2000 people. That was just last year alone. teaching them how to make their own AR tools and specifically how to use it for purpose. So AR for purpose. I don't do a lot of the silly goofy lenses. It's like, how can we really make the world a better place using technology?
[00:08:29.875] Kent Bye: Great. Well, I want to ask one other question around the actual spectacles and your impression on it before we start to dive into a little bit of your process and what you were able to create here during this lens-a-thon. But yeah, I'd love to hear some of your initial thoughts on the Snap Spectacles and what your impressions are, what you're excited about, and just what it was like to be able to make something for it over the course of 24 hours.
[00:08:50.103] Alexis Zerafa: Yeah, I mean, I think in the wearable space, right, it's not a sprint. It's a marathon. So any advancement seems incredible. And I just had to take a step back while I had them on and sit in the moment and realize, like, it's happening. Like, I see things around me, and I'm not holding anything. And I think it's wonderful. Like, obviously, we have leaps and bounds to go with the FOV and the form factor. But I think where we're at right now is incredible. And the fact that they actually were able to manufacture enough for everybody is a huge step. So I'm really excited to see what we do with them.
[00:09:19.893] Lafiya Watson Ramirez: Yeah, I agree with Alexis. There's a long way to go, but it's so impressive. And I think one thing that really helps me was to hear some of the engineers and see how excited they were about what they've created. And that really just brought into focus how much work it took to get to where we are. And we know that it's going to keep developing and go even further. But just the amount of technology that they packed into this tiny little thing, four cameras. I had a wonderful engineer who was onboarding me yesterday, and she was just excitedly telling me where the cameras were hidden and how they did their best to hide them. And it really made me happy to see just how far they've come and how hard the engineers have worked to bring this to life.
[00:09:59.639] Meredith Binnette: Yeah, I agree with all of that. I think it's really, really interesting to think about the future roadmap of wearables and AR glasses because everybody's coming at it from different angles and some people just think it's stupid and they're never going to wear glasses all day and that kind of thing. But I think it will ultimately be really, really powerful and we can add a digital layer to things in a way that is personal and helps you communicate in a way that does... I don't know. It could take you away from your device. I think the ultimate goal is that you're on your phone less if you have glasses, replacing one screen with another screen. But there's real potential in integrating it with your body and just being a little more flexible instead of this kind of phone screen that everybody's tied to. It kind of opens up a little more. So I think that there is potential there, although it's a fun, moral, ethical dilemma of how this moves into the future.
[00:10:51.087] Heather Dunaway Smith: 100%. Couldn't agree with that more. Yeah, I mean, the exciting possibility is that we can get people stopping, you know, looking in their laps, looking down, and actually looking out at the world. That's at least one potential. I've actually used several different specs, and I've seen the evolution of them. And we've really come up far away. These new ones are... Honestly, really impressive. I'm super excited to develop for them. Again, there's a long way to go. The field of view is limited. There are limitations to it, for sure. But the leaps and bounds that we've come in the last few years is pretty impressive. They're not overheating constantly, stuff like that. There's all these challenges that people don't really know behind the scenes. This is a really difficult problem to solve, and I actually think they're doing a really good job doing it. So yeah, again, it's just like a super exciting time to kind of make for all of these new things. But it's evolving really, really quickly. So you really got to keep on your toes.
[00:11:44.209] Nella: Couldn't agree more with my fellow dreamers here. I've worked on a lot of spectacle projects and I thought and feel they were all super cool. This was the first time that I felt like a user. We went to the Snap Partner Summit and we had this amazing experience with Es Devlin who created a council session for us using AR and it was the first time that I really saw the vision of what the world might look like in the next 10 years and I feel like there's a lot of opportunity for creators and coders who are up and coming and I think just so that I'm not repeating stuff that there's a lot of hope for where we can be going with this tool and we can really create something pretty am I allowed to swear on your podcast fucking cool we can create We can create something fucking cool, and we really did. We just did it. Like, we just did it in 10 hours, right? And yeah, for me, that's the feeling that I'm kind of having right now. I'm like, fuck, imagine what I could do in, like, straight up, like, give me a month. Give me a month, you know? Yeah, my hope is to see the world absolutely blossom from this.
[00:12:51.960] Kent Bye: Yeah, I agree that the Es Devlin piece of council was one of the more compelling experiences I've seen so far. That really, I think, is like a pole in the ground saying this is what we want to try to accomplish, which is like these community rituals of people coming together, having a shared reality together, but also interactive and participatory. So yeah, that kind of like location-based entertainment context, I feel like is a sweet spot for where the early beginnings of where this dev kit might go in terms of incubating a larger flywheel of an ecosystem of people being able to make experiences and experience them. So yeah, there's a lot of these glasses where Snapchat seems to be willing them into existence despite all other indicators for why it may or may not be a good idea. So I think it's... As an XR journalist for the last decade, it's sort of baffling in some sense. But the thing that keeps me optimistic around it is just how much Snap is both innovating technologically, but also investing in the developer ecosystem and having events like this
[00:13:50.595] Nella: The community aspect is incredible. I mean, I love what you said, Meredith, about like you come to these events and you really build lifelong connections and you end up working with people and not only do you bring people together through the technology that you create, but it's really through the entire Snap ecosystem as well.
[00:14:08.311] Heather Dunaway Smith: We're all developing together at the same time. So it's almost like we're this creative cohort that's informal and sharing insights with one another, learning together, having the same frustration, the same like, what the fuck am I doing? Why is this not working? And there's seriously limited documentation on how to solve a lot of these things. So you're kind of just in the void. But luckily, we've had events like this where actually able to meet people, have folks to reach out to when we're in the dark.
[00:14:36.724] Kent Bye: So before we start to wrap up, I'd love to hear just some feedback of what it was like to develop your experience, because it was basically a very sparse experience in terms of you push. There's a book, and you see a dream that sort of animates. It's like a minimum viable product that will be expanding out into having much more interactive people recording and maybe having their dreams be translated into some sort of spatial representation. But yeah, I'd love to hear just some of the feedback on the process of putting this all together.
[00:15:03.297] Alexis Zerafa: Yeah, I think we did a great job this time at editing. Given past experience when you're using a tool that was released a mere few hours ago, you know, you're going to have to assume that a lot of things are going to break. So we decided amongst the team that we would aim for functionality to be extremely simple, but for the idea and the future concept to be very well fleshed out. And I think that guided us super well.
[00:15:26.676] Lafiya Watson Ramirez: One hundred percent. One hundred percent. No, it was it was I've kind of backed away from hackathons because of that. You know, it's just very anxiety inducing. So this was such a wonderful team to work with. It really was a dream team. No, you know, not to get all cheesy, but it just I think all of us came in with the same mindset that we were going to do the best we can. We were going to have fun, learn as much as we could. And just, you know, it worked amazing. And thankfully it worked. But we just weren't going to put that pressure on ourselves and just play with the product and just be happy that we have this opportunity to do this.
[00:15:57.056] Kent Bye: Yeah, there's only so much you can do in the course of 10 hours. So yeah, shipping anything is a miracle a lot of times.
[00:16:03.178] Nella: There was real impact. Like we literally inspired ourselves. We didn't know we could do this. So although like obviously it was super limiting what we could actually create. I mean, it was a total inspiration project. Like I'm going out there thinking like, oh my gosh, how many more things can we do? Right. So it worked. It worked.
[00:16:22.259] Kent Bye: Awesome. Well, final question to wrap everything up. I'd love to hear what each of you think the ultimate potential of spatial computing and this type of immersive storytelling might be and what it might be able to enable.
[00:16:32.524] Alexis Zerafa: Yeah. For me, the main goal and the thing that I see being extremely useful is using these as an accessibility device, something that's assistive and intuitive. So I really hope we'll be able to create those types of experiences so much easier now that it's hands-free with a lighter form factor.
[00:16:47.588] Lafiya Watson Ramirez: I think like Heather, I'm always really interested in storytelling, and I'm very obsessed with location-based AR, so just I think the opportunity to layer stories, give more context to locations and statues, and just to be able to kind of open up more history and more context to different parts of the world.
[00:17:03.459] Meredith Binnette: I always bring this up, but there's this lovely article that I read like years and years ago. It's in Wired and it's the mirror world and it's like into the mirror world, something like that. But it really talks about how the potential of AR is to create digital duplicates and copies of our worlds that add context and the author says this really great thing about how we enter the photonic era where you know the same way that kids learn textual literacy and you know they learn how to type future kids will learn how to you know sculpt in 3D and build 3D environments and so that's what I really see is that like we're 3D-ifying everything where you know creating a more spatial more interactive intuitive world that is beyond you know 2D user interfaces.
[00:17:44.352] Heather Dunaway Smith: I think, well, stories are like how we teach one another how to survive things. It's like how we communicate some of the toughest lessons in life. And I feel like there's this new opportunity to tell stories with other people, like in the same shared experience. It's like a different way of accessing somebody's heart and mind. And so I feel like it's really going to kind of extend communication and allow for stories to feel more personal. And I'm excited by that.
[00:18:09.488] Nella: I completely agree and I would love to kind of combine your and Lufea's feelings which is I feel like AR really has the power to help us communicate. I come from a country where there's a lot of very difficult topics in terms of race and wealth sexuality and gender and it can be really useful to sometimes show rather than tell and experience rather than give you a piece of paper or you know just see it on social media you're absolutely immersed into the situation and that's my hope to really build connection and help people help people communicate it's hard sometimes awesome any other final thoughts or anything else that's often said that you'd like to say to the broader immersive community
[00:18:54.022] Alexis Zerafa: No, just if you think that this isn't the space for you because you don't see yourself, you could be the first one to represent yourself. So don't let that hold you back.
[00:19:02.907] Nella: Anyone can start. Anyone can start. I think people also feel that AR and technology and these spaces are like really clicky or they're really difficult and really confusing. But I mean, if you've just turned on this podcast and it's the first episode that you're hearing and you're thinking it's totally out of reach, it's absolutely not. Go on. You don't have to buy spectacles, but Lens Studio is totally free. There's a bunch of tutorials. Just start and come meet us next year.
[00:19:27.950] Heather Dunaway Smith: Yeah, just to pivot off of that, we're actually at this moment of transition in digital art where things that were previously really, really hard are becoming much, much more accessible, like 3D browser-based tools, for example. Like, 3D is a very deep pool, it's very intimidating, but it's getting easier and easier every day, so I really do think that everybody can join in and have fun.
[00:19:48.271] Meredith Binnette: I would also just say that, like, on the topic of moral dilemmas, I think that there's so much in the tech world of, you know, waves of hype and then bursting bubbles and all these things. And I think navigating that is really important. And, you know, I don't know all the answers and Snap doesn't know all the answers. Nobody knows. But I do think that being in the space and testing technology and working with it and making things is how we figure out what will work.
[00:20:12.563] Kent Bye: Awesome. Well, thank you so much to the dream team of this group of pioneering AR developers coming together and pushing the edge of what's even possible in the technology. I look forward to see where it continues to develop and evolve. And yeah, I'm just really excited to see all the artistic and creative imagination that I'm seeing here in this developer ecosystem. And yeah, just looking forward to see where you take it all in the future. So thank you so much.
[00:20:33.483] Nella: Thank you. Thank you. Thank you, Ken. Have a lovely day.
[00:20:38.188] Kent Bye: Thanks again for listening to this episode of the Voices of VR podcast. That's a part of my larger series of doing a deep dive into both the announcements around Snap Spectacles, as well as the AR ecosystem at Snap. What I do here at the Voices of VR podcast is fairly unique. I really like to lean into oral history, so to capture the stories of people who are on the front lines, but also to have my own experiences and to try to give a holistic picture of what's happening, not only with the company, but also the ecosystem of developers that they've been able to cultivate. And so for me, I find the most valuable information that comes from the independent artists and creators and developers who are at the front lines of pushing the edges of what this technology can do and listening to what their dreams and aspirations are for where this technology is going to go in the future. So I feel like that's a little bit different approach than what anybody else is doing. But it also takes a lot of time and energy to go to these places and to do these interviews and put it together in this type of production. So if you find value in that, then please do consider becoming a member of the Patreon. Just $5 a month will go a long way of helping me to sustain this type of coverage. And if you could give more, $10 or $20 or $50 a month, that has also been a huge help for allowing me to continue to bring this coverage. So you can become a member and donate today at patreon.com slash voices of VR. Thanks for listening.
