Don Allen Stevenson III aspires by become the Bob Ross of the Metaverse by creating AR filters and creative technology XR projects, and XR developer that he promotes on this social media channels (@donalleniii on Instagram, X (Twitter), and TikTok) with an educational angle. I had a chance to catch up with him at AWE to hear a bit more about how he navigates these liminal spaces of emerging technology as he consistently has to reinvent his self-identifying job description to keep up with the rapidly evolving field of technology. He also dives into the many ways that he’s using different AI technology from ML models integrated into different AR filters to generative AI to the many ways he’s been using ChatGPT throughout his day-to-day tasks of running his business as an independent content creator.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So this is episode 10 of 17, looking at the intersection of XR and AI. And today's episode is with Don Allen Stevenson, the third who describes himself as the Bob Ross of the metaverse. So he's making AR filters and doing different creative technology projects as an XR developer, educator, and social media content creator. I had a chance to catch up with him to hear how he's creating these different augmented reality filters and just his job as this kind of liminal space of both educator and creative technologists and social media content creator. I mean, sort of jumping in between all these different identities and roles. And I just wanted to get a bit of a beat of his career path and how he's managing all that. But he also talks a lot about how he's been integrating artificial intelligence and machine learning, not only in the augmented reality features for different aspects of body tracking and different ML models, he's able to integrate into Snapchat filters, but also using ChatGPT as sort of a dialectical partner to get business advice and to be able to bounce things off. He actually has been using ChatGPT for a whole wide range of different stuff that he dives deep into in the course of this conversation. So he's someone who I see as another creative technologist and artist and XR developer who is at the forefront of all these different technologies and also someone who seems to be fully embracing all the different possibilities of what these AI technologies are able to afford to him as an independent creator. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Don happened on Thursday, June 1st, 2023 at the Augmented World Expo in Santa Clara, California. So with that, let's go ahead and dive right in.
[00:01:56.138] Don Allen Stevenson III: All right, hello. My name is Don Allen Stevenson III. I go by Don Allen III. And in the realm of XR, I try to be like the Bob Ross or Mr. Rogers of the Metaverse.
[00:02:04.802] Kent Bye: Okay, great. And maybe you could give a bit more context as to your background and your journey into becoming the Bob Ross of the Metaverse.
[00:02:10.977] Don Allen Stevenson III: Oh, I guess my background was I worked at DreamWorks for three years. I taught all of our software to all of our artists. So I learned the whole pipeline around 3D. And then on my free time in the evenings, I would work on my own personal projects and share it online. I got the idea of starting to put my 3D skills on top of live action video that I had filmed on my phone. And other people started calling that augmented reality. And I was like, oh, I thought of these as real-time visual effects or post-processed VFX. But I like the idea of augmenting the reality of what I was filming. And so I started calling it augmented reality, even though it was originally visual effects that I tracked in After Effects. But then eventually I'm like, oh. And so clients would say, hey, do you do augmented reality? I see your Instagram. And I'd say, yeah. And then they say, cool, here's a gig. We need it to actually be a real augmented reality. And then I had to actually learn, how do you make it work in real time? I didn't know at the time. And since I've kind of worked and collaborated with a lot of different teams, a lot of different companies and agencies and corporations with AR projects, AI stuff, and everything in between.
[00:03:18.759] Kent Bye: And so what were some of your first projects that you did in, let's say, proper AR versus these visual effects pieces that you're doing?
[00:03:25.255] Don Allen Stevenson III: Yeah, some of the first ones were face filters on Spark AR around like 2018. I started making like art face filters and then started making like 3D files that would work in web AR and then some like designing experiences and apps like Adobe Aero and then doing puzzle games that you could do through your phone. Those are some of the early ones. Or some of them are just learning experiences, teaching people about history or technology, and using a filter as an educational tool.
[00:03:59.279] Kent Bye: I think some of the first videos that I came across of yours was when you were doing full body tracking stuff, where you're swapping out some of your... Have a memory of you being a monkey, or maybe describe what you were doing there.
[00:04:11.768] Don Allen Stevenson III: Yeah, so I do a lot of full-body avatars or full-body tracked on augmented reality clothing. So I've done some stuff like Lil Nas' teams like for their outfits for their like Met Gala, but they want a virtual version of it so that other people could wear. I've done stuff like these are full-body suits that usually I would deploy in Lens Studio because they have a very good tracking model for your full body. They track legs, arms, fingers, faces. with a lot of precision. And so the one that you're referring to was one of those Bored Ape characters during the hype cycle of that crypto movement. I wanted to try to bring value to augmented reality for the crypto space. And I figured that there's a lot of value being put onto these monkey characters and the Bored Ape Yacht Club group. So I decided a 3D model in ZBrush, a Bored Ape, template that I then textured and made look like other people's owned NFTs. And then I decided to add physics to the ears so that when you shake your head, there's a delay on the ears so they actually flop around. It has mouth tracking. It tracks about like 51-ish facial expressions and eye movements. And then I used a machine learning model to remove the live-action person from the shot so that the only thing that you see in the lens, in the filter, is just the character driven by a real person. It's almost like doing like real-time motion capture and character performance and performance capture in one little filter. And then I would distribute that usually through Lens Studio because they have the As of right now, a really good body tracking model. And then, yeah, I've done like outfits, like a lot of dresses and like with wings that make you look like Ariana Grande's Fortnite skin. Collaborate with other artists like Paige Piskin to do the face makeup. I usually would make trends into outfits that people could wear. What do you mean by trends? During the pandemic, there was that popular trend of Squid Games and I was watching it and I was seeing like this was like a very big cultural movement. So I decided to make One of the pink soldier outfits, a full body trackable thing, had like millions of people wearing it on Snap just because they wanted to be a part of that cultural trend, that moment of, whoa, I love this TV show and all my friends are talking about it. And then now I'm going to make a tool for them. In my case, the tool is a full-body filter that makes you embody a character from a show that's popular. And the Fortnite skin was the same idea. When Ariana Grande performed at Fortnite, she had this really cool outfit that people that play the Fortnite game could later purchase and have on their own avatars. And I figured, you know, it's really cool you can get this in-game asset and use it in the world, but I thought, how nice would it be if you could actually take it not just in a video game, what if you could actually wear that outfit on your physical body? So I went into an app called Gravity Sketch in VR and modeled out all the geometry for the dress. I then took that geometry in the Cinema 4D, rigged it up, gave it bones, and animated the wings, and animated some movement onto the outfit. Then took all of that, tracked it into Lens Studio, and then had my friend Paige Piskin do the face makeup. She did the really incredible eye makeup and eye tracking and makes the eyes glow white. It gives you sparkles on your cheeks. And then we deployed that one around the same time as Fortnite launching the skin. And so in my mind, that's another way of hopping on a trend. and making an immersive augmented reality experience around that trend. And then it yielded a lot of people trying it on and a lot of people wearing it and then making content around it. They kind of like promote it for you. Like I didn't have to pay for ads. They made the ads, you know, and they're showing you real life use cases, which is really exciting and fun to see.
[00:08:02.261] Kent Bye: I noticed that you've been able to cultivate quite a following on Instagram with both showing off these different filters and having subscriptions and have a way that you're, in some ways, a social media content creator, but you're also an XR artist. And so I'm wondering how you self-identify if you are feeling like this is a way of advertising your artistic expressions. You're putting out all these filters out as a fun, creative exercise. But if that's leading to these other gigs that are paid gigs, or if it's about you just being an artist and creative, and you're more of a content creator. So I'm wondering how you think of yourself.
[00:08:37.707] Don Allen Stevenson III: Yeah, so I mean, that's a weird one right now, to be honest. Because I would say maybe I change my job title maybe every month at this point. So I kind of change it relative to what my current goals are and then what I think is going to be relevant next month. And so right now, I've been using the title of like advisor or a speaker. But then if I went back a couple of months, I would say XR creator. If I went back last year, I would say metaverse artisan. If I went back a few, I would say specialist trainer at DreamWorks. If I went back further, I'd be a motion graphics designer. It just really, like for me, like the title, I think like it's so important right now to not have a specific title because the amount of innovation in all these tech spaces is so changing right now that I think becoming static in your title and your identity, I don't think is a very safe method for being able to exist later on. So maybe it might sound pretty chaotic, but I actually feel a lot more comfort in shifting titles quite frequently. Even though other people might see that and be like, that should cause you a lot more anxiety. For me, that gives me relief knowing that I'm not staying the same. I try to put a past version of myself out of work every month with a skill I learned. So whatever past Don was doing that was not efficient, My plan is future Don will definitely, he'll put past Don out of work every month. And if he's not doing that, then he's not helping himself out.
[00:10:08.863] Kent Bye: Part of the reason why I ask is, as I see you from the outside, I see all these different things you're creating. And as I listen to you, part of it is that you're existing in this luminal space of this transition between something that hasn't fully emerged or developed yet. So it's an emerging market of emerging technologies that you're exploring. But yet, part of it is like, OK, well, what's the intention for doing some of these things? Because part of that job title means oh well it's a part of becoming a consultant or it's part of like doing a filter designer or it's being a part of like an artist that is just being a creative and so it's a part of just trying to see like what the intention of all these different things of where it's headed and sometimes those labels help to suss out what the deeper intention is for the different artistic expressions but because it is a liminal space I understand that that may not be clear maybe for yourself or the wider community but It's also quite interesting to see these emerging job titles that haven't really existed yet. And with these emerging markets, we don't have a clear sense of how money is being made. And so other people may be looking at you wanting to do what you're doing, but not really have a roadmap because you're already walking a pathless path in some sense. So anyway, that's sort of an elaboration of that.
[00:11:18.680] Don Allen Stevenson III: I can definitely share more about the business side. There is a practical way of making money doing the liminal space thing. Because I've worked at corporate. And I've worked for a company. And now the last three years I've been independent. And my mental health has improved. My physical health has improved. Relationship health has improved. And financial health. All from being independent in this liminal space. Because there's a lot of unknowns. I would say maybe I guess when it comes to the business side of things, most of my time is spending writing contracts and reading contracts. And I read them very carefully. And I talk to the people. I also chat with AI to help read over contracts at this point. I've trained an AI model. on what kind of stuff that I don't like to see in contracts and it knows my preferences at this point. So I can have it read contracts and it will highlight to me, hey, you might want to read page two. This item here is a language that you've told me that you want your human self to read over. So the contract thing is a helpful way of managing work. The other one is I know all my values super clearly. I documented them. And I publicly shared on my website what all my values are. And I share what my, like, life goals are and my intentions. And I only work with people that are aligned with these things. So, like, that's kind of why it shifts around so much is because the kind of entity I'll work with is dependent on if the values are aligned with what my current definition of success is. My current definition of success is based off of how well I can balance my mental, physical, social, health, life and take on the new project or the new client. So if a new client for now helps me with my definition of success and my value system, I'll work with them. But if that same client's goals for next month change, that may not be very aligned with my mental and physical health. So that's kind of what caused the shift. I don't know if that helps answer the question.
[00:13:16.501] Kent Bye: Yeah, just a quick follow-up, because you mentioned the balance, the work-life balance. Is that a goal, or is that part of the values? And maybe you could elaborate on what those values are.
[00:13:24.543] Don Allen Stevenson III: Yeah, so I mean, I guess when it comes to work-life balance, believe it or not, I have a serious nerve damage problem on my whole left half of my body. I have chronic nerve pain. So I've gotten very good at managing my chronic nerve pain, but I'm always sitting at a 5 out of 10 in pain. This has been the case since I've been like 16. So as a result, I've become very militant about how much energy I use because I know if I take on certain clients, it's actually going to contribute to my physical nerve pain on the left half of my body increasing. So I know very quickly if when talking through them, reading through their contract or hearing their language about how they want to work, I know like, oh, wow, this person's probably going to increase my nerve pain to like a seven. I don't want to deal with that. I'm not working with them. But then I'll talk to somebody else and a new client, a new person, and I'm like, oh, wow, I'll actually probably be able to manage my nerve pain quite well with this client. I'll take them on. So it's a blessing and a curse to have serious nerve damage. It's also invisible because you can't see it on me. But I have like a big scar on the back for like the brain surgery I had to have and everything. It's a birth defect, you know, totally random thing on a skull bone shape around a brain stem. But that's kind of how I know who to work with and who not, is I kind of just check in with my left arm. My left arm is like... Do you really want to endure more pain than you normally do? And sometimes it's worth it. If the values are aligned, if it's going to help a lot of people, or if I feel like working with their team or technology is going to improve people's lives, then I'm like, OK, I'm down to take on a little bit more pain for the next few weeks or the next month to work with this person. But if it's not going to help people and I have to take on more physical pain, I'm just not interested in that at all anymore. I would have been more interested in pushing that in the past. So one of my values is being healthier with my nerve pain. So my brain will overwork always. It can overwork. It's great in capitalist structures where they want you to overwork for everything, but it's horrible for physical health. So that's kind of like the barometer I would use to know that.
[00:15:33.604] Kent Bye: Well, we talked about this liminal space and how you're constantly evolving and changing. And we're talking about contracts. And so maybe you could help ground me in a certain context, because what are these round filters that you're creating? Or what are the types of projects in the past? And I understand that you're changing and evolving and growing, and that's changing maybe in the future. But where did it begin where you started to take on these clients to have these contracts in the first place?
[00:15:56.743] Don Allen Stevenson III: Ooh, I guess it might have started with motion graphics way back in 2016, 2015, 2016 time. It would start off like I would make animations in programs, share them online, and then people would reach out through Instagram that say, hey, how much do you charge for animations? And then I'd talk to them about my pricing. And then if they sounded like there was some chemistry for a client, then a contract might be like, what is the agreement between that entity and me? And in there, I put things in like, what is the idea of success for the client? Like, when does the project start? When does the project end? What happens if it goes over a certain amount of time? How many revisions are in that cycle? So it started off with motion graphics. And then it's kind of transitioned that same kind of reasoning of like project-based contracts have transitioned into AR filters. So sometimes when you do like stuff for, I've done projects with like with Meta's team and TikTok and Snap with all their AR departments. And those ones are like bigger scale contracts. And in those ones, I just kind of like look over them and make sure I agree with that whole idea like will help with my mental, physical health by taking it on. And if it's yes, then great, I'll take it on. If not, then I'll say no. Yeah, so there's like individual, like person contracts for projects. And then there's like corporate big behemoths contracts. And those, they're different. But the similarities that they both have contracts, and I just work off of that agreement.
[00:17:27.489] Kent Bye: OK, and I noticed that you were a big part of the MetaQuest Pro launch, where you had early access and you actually were an actor in some sense of actually being in some of these launch videos with other people like Karen X. Chang, who's also been doing a lot of really amazing filters, creating prototypes and instructional videos and just really exploring the potentials. And so maybe you could talk a bit about how that came about to be featured in the MetaQuest Pro launch and have early access to some of that to explore some of that technology.
[00:17:57.223] Don Allen Stevenson III: Happy to share. So what I found is that if you're using a new feature on any of their platforms first, and you make really positive use cases of the tool and technology, then people from meta will reach out to you directly with opportunities. So the example I would give is the very first, I think within the first like seven hours that Spark AR became announced as a thing that the public was allowed to use, I immediately was like, I need to make something. and get it out there on their platform immediately because I'm guessing that they're going to be looking at what people are using this brand new feature, this brand new tool that's never been out before. And so that's my hack, is if you get in there as soon as it's happening and you put in a lot of love and energy and kindness into what you build, It actually supports their marketing efforts and it gives them positive use cases and they'll actually want more from you than just your fun project that you did as a demo. And if you do it consistently, then they start to follow your account and they start waiting for you to make your next thing. So I've done that enough times with different companies where I put my ideas out there of what I want to see in the world. And sometimes it's a VFX video, sometimes an AR filter. You start doing that enough times, then they reach out because it's going to help their business. They'll be like, oh, my God, we didn't think about using this tool in this way. That's really helpful for us. Are you interested in a project? And then they they have you sign a lot of NDAs and you can't talk about stuff or sometimes you can. I usually any company, not just Meta, but I always kind of like try to work on if they're too exclusionary with the speech. I won't enjoy that, because I like to share and teach knowledge. I feel like I'm an educator at the core. So some people, if they say you're not allowed to talk about it if you sign this NDA, I'll say no, because I'm like, I'd rather be able to speak. So I'll just wait for it to become a public thing, and then I'll make a cool tutorial and talk about it. And then usually you say that, and then they're like, OK, you can talk about it. And I'm like, great. Let me sign your NDA. I'm happy to sign, because I want to be able to teach stuff. So those usually come about through that. And it's usually, again, like a direct message that you'll receive through Instagram. Even DreamWorks, I got that job through Instagram. They saw my live streams. I was teaching 3D software for years, and they would quietly not identify themselves as DreamWorkers, were tuning into the live streams, just watching how I would teach. And then months later, when an educational role opened up at the studio, Like, oh my god, you teach 3D all the time. Literally got a direct message from DreamWorks saying, would you like to teach software at our company? And I'm like, oh my god, you're on Instagram? I thought you're like a film studio. So that same principle transfers for the AR stuff.
[00:20:45.750] Kent Bye: Yeah, it sounds like you've been able to really cultivate this sense of tuning in to your own creative imagination and expression and being present in the moment, listening to what's happening, and then aligning that with what you desire, want to create, or explore, and then catching a wave, metaphorically. Like you're out waiting for the next waves, and so you're kind of listening to the larger zeitgeist, and then trying to figure out what those next waves to catch, and then make something, and then be first, and then From there, things just kind of aggregate over time. That's sort of a metaphor that I get as I'm listening to you.
[00:21:17.364] Don Allen Stevenson III: Yeah, like, got to be there first-ish. Ideally first-ish. And then it's got to be a positive thing. Because, like, they don't want to endorse or celebrate or give opportunities to people that show their tool in, like, the most horrific way for humanity. Like, I see other people, like, will take a new tool, they find the worst thing that they can do with it, and they share their results on Reddit and Twitter. Those people are not getting reached out to at all. then that means they have to maybe take on other jobs that aren't totally aligned with their values. So for me, share positive things of what you would want to do with tech and then that looks good for them, helps their business. They'll pay you a big contract so that you can afford to teach people for free. Otherwise, I'd have to charge more money to try to make a living, but I'd rather not have to monetize people as much as just monetize the brands that need help communicating because they struggle with that.
[00:22:17.567] Kent Bye: Yeah, that makes sense. And I'm curious if you could dive into a little bit of your explorations of things around artificial intelligence. You mentioned Chachabit and different ways that you're using that for some of the filters that you're working on. So when did you first come across some of the different potentials of what you could do with AI and some of your work?
[00:22:34.557] Don Allen Stevenson III: Yeah, about a year ago, I started off with the Wambo Dream AI on iPhone. And it was a very simple style transfer. It's gotten a lot better. Now it's totally different than what it was a year ago. But I just like playing with that tool. You bring in an image, it can do a style transfer on it. And then I started sharing my results with WamboDream. And then Karen X. Chang noticed that I was making this AI stuff. And she got me early access to Dolly 2. And then I was like, oh, wow, this tool is so freaking awesome. And it's so fast. And the potential is huge. So I would say whenever Dolly 2 and around that time, like that month, is kind of when I started getting into generative AI.
[00:23:16.741] Kent Bye: And what type of stuff have you been doing with it so far?
[00:23:19.420] Don Allen Stevenson III: Everything. I mean, I use it to generate, oh man, I use it for everything. I use it every day, many times a day. I've used it to create pseudo-assistive lawyers. I've used it as brainstorming buddies. I've used it to help me message things better in writing. I use it to create scripts for Blender. I use it to write code for apps. I've been using it to create film ideas and blog posts. I use it to create my camp. I use it to hire people, I use it to create job descriptions, I use it for everything.
[00:23:54.594] Kent Bye: What's your camp?
[00:23:55.835] Don Allen Stevenson III: I started a camp called Camp Future Proof, and our camp's on June 16th, 17th, and 18th in the Santa Cruz Mountains, and it is the furthest thing away from technology and AI and AR and VR, and I love all those things. I just think I've been a little out of balance, so it's like a three-day, two-night 16, 17, 18, and I'm going to try to bring people together to learn about ikigai and finding purpose that's not just ascribed to your profession, but rather a mixture of what you love doing, what you're good at, what the world needs, and also what you can be paid for. The intersection of those four circles is an ikigai, and I think that we're going to need a lot more of that in a world where AI dominates and takes mostly everything. It's going to take most things away. So I figure what's going to be left, and when we take away a lot of our professions, I think we'll still be human. And so I want to kind of help people see purpose, and meaning, and value in humanity. We've got a number of people that are attending and still selling tickets, and I'm excited for it.
[00:24:58.905] Kent Bye: So how did the generative AI help you put together the camp?
[00:25:02.830] Don Allen Stevenson III: So I'm really bad at organizing things, so I asked it to help me organize a strategy for making a camp, and it gave me a to-do list. I'm really good with that part. It was strong in the area that I'm weak. I just needed the structure. I needed a scaffolding, but I'm really good off of a scaffolding. I can improvise and build everything else, because that's my strength is the improv part. I can improvise off of anything. But if I don't have a structure, I can't make anything. So I used ChatGPT and GPT-4 to brainstorm and ideate on a structure for a camp. And I also used it to structure who I would need to hire to make that happen. And then I had it build a job description for those people. And then I shared that job description and hired two people who helped make it happen. So AI has taken away so many jobs, but yet I can confidently say I've used AI to hire people. I've hired people I've never worked with before, because AI suggested I should hire them.
[00:25:57.188] Kent Bye: So it can't do everything. You needed to actually hire the labor of two people to help actually pull it off. But it was able to at least give you the structure to be able to know what you do, to be able to bring people in to have this happen. You needed those event organizers to kind of close those gaps in that there's still some jobs that are needed.
[00:26:12.900] Don Allen Stevenson III: Yeah, specifically I hired a chief heart officer. I think those kind of roles are going to exist forever. Because I think the human heart and social connection and human interaction, I think you're going to need a lot more of that in this future that we're accelerating into. So that was like the first hire was someone who could make sure that my business goals are always going to stay aligned with the heart goals. I don't want to separate business from heart. So that was like the first hire. And she's also an event organizer. And she helped basically coordinate the right people to secure a vendor, a location, a time of date, and help coordinate all that conversation. I don't know how to do that. But my chief heart officer does. And GPT-4 suggested that I needed a chief heart officer based off of the problem I was saying I was having.
[00:27:00.102] Kent Bye: Wow, OK. So when you're actually at this camp, what is your intention for the types of conversation? Because obviously we talked about how you're in this phase where you're in this liminal space, but still being able to live in this ikigai where you're at this intersection of all these things. Is the intention to help other people achieve that same state that you're at?
[00:27:19.626] Don Allen Stevenson III: 100%. I want to help more people find Ikigai. So the goal is if people come to camp, maybe not knowing what that is, I would like them to leave with a lot more clear understanding of how to find theirs. If not, we'll probably find it during the camp. And then I would like to give them the skills to help other people find it when they go home and spread that idea like a virus, a really positive virus.
[00:27:42.410] Kent Bye: Awesome. And so are there any other creative projects that you're working on right now that you're really excited about?
[00:27:47.497] Don Allen Stevenson III: A ton. A lot of NDA ones. A lot of fun projects. I'll share them as soon as I possibly can. Okay, here's a public one. I'm doing a lot more augmented reality work with Reality Composer and Reality Kit. and then filming those concepts with an iPad. So like sculpting some props and then filming a person interacting with those props in live action, then taking that video result and running it through a generative AI model to style transfer the whole thing. And it makes some very cool results for storytelling. I was trying to find ways to bring AR skills and VR skills and XR skills into the AI age because I was getting worried that you can do a lot of our heavy lifting work is going to start being eaten up by AI systems. So then I found that you can make assets and props that are animated or coded or interactive. and then use that as a storytelling tool, and then use that to help facilitate AI-generated images and video. So I'll be doing a lot more short films with like Wonder Dynamics. I'll do a lot more concepts with RunwayML, a lot of stuff with Kyber, and other AI models that facilitate video and image generation.
[00:28:59.915] Kent Bye: But you're giving it the baseline of like you're shooting some scenes on an iPhone, and then you're getting like the architecture of a scene, but then you're replacing it out with something else?
[00:29:07.194] Don Allen Stevenson III: Yeah. So imagine I sculpted, for example, a cliff face on Nomad Sculpt on iPad, textured it in the Procreate app on iPad, then brought it back into Reality Composer on the iPad, put that asset behind the subject so that it uses the real-world person occlusion from Reality Kit, and then we get this video of an actor performing in front of a cliff face. Then I can take all of that video and run that through RunwayML, do a style transfer based off of a frame that I made out of Mid Journey. A lot of extra steps, but the result is a really, really, really wild piece of content that would have taken me so long to make the other way.
[00:29:48.100] Kent Bye: Have you been playing with stable diffusion at all?
[00:29:50.347] Don Allen Stevenson III: I have been playing with Stable Diffusion. I keep struggling to get it to run on my PC. And the PC, it's a good PC. It's got Nvidia RTX 3090 in there. I'm just screwing up with the Python scripting part. I don't know if I have the wrong version. I have a Linux computer, a PC, and a Mac. I use all three for different reasons. The PC, I just keep not getting it set up, so I just keep going back to my Mac computer and doing other AI tools that work out of the box. I would love the flexibility and the... I feel like Stable Diffusion is like the Photoshop. With the level of customization and control, it's the peak. But at least right now, getting into it is so cumbersome, I'm struggling to even get it running on my device.
[00:30:36.058] Kent Bye: Yeah, I've just started to install some stuff. I had WhisperX, which is like a fork of Whisper to be able to get transcripts of my 1,200 podcasts and kind of playing with that a little bit. But I used Conda. They suggested Conda. And I think Conda is trying to do like managing these different environments because there's different requirements for some of these different models that need a specific version of a certain library. And if you just install it into one, then it can get complex or confused. I happen to do stable diffusion and installed fine. and was playing around with different models. I'm not using it to create anything. I just wanted to run through its paces. And I like the idea that you can do more of an open source version. But I know that Midjourney, they've got a very specific model that has really great results. And talking to Brandon Jones, he was talking about WebGPU and how in the future, they want to have the web as this more cross-platform device where you can have this type of compute and don't have to worry about getting everything working on your personal machine, of getting all the right libraries. So having something with WebAssembly integrated with WebGPU may be a future where some of this stuff is like lower friction for people to get running, but there's still a lot of like hand-holding to get it working and yeah.
[00:31:43.845] Don Allen Stevenson III: It's going to get better. I mean, it's like five months in for a lot of the stuff. It's like no time at all. This time next year, I bet someone's going to make a tool that very seamlessly daisy chains all of this pipeline with a very clean interface. And they get it at a price point that works for people, like maybe a free tier, a medium tier, a pro tier. That's going to happen. And when that does, Great. But until then, I'm going to be daisy chaining all those tools myself and just kind of figuring out which ones make sense for whichever goal is relevant for just that day.
[00:32:16.860] Kent Bye: And I'd love to hear some of your reflections on the different AR platforms from both Snap and Instagram and TikTok. There's both the technological architectures for each of those, but there's also the network of people that are on each of those, which are, I think, two different things of the distribution audience of these things, as well as the tools of what they enable. So I'd love to hear some of your reflections of how you make sense of each of these different AR platforms.
[00:32:41.760] Don Allen Stevenson III: Yeah, so I use all of them for very different reasons and I actually can narrow down why. So Spark AR is great if you want to make money a lot because it seems like a lot of brands that are already established and have a budget, a marketing budget, they've been pretty comfortable paying more when it's on Spark AR. But then it doesn't have as much control, doesn't have as much features as the other platforms like Lens Studio. So if you want to get paid a lot quickly, I would still recommend Spark AR. If you want to get a lot of views and you want a lot of people to see and experiment with your AR, and that's the most important thing, but being paid a lot is not important, and having the most cutting edge AR is not important, I would go TikTok Effect House. You get a lot of eyeballs. And also, you get to see real people use it in actual practical ways. That is so helpful when you're doing research on what is the right interaction for this AR experience. When you want the most cutting edge AR that's publicly available, but you don't want to get paid as much, and it has a pretty high rate of usage, but you won't see how people are using it, I would deploy it on Lens Studio with Snapchat. That's where I get these body tracking algorithms, the ability to deploy custom machine learning models. That stuff is extremely powerful and very efficient and very exciting. But it's usually not people, at least for me, have not wanted to pay for stuff as much when I deploy it as a snap filter because they don't have a lot of their marketing budget there. Those are those three when they're social. The fourth one that I would include in there is Reality Composer. I do demos with that one when I want to show people what the future quality of AR is going to look like because they don't have as much caps on file size. They have a very good lighting and tracking system. It helps kind of get people excited for what it's going to be. And you can deploy it to people's devices natively. You can airdrop Reality Composer files to any iOS device, and it opens up and works on their phone. So it kind of creates that really magical lower friction, assuming that they have an iOS device. And then if you're into Android, I'm really into ARCore. You can make some fantastical experiences deployed on there. And especially with Google's biggest launch with Google Maps, working in partnership with Adobe Aero, I think those two are really important to start using now, too. If you want to tap into the Google ecosystem, I'm guessing they're going to maybe make it so that AR experiences will start to live in the Google Maps app. Probably they'll start with landmarks, is what my guess is, like known landmarks. And they'll probably deploy AR experiences either built in Unity or built some of them on Adobe Aero, and they'll play it in there. So that's kind of my feeling. I'm sorry I'm not putting my foot in one of them. It's really like each one has a different purpose for me.
[00:35:30.654] Kent Bye: I expected there was going to be different trade-offs, and so it's helpful to get that landscape. And a couple of follow-ups. Is the Reality Composer, is that the Apple version?
[00:35:38.961] Don Allen Stevenson III: Yeah, so Reality Composer is Apple's AR development system. So they have ARKit, they're on ARKit version 5. I'm guessing that on June 5th, we're probably going to see ARKit 6 get announced, as well as all their new headset and their whole new ecosystem. Very confident that that's going to happen at this point.
[00:35:57.692] Kent Bye: Yeah, me too. I'm excited to see how it all plays out. Are you going to actually be there and be able to see some of the demos live? Because they have opened up to some limited set of developers. Did you get an invite to go check it out on Monday the 5th?
[00:36:10.781] Don Allen Stevenson III: I don't know if it's confirmed yet. So as of now, I'll say no. But it's possible I'll be there. Regardless, I'll be outside. I'm just going to go to the campus anyway. I'll just be there physically.
[00:36:21.125] Kent Bye: It's a good plan just to be able to network with folks as well. And yeah, it's a really exciting time. Another follow-up that I wanted to ask is that you mentioned that with the Snap lens that you don't always see, what happens? Is that because people are using it in private DMs? Or why is it that you don't always have more visibility for how people are actually using it?
[00:36:37.770] Don Allen Stevenson III: The nature of the app is more private. People use Snap, at least from my experience. They use it more privately with their friends. So I might make a really cool AR experience, but I don't always get to see how they're using it. Now, they do have a feature where people can share on a public space within Snap. That's not the default user behavior. They usually share it privately for fun inside jokes with their immediate friends, loved ones, and family, and that's hard wall. AR developers don't see how they use it. All we see is like usage of time. I used to get, I don't know if I haven't checked this in a while, but I could see how many human hours have been spent using lenses. That's a bizarre number to see, especially for the ones that like, you know, several million people use and you just see like hundreds of hours of people, hundreds of hours of human life spent using your lens. Bizarre feeling. But like to that kind of metric, it's very helpful with that kind of stuff. They actually do share more data with you than Spark AR does. But at least with TikTok, you see everything that people make with it every time. So they give you a pretty good data set of how people are using it and what. They even give you demographics. If they can pull some of that data, you can see some demographics. But with Snap, I love their tool because it has the most cutting-edge AR. But the platform part, it's hard for me to see how people are using it in the field. And I need to know that if I want to refine the experience or do a follow-up to it. I'm kind of hoping and asking people, send me what you did with your Snap. So usually what I end up doing is making a really cool Snap experience, recording a video with it, and sharing that to other platforms. And then I put a link to the Snap lens into Instagram, into LinkedIn, into TikTok, into Twitter. Not TikTok, they don't support links. But I try to put it in the places where links can work and say, if you want to try it, here's the link. You're going to have to use it on Snap. I know I'm taking you off the platform, and I apologize for that. But that's where this lens works. And the real ones will love to do that and they'll try it out and they'll try out your experience. And I try to encourage people to post on platforms and tag me, but it's not a normal behavior in Snap, whereas it's the default behavior on TikTok and it's a sometimes behavior on Spark AR.
[00:38:54.028] Kent Bye: That's really quite fascinating So I guess for you, what do you think the ultimate potential of all this XR? virtuality augmented reality mixed with AI What the ultimate potential of all that might be and what it might be able to enable?
[00:39:10.201] Don Allen Stevenson III: Yeah, so the potential which I'm most excited for is Jarvis will be a real thing for a lot of us Like the visualization, the collaboration, is kind of going to be when you're able to mix AI in an intuitive enough way with a visual or an audio experience that's accessible and relevant to the user. We'll have these superhuman AI assistants that have the ability to also create visualizations, audios, and other kinds of data and formats. So I think that's how, for me, that's like the biggest potential is having Jarvis not be a thing that one character in the Marvel Universe has, but rather a normal thing that people have, like a phone. You're just like, oh yeah, of course you have a phone. Like I can probably call you, I can probably send you a message. It's almost like... I expect that's a thing. I think eventually when we mix AI and AR together, it's going to be a semi-normal thing. It doesn't feel normal now by any means. I think it might become a semi-normal thing that I might interface with you as a human. I also might interface with you and your assistant. And your assistant is a mixture of like, it could be an augmented reality overlay, it could sometimes appear as an image, it could sometimes be a chat bot, it could sometimes be... I feel like it would also transform based off the context and the use case. But imagine having a visualization system. I think it's been so great for tutoring and education and medical and like museums. Also, it can be used in a lot of horrific ways, but I'm going to try to emphasize the positive ones because the evil ones are happening no matter what. So, the least I can do is do the positive ones.
[00:40:46.386] Kent Bye: Are there any experiences in XR that you want to have?
[00:40:49.909] Don Allen Stevenson III: In XR? Everything. You want to do it all? Yeah. Well, I mean, I guess, what do you mean by that?
[00:40:56.312] Kent Bye: Well, if there's any specific context of, like, a lot of people are driven by a certain thing that they imagine or that they want to experience, like, from a physiological level. And so, you know, whether it's art or entertainment or friends or family or, you know, if there's anything that's like a dream experience that you've been waiting to have within XR.
[00:41:14.085] Don Allen Stevenson III: Yeah, I would like the experience of having a really awesome teacher that sparks your interest in learning. be fully supported and adjusted in XR, and it's adjusted to the learning style and the learning preference and the inspirations of the user. Basically, if you can imagine the best teacher or mentor you ever had in your life in physical reality, if you're a person who's lucky enough to have had that in-person experience, I would love to see an XR experience that does that, gives you that level of meaning and value out of the interaction, that you grow as a person because of the interaction, and it helps you find that spark, that interest, that passion that maybe you didn't even know you had. That's the kind of XR experience I would like to see.
[00:42:01.680] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the broader Immersive community?
[00:42:07.288] Don Allen Stevenson III: I'm so excited for our community because it's like full of generalists and I'm very excited that generalists are probably going to be fine in this like AI takeover stuff because the generalists are the ones that can kind of move and shift around. I feel like XR merges so many different fields into one medium. So it's like by nature, it's already a generalist field. And I'm so excited that we're hopefully going to be the ones that are going to be thriving in this very big shift of technology.
[00:42:37.059] Kent Bye: Awesome. Well, I've been really inspired by hearing a little bit more about your story and navigating these liminal spaces and all the stuff that you've been doing. And I highly recommend folks go check out your social media. Where can people follow you and track your work?
[00:42:48.971] Don Allen Stevenson III: Thank you so much. Yeah, Don Allen, I-I-I. It's D-O-N-A-L-L-E-N-I-I-I. And I have it on all the platforms.
[00:42:57.633] Kent Bye: Awesome. Well, thank you so much. I highly recommend folks go check you out. I follow you on Instagram. Love seeing all your stuff. And yeah, thanks for taking the time to help unpack your journey and talk about some of your creative processes. So thank you.
[00:43:08.198] Don Allen Stevenson III: Thank you so much. It's been a pleasure. Thank you for having me.
[00:43:11.199] Kent Bye: So that was Don Allen Stevenson, the third who tries to be like the Bob Ross of the Metaverse. And he's making AR filters and creative technology projects. And he's an XR developer, educator, and social media content creator. So I have a number of different takeaways about this interview is that, first of all, Well, it was super fascinating just to hear all the different intersections that Don is working in. And he sort of describes himself as somebody who tries to reinvent himself and put himself out of business each and every month. And as somebody who is embracing the full potentials of the intersection of AI and XR, he's sort of realizing that if more and more of these artificial intelligent systems are going to be able to replace some of the different creative coding aspects, then he wants to be on the forefront of all the different new technologies that are coming out. He says that part of his strategy and his hack is to look at all these different AR companies as they release different platforms and different capabilities. He wants to be first-ish when it comes to taking the possibilities of these different technologies and putting out some type of prototype to show what the possibilities are. And it's from that process that he's been able to cultivate relationships with all these big augmented reality companies, either working directly with them or to do different projects and doing a lot of different contract work on AR filters and XR development, but also just generally being an educator and just being this independent, creative technologist and artist as he's working across all these different media. But yeah, super fascinated to hear that he was able to use ChatGBT as a consultant to get a bit of a strategy and business advice and actually hire an event planner and put on this whole camp and trying to achieve this ikigai of this intersection of what the world needs, what you can do, what you're capable of, and what you're able to get paid for and make a living off of. And so, yeah, the sweet spot of being in the midst of all those different intersections and how he's been able to do that, but also helping other folks as well. And yeah, just generally embracing all the different promises of artificial intelligence from both the content creation and his pipeline and augmented reality filters and all the embodied computer vision aspects, but also just as a conversational agent and Generative AI and the full stack of all the ways that he's been integrating AI technology So super fascinating to hear someone who's really on the bleeding edge of all this stuff and also super fascinating to hear how he starts to think about the different AR ecosystems between Snapchat where has the best technology, but it's very private so you don't get a lot of exposure for how people are using things then tick-tock you get a lot of details for how people are using it to be able to rapidly iterate on that and And then Instagram, which often doesn't have the most cutting edge technology, but usually has the biggest budgets for how people are still doing different ad campaigns. And then the reality composer within Apple, as well as ARCore within Google, different ways that he's also exploring those technologies, but still different technology stacks. And for each of these different things that he's putting out, having all these different social media channels, he's built up a following in each of them. trying to be in conversation with a different community as he's developing these things, and trying to get as much feedback as he can. But yeah, just the trade-offs of all these different platforms from the perspective of someone who's a maker and a creator in the space of augmented reality filters. So, that's all I have for today, and I just wanted to thank you for listening to the WSYS VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a WSYS support podcast, and so I do rely upon donations from people like yourself in order to continue bringing this coverage. So you can become a member and donate today at patreon.com. Thanks for listening.
