On April 22, 2020, Magic Leap announced layoffs of somewhere between 600 to 1000 employees, which amounted to around 1/3 to 1/2 of their total staff. On December 2019, The Information reported that Magic Leap had only sold 6,000 AR headsets against their target goal of over 100,000. The full story as to what exactly happened and why has yet to be fully told, and it’s stories like these that reinforce dominant media narratives around the hype around Magic Leap.
After listening to the stories of a number of Magic Leap employees, then there are alternative stories about what the experience of working there meant to them and what the company has to contribute to the overall XR industry. Portions of the $2.6 billion raised through Magic Leap were spent on salaries and helping to bootstrap an entire cohort of XR professionals who are now working at a number of Big Tech companies. The Protocol reported in June of 2020 that there were a number of laid Magic Leap employees who ended up at Apple (~40%), Facebook (~20%), Google (~20%), Microsoft (~15%), and Amazon (~5%). [Note that these are rough estimates based upon the proportions of the pie graph].
Andre Elijah hosts a bi-weekly discussion on Clubhouse called the “No BS Realities of AR/VR” along with co-moderators Daliso Ngoma & Azad Balabanian. There were seven former Magic Leap employees who participated in a retrospective discussion reflecting on their time working on a lot of creative and difficult challenges of trying to shape the emerging spatial computing medium of AR. Elijah got consent from each of the speakers and shared the recording with me to air as a special edition of the Voices of VR podcast since there’s a lot of oral history perspectives of their time working there, what went wrong from their perspective, and why some of them still see the overall experience as one of the most exalted times of their career.
Here’s the list of seven former Magic Leap employees who were a part of the conversation:
- Anastasia Devana – Audio Director (until April 2020)
- Steve Lukas – Head of Developer Relations Engineering at Magic Leap (until April 2021)
- Paul Reynolds – Senior Director, SDK & Applications (until May 2016)
- Dave Shumway – Lead Audio Designer/Composer (until April 2020)
- Jeremy Vanhoozer – VP, Creative Content (until October 2020)
- Tim Stutts – Lead Interaction Designer (until June 2020 April 2020 [Updated: April 29, 2021])
- Joe Gabriel – Developer Relations Community & Program Manager (until April 2020)
They also share more details on the drama of trying to get The Last Light released onto the store after it’s premiere at SXSW was cancelled, but also after Magic Leap’s pivot from entertainment to enterprise applications. There’s some reflections about the unique challenges of working on an emergent medium, what their content strategy way, what it was like to work with their co-founder and former CEO Rony Abovitz, as well as some of their final thoughts and reflections of their time at Magic Leap.
There were a lot of new insights and perspectives that I hadn’t heard before, and while this conversation won’t fully answer all of the questions of what exactly happened at Magic Leap and why. But I do think it add a lot of valuable testimony about the early days of consumer AR from the lens of what it was like to be a part of a company that will ultimately be a part of historical evolution of the augmented reality medium.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye and welcome to the Voices of VR podcast. So on April 22nd, 2020, Magic Leap ended up laying off about half of its staff. So Magic Leap is known as one of those companies that had a whole bunch of hype around it, and they were trying to create this really expensive augmented reality headset that was going to be sold to the consumer market for entertainment. And I think that maybe it just wasn't the right time in terms of the market wasn't ready for a consumer augmented reality head-mounted display device. But at the same time, there's been lots of different innovations that Magic Leap has been able to be involved with, and they also were pushing forward a lot of the different content. If you listen to the actual employees, their experience of working there is actually one of the most exalted experiences of their entire career, just in terms of the boundless creativity and the challenges that they were trying to face. So, Andrei Laja is somebody who hosts these discussions, along with some other co-hosts, including Dulicio Nguma and Azad Balabanian. So, he hosts these discussions called the No BS ARVR Realities Within Clubhouse. And there was a bit of a retrospective from some of the employees that either survived the event or ended up getting laid off and moving on to the next bold adventure within the immersive space. So, Andre was actually hosting this conversation, starting with just two former Magic Leap employees, but then had another five that had shown up throughout the course of the conversation. And I've been participating in a number of different Clubhouse discussions with Everything Immersive Club, with Noah Nelson, and for a while, the Virtual Worlds Club, and haven't really recorded any of them. Actually, somebody else recorded this conversation, and all the different speakers consented to Andre Elijah for this to be aired. This gives you a little bit of a flavor of some of the different conversations that are happening in Clubhouse. This is kind of a typical Clubhouse discussion. There's a little bit of talking, and then as people go on, there's other people that are showing up and then hopping in and jumping into the conversation. I was able to ask a question there at the end. I think for me, the striking thing was just hearing some of the different experiences from Magic Leap employees that may be contradicting some of the major media narratives. Also, just what happened with Magic Leap? What were some of the different stories of what was happening with the content and if there was other decisions that were made? A lot of the people that are on this discussion weren't necessarily in a strategic position. They were more of working on their own very specific parts, mostly a lot of content creation and operating system and developer relations. And so there's a random sampling of different perspectives within the Magic Leap experience. So that's where we're coming today, on today's episode of the Voices of VR podcast. So this conversation is featuring a number of former Magic Leap employees, including Anastasia Davana, who is an audio director who left Magic Leap April 2020. Steve Lucas, head of developer relations engineering at Magic Leap, who left just actually last Friday, April 2021. Paul Reynolds, who is a senior director of SDK and applications, left in May 2016. Dave Shumway is a lead audio designer and composer who left in April 2020. Jeremy Vanhooser, VP of creative content, who left in October of 2020. Tim Stutz, who is a lead interaction designer who left in June 2020, and then Joe Gabriel, a developer relations community and program manager who left on April 2020. So just a quick note on the audio recording is that, uh, whomever recorded it was actually outside with wind and was making other noises. And so I tried to clean up as much as I could, but unfortunately there's sometimes some poor audio quality, but I think the conversation was interesting enough that I still wanted to put it out as a podcaster. So this conversation happened on Saturday, April 24th, 2021. So with that, let's go ahead and dive right in.
[00:03:47.894] Andre Elijah: Okay. So we're going to, we're going to start this off. The timing of this couldn't be better. A year ago was The Great Purge, The Snap, also the layoffs at Magic Leap. Obviously, ML was one of the most hyped tech companies of all time. Starting from the initial announcement, I remember I started hearing about them just through Hollywood circles and entertainment industry. Uh, because Magic Leap was really trying to engage with them at least behind the scenes and court the executives to get content. I probably heard way more than I should have before the thing went live. I know a lot of people that went down to Florida to check out the early versions of the giant ass helmet one that you'd stick your head in. And then, uh, I started hearing about all the devs that got to get devices in their offices with a security camera. pointed at the safe and they had to have like a blacked out room or whatever with a feed that Magic Leap had access to to track who the hell's working with the devices. Heard rumors about all kinds of things since then. And it's great to be able to have a few Magic Leap folks to actually, you know, speak to their experiences at the company, both good and bad. It's a divisive company. I have a headset on my shelf right in front of me. And I love it, despite the company's issues. But I guess we're gonna just get started, because everyone wants to know, you know, internally, what was it like working there in the good days? What was it like working there in the bad days? And just kind of navigate us through your perspective on all this. Go start with Steve, and then go over to Anastasia on this one.
[00:05:32.972] Steve Lukas: Well, I'd like to actually defer to Anastasia first, because she predates me in the company.
[00:05:39.563] Anastasia Devana: All right. Yeah, so I've been there from 2015 to, well, to the snap, the rapture, the event. I like to call it the event, actually. It seems respectful and has enough gravitas. So altogether, a little over five years. And when I was interviewing people while I was there, and people asked me that same question, like, what is it like working there? I always said it was the hardest and the most awesome job I've ever had. And I mean, looking back at that, it's still true. It was an awesome job. It was an awesome place to work, great team, great people to work with, interesting problems, lots of problems to solve, which keeps me happy. Even when I joined in the beginning. Like I didn't join for the technology. I joined because of meeting the people who work there and just like the amount of brains and energy and just like the amount of like cool, awesome people. Like, okay, even if this thing crashes and burns in a few years, like this would have been worth it. This would have been a great ride. That was my thinking in the beginning. And, you know, I think it was accurate. The end.
[00:06:52.608] Andre Elijah: Nice to see you.
[00:06:54.808] Steve Lukas: Thank you. Well, many of you know me as XMagic with DevRel, but some of you also know me as one of the early Qualcomm Ventures scouts that went to go take a look at the company back in 2014, when we entered the Series B round. So what I'd like to just point out is the impression that I got as an investor coming in and seeing the technology. Why did people put in so much money into Magic Leap I feel is a very valid question, especially considering how much funding came into the company and how few AR headsets are out there. And I'm just speaking with public information here, but I saw what was described as the beast. which was their early optical demos, which were amazing. And I had seen other competing headsets beforehand. So when I came in with my group, then we got to see the Dr. G demo, the shooter and the robots, that was a concept demo. And we understood that. Coming in, we said, OK, this is a concept video. This is what they're saying is what they're going to be producing you know over the time you know obviously this wasn't available right then because they're getting investment to build it so we went took a look and i said i don't want to go first i want someone else to go first and i want to know what they have to say because i'm worried that i'm just going to be disappointed because i've been disappointed in the past And after my colleague went in and came out, then I was like, what do you think? And he said, it's good. So I'm like, all right, let me take a look. And I was blown away. I hadn't seen any technology like this. And the way that they showed it off with the optics and just the perception, being able to perceive depth with a single eye, so it wasn't stereoscopic. I mean, that was phenomenal. So I said, everyone, you got to come down here and take a look at this. And so, you know, funny thing to me is that I sat on the opposite side of Rony's desk for three hours as he was pitching us. And I don't think I've spent more than 30 minutes with them after I joined the company later, but that's a whole other thing. So then as an investor, I'm thinking, couldn't they just sell this tech and make like, I don't know, a hundred million dollars or something and just be done. But then they showed us the rest of the tour. and I got to see people from ex-NASA and looking at the hardware and I got to see what developments they were making in the optics and some of the computer vision algorithms they were using. So I said they're putting together a world-class team to build this product and to build this hardware. I said that makes sense to me and that's why I recommended that we put some money into the company. So that kind of answers some of these questions of like, what was the energy like for someone coming in from the outside at that moment back in 2014? So years later, I did join right after the launch. And while the hardware had evolved to a different form factor and the optics did look different from what I had seen back in 2014, then the energy in the company when I interviewed was palpable. I walked away saying, I really want to work with these people because they're all so passionate. And as I'm sure you know, that there were no leaks back then. And that was a testament to just how much everyone cared about what they were building. And everyone believed in it, that no one wanted to just sell out and give away secrets or, you know, leak anything. They're just like, this is a future that we all believe in together. I mean, these are things that you don't hear about, right? Like that people don't talk about, is it? Just what was that culture inside Magic Leap? And after launch, when I joined, then, I mean, Andre, you were there at LeapCon, the energy, the developer community, everyone was so fired up. It was amazing because we all tasted the future and we were all working towards it. So it was exciting. We would work. way more than a 40-hour work week, and we did it happily because we all believed in it. I mean, Tricia Katz, I'm sure you all know from the evangelism team on DevRel, then there were times that she was flying to three different cities a week running workshops, and I mean, that takes a toll. But she did it for close to a year. to complete the mission. And there were so many stories like that of people working long hours but happily doing it because we all were working together on this mission. And so that's something that, again, I feel like is really missed a lot. So I can just keep going, but I want to pause and be able to allow others to interject.
[00:11:20.267] Andre Elijah: Yeah, I wanted to talk about LeapCon for a second. Yes. Because you were tweeting, obviously, when I did the demo on your unit at the hotel bar, and I'm sure we'll get to that night at some point in this whole thing, because that night was pretty special. But just talking about the energy and everything happening at LeapCon, it's funny, because when the ML was finally revealed, that fateful day on that Twitch stream, I think it was a Twitch stream, with Alan and Shanna, collectively, the world kind of let out a grunt and a groan, because the content that was shown to us was not what we were led to believe would be the content. for you know the years prior between the original concept video of the whale in the gym or the elephant in the hand and even Dr. G. I remember when they showed that initial demo of like the little golem guy that looked like match up from Pokemon you know just kind of throwing the rocks or whatever it was and it was like oh shit What the fuck were we all just hyping and getting excited about all this time? And, you know, people were talking about what they're building secretly with Magic Leap, you know, a lot of medical companies and the traditional industrial applications that you would see on a HoloLens. And then there's this vibrant creator community of indies. And that's what I got hyped about when I showed up in L.A. because that happened the same weekend as Canadian Thanksgiving. And I had left family early to hop on a flight down to LA and check in at the hotel and show up for my first day of LeapCon. And the first day of LeapCon was admittedly a complete disaster with it not actually starting on day one because none of the shit arrived to start the conference. And so we're all in this building, and it was full of Indies that didn't care that day one wasn't happening, because they got to show off what they're building, they got to talk about the promise and the future of the company. And the excitement was absolutely insane. So from your perspective, just talk about what the indies were doing in the space. And there's a lot of people like Alex Colom. I still remember his demos. We had bad VR there that were kind of killing it, showing off their visualization stuff. Can you just talk about what it was like being on the Magic Leap side, but then seeing the excitement despite everything going wrong?
[00:13:41.213] Steve Lukas: The developer community was amazingly top-notch. That was such a fire and excitement that helped kind of egg on the developed relations team and just it was so satisfying working with all the indies. And the thing is, you know, AR, VR, that whole space, there's a lot of investment needs to be put in to create quality content and as indies, then your resources are limited. So we wanted to help as much as we could, but the thing is, I mean, you know how long games take to make, right? XR is really easy to make some awesome prototypes, and those are important. Taking those all the way to production and having to think about all the different ways a human being will interact with your application in 3D space, That's really hard. It takes a lot. And that doesn't even count having to deal with all the revisions in software, like all the SDK revs, both on the platform, like the Magic Leap SDK, as well as the game engine, like Unity or Unreal, and trying to keep up with all of that with the same project even. I mean, I was looking at a project last week that just took me a full day to try to figure out why a shader wouldn't compile anymore in Unity 2020.2 when it was working fine in 2020.1. And that's not even innovating. That's just feature parity. So there's a lot of challenges in just building these applications. We understood that. So as a DevRel team, we try to do our best to support the indie devs and try to help do end runs on the issues they're facing. But again, there's a lot. And XR is still emerging. There is no standards for UI save for what MRTK is doing right now. And MRTK wasn't even compatible with Magic Leap until five days ago. So on the inside we were doing our best and we were trying to provide as much as we could and listen to the developer community as much as we could because we wanted to keep them alive. These are the people who bought devices right at launch and these are your number one supporters and you don't want to say well hey sorry you're not giving me ROI so we're gonna look over here. You know as a DevRel team you want to embrace that energy and you know take it to the next level and use it to inspire more people to come into the scene. I mean that whole bow and arrow thing was something that I did before I even joined the company. I was looking at what can we do with this device and I was thinking what can we do on this device we can't do on any other device. So I took the SteamVR longbow application that's open source, right, from Valve, and then ported it to Magic Leap, and then was like, this, I can't do this on anything else. And that's why I brought it to you, Andre, and it sounds like, tell me if I'm wrong, did you say that that's why you bought a device?
[00:16:22.128] Andre Elijah: Yeah. I mean, it was you and your demos and just kind of the excitement from everyone in LA on that first day. Cause I remember I was on my way back to the hotel and I got a message from Alan cause I pinged him saying, Hey, can we like buy magic leaps? Like what's going on? I remember I was back at the hotel and he pinged me and he was like, yep, I can get you in to buy one. I was like, shit, I'm heading back. And so I literally just got back to the hotel and I turned around, went back. and then picked one up and you know that was that night and those couple days playing all those demos and just talking to devs you know it felt like we were in uncharted space yeah you know anyone that was coming from the vr side of things you know the vr community around the oculus dev kits and early day riffs that was already forming slash semi-formed so wasn't as exploratory and experimental as what was going on with magic leap And you had people that had access to devices for their day jobs, but then at night they're all tinkering and messing around with this stuff, just trying to see where they can push it and what kind of weird experiences they can make. And then there's you, that obviously, you know, you had the Longbow demo. But we can also talk about SpaceYate, your app, and what we were doing that night in the hotel bar where you were setting up, I think it was the hotspot on your phone that was running SpaceYate. We had an iPad and then we had a Magic Leap, and all three users using each of those devices were all drawing and communicating with each other in 3D space, all completely synced up. drunk in a bar. And that was the coolest thing ever. After we partied underneath a space shuttle. That was the coolest night, probably, in my entire career in Immersive. Oh man.
[00:18:09.243] Steve Lukas: Why you gotta trigger the emotions? Sighted And that was the goal and it's so meaningful for me to hear you say that Because one word you didn't use I mean you said in a hotel bar that was all completely untethered Then a lot of great VR is definitely tethered and so being able to do these experiences Anywhere this all happened what a month after Magic Leafs launch or maybe two months after yeah, but I talked to end of October Right, right. I had actually ported it over a week after Magic Leap's launch. This was before I joined the company. And I ported it over in like a day, what you saw there. And that's because I had it running on HoloLens and ARCore, ARKit, VR. I had them on everything beforehand because I had that startup. And the whole concept at the start was to have a cross-platform framework so that all the devices could play together. But then what really opened things up is when I was able to do demos in the HoloLens, because I said, I can do this anywhere. So I was going to the mall. I did a bow and arrow on HoloLens. And I took it to the mall to just try to play it out. And then I saw the kids moving train. And I was like, oh, no. Are we going to do this? So, I paid four bucks and jumped on this moving kids train that went around the mall and then I set up targets and I did a shooting gallery on a moving train in a HoloLens and I said this is the most amazing thing I've ever done and it's because I could do it anywhere. Now, imagine I did it on Wi-Fi. Well, guess what? That train's going all the way around the mall, and so you lose Wi-Fi if you're switching access points and routers. I was tethered to my phone on me, so I said, this is how you do XR anywhere, and not have to have any dependencies. And so then when Magic Leap came out, it's like, okay, now we can do this kind of thing, but now do it with a controller that I can hold down by my side and have the precision six off control, which you couldn't do with HoloLens. So these are the things that I wanted to show and bring. And that's, you described exactly the vision that I've had for this space for a long time.
[00:20:07.987] Andre Elijah: So let's keep on talking about content, because obviously you're building it on your own time with your own company and then join the organization. Anastasia, you were working with studios as audio director and making all that happen and sound great. Was there a content issue from your perspective? If content was different, would the platform taken off more so? I remember at WeebCon, I was the first in line with an appointment for the Star Wars demo with those stupid little alien furry things. Oh, they're porgs. And I was hanging out with Vicky Dobbs-Beck, who sat in on my session. And, you know, the content was fun. It was cool, whatever. She was like, what do you think of the porgs? And I'm like, I fucking hate porgs. I don't know why this is a demo. Like, it's cute. It's fun. But porgs aren't going to sell me a $3,000 device. And I don't know if they'll sell anyone else a device. What kind of experiences were you guys hoping would take off on the platform? What were you internally pushing? Obviously, Magic Leap Studios, had the undersea demo, had the last light, which we'll talk about as well, because I think that's the last greatest thing that came out on that platform. And then, you know, there's there's some collaborations with some big brands. What were you guys hoping would come out of that on the content side from the big partners or the indies? But yeah, I'd love to get your take on content.
[00:21:29.443] Anastasia Devana: Yeah, so I mean, my take in my professional role, I was part of the Magic Leap Studios, you know, which is the first party content creation branch of Magic Leap. So the goal for the studios was to create best showcases of the device and of the actual medium and technology, having the benefit of the earliest access to the platform, really. So I can't speak to like overall content strategy for the whole platform. I'm not sure if there was one, but I think at least like what studios was making, it's all really goes together with kind of like what the original target market was. The device was targeted as creator. I think they call it like creator edition. So they didn't call it dev kit. You know, it magically was trying to go for the consumer market right away, basically. So there wasn't a lot of. useful kind of utility slash enterprise slash, you know, productivity apps in terms of like partnerships and, you know, first party, third party partnerships. You know, there was emphasis on creative content, which it was super fun for me. I can't really complain. You know, that's definitely more fun to work on than, than the dry sort of business app. So actually the flagship first party piece of content for the Magic Leap One launch was Create. I believe Andrzej came out a little bit after that. And CREATE serves several purposes. For those who haven't seen the device or tried it or don't know what CREATE is, it's basically like a playful, almost like a toy box, sandbox experience where you lightly kind of learn some of the interactions and kind of learn about what the spatial computing is. They're basically like building blocks that respect physics and respect your environment. So for example, if you take the bouncy ball off the menu, it will kind of bounce around your room. So you learn to like grab things and move things and snap things together. There are also animated 3D characters that can walk around your space and they had some delightful little AI behaviors, you know, kind of built in. So for example, if you put a UFO into your space and you put a T-Rex, you know, UFO will go and like abduct it. So they were kind of fun little scenarios like that. So basically introducing the users to the space in a really light, fun, playful way. I think it did accomplish that quite well. And also the other role that it played, it actually improved the platform because some of the platform features like meshing, like the control and the video capture, if you can believe it, it was even worse before. But those features got improved through the process of the studios working on this project and saying like, Hey, this is not going to be good enough. you know, we need to improve this, or we need to make this developer tools better, and stuff like that. Does that answer it?
[00:24:25.425] Andre Elijah: Yeah, totally. That's actually really cool. Steve, what about your thoughts on content side, dude?
[00:24:31.470] Steve Lukas: So again, I came in after launch. So there was a lot set up leading up to launch. And I do see Paul Reynolds and Dave Shumway in the audience. And if either of them would like to come to the stage, if you want to raise your hand, I would love to hear your thoughts coming in, because of course, In the studio side, then there was a lot of content creation where that was driven by content strategy, right? Internal content strategy for first party. So as far as the content, then just take a look at what was launched, right? The content deals that were set up, you know, there was Angry Birds, there was Dr. G by LeapCon, you know, there was Create. It was a lot of games and entertainment. So this was set up as a games and entertainment platform. And we did bring in a lot of folks from different AAA game studios to come in. I mean, we had world-class creators working at Magically, building some of the most amazing things in the space that I'd ever seen. And again, we've talked about the 6DOF control, right? And the games that you may have played on Magically with the 6DOF controller in AR space, I've never seen any other platform be able to deliver that, which means that without a magic leap, you wouldn't have played these games today because it was technically not possible on any other platform in the market. So we were going after that. It looked like, and again, Paul, Dave, I'll leave it up to you to continue.
[00:25:52.283] Paul Reynolds: Hey everyone. Yeah. So what you're talking about, the content strategy from the early days, maybe.
[00:26:01.552] Andre Elijah: Yeah. And just your thoughts on it. And was it achieving goals that you wish they had gone in a different direction, et cetera, et cetera, just wherever you want to go with it.
[00:26:10.076] Paul Reynolds: Yeah. So I was working with Graham divine in 2013. I've been working with him for a few years and he got pulled in around that time to start a quote unquote studio, like basically a content group. Once he got committed and settled, he pulled, me and a few other people that he had worked with in. Our original plan was to build a lot of content internally. I think it's worth mentioning Roni was always involved in content and very excited about it, had a lot of opinions about it, was also really excited about the third parties that wanted to be a part of it. So back in the early days, we were going to build basically a game studio in Santa Cruz. We had an office there, and I was going to be a part of that. And then we were going to essentially build up multiple content teams. And I think a lot of people don't realize how expensive and hard that is. But the general idea was let's get people experienced in content creation working with this technology as soon as possible and don't give them limits in what we want them to do or what they can do. And so I think it was well-intentioned, but there was a few problems back at that time. One was that while that's a nice idea of like, hey, just let really creative, smart people that know how to build content build whatever they want and will build, hardware and technology to support those ideas. That's not really how a lot of products or art get built. So normally, anyone that creates things like this, some of their first questions are like, well, what can it do and what can't it do? Because I want to know what can I push and what's just not available to me, because that's the constraints I'm going to work in. Back in the early, early days, it was always like, well, what do you want to build? Because we're going to build the technology to support it. Then the other problem, which obviously I wound up focusing on, was the workflow and the SDK. Back then, we were brought on to build demos for the benchtop first. At the time, the workflow for the benchtop was, a Unity plug-in that Mick Orbeck had built that every time the benchtop changed, the Unity plug-in had to be changed and it was a zip file that was emailed around. The benchtop was, at that time, we were still raising the Series A and that was the primary demo machine. The only other demo was, I think we had a fiber scanning demo then and also a WD-1, which we never showed. We never put the WD-1 on anyone, which was the first wearable piece of tech. So back in the days, and I think this is a common theme throughout all the years, there's the grand ambitious vision and like, Let's just unleash people. Let's unlock people and don't give them boundaries. And then there's the people that want to do this stuff and work on it. And they kind of want boundaries to know, like, what are we focusing on? What are we building? So the early days of content were interesting. We pushed really hard on getting some more structure. It was always difficult. And then the third parties coming in always threw a wrinkle into it. And so, yeah, it was kind of crazy in the early days. We did a few things, like we made up demos to build and try to test all the key pieces of technology. The Santa Cruz team, while they were in place, built a demo called Gerald Goes Bananas, where Gerald's one of the six-armed gorilla guy. And we had a trackable what we called the totem, the gun that would shoot bananas into a basket and he would try to throw them back at you and it was just a way to test. We were really just trying to like get some sort of handle around connecting the dots between the amazing stuff we saw at the benchtop and saying, OK, I want to make content for that and then saying this is eventually going to be wearable and hands free and portable and it's going to be aware of your environment. And we just work from that angle of like, what are all these little pieces coming together and trying to help pull them together? So that was a lot of the first, you know, like I would say, the 2014 ish year of Focus. That's really cool.
[00:30:48.097] Andre Elijah: Dave, anything you'd like to jump in on?
[00:30:51.520] Dave Shumway: Yeah, maybe for a sec. I mean, a lot of it has already been said by Paul, but just this idea that from the very beginning, I think, Andre, you mentioned all the folks from the Hollywood set being courted by Magic Leap. There really was a desire and an interest to make content at the highest possible level. What everyone ran into really quick was this brick wall of reality in terms of what could the tech do, how fast was it moving, how quickly would I have to update my entire infrastructure and pipeline and project in order to take advantage of the new features that were coming from all these third-party engines and pieces of software that weren't made to mix with this yet. So the desire and the support from like Roni and others was always there to make the coolest, biggest, most amazing, spectacular stuff. But yeah, those early years were a pretty rude wake-up call of maybe just how far off we thought we were from being able to make that tip-top Hollywood top shelf kind of content.
[00:31:50.345] Andre Elijah: So I'd like to jump from that to almost the end with The Last Light, which I know Anastasia had a huge part in and I want her to kind of drive this next part of the chat. But the last light was a piece of narrative content that was made by the studio's team for the Magic Leap for the festival circuit. I think it was accepted to South By if I'm not mistaken. And ultimately around the time of the layoffs, everyone from Magic Leaf wanted it to get released and it wasn't going to be. And then there was a huge outcry from the community, at least from what I saw on Twitter and Reddit and etc. And ultimately ended up shipping to people's devices and we got to experience it and it was amazing. It made me cry. like a wimp and it was really beautiful so can you speak to it at all Anastasia about you know the process of making it and you know what the goals were for it and ultimately you know the decisions that led to it not being released and then somehow magically showing up in the store oh boy This is the one. This is the one.
[00:32:56.841] Anastasia Devana: Loaded questions. By the way, Dave was also a huge part of last night. He was the audio lead on it. And I also see Jeremy sneak in. He was the writer and creative director on it. So Jeremy, if you want to raise a hand, if you want to say anything about it. But in the meantime, Dave, do you want to talk about how it started? Because you were there actually earlier on that project.
[00:33:21.635] Dave Shumway: Yeah, I mean, again, to my previous comment, we always from the very beginning wanted to do really top shelf storytelling. And while things like Create and Undersea were really effective in showing what the device could do as far as like those initial steps for creators and developers to figure out what they could do and how the virtual content would interact with the real space. The desire, you know, from Rony on down from the beginning was, how do we tell a story? And is it different? And can it be different if it's happening in your room versus just watching it kind of abstracted remotely on a TV at a distance or, you know, maybe even less intimate on the phone, which is how most of us watch stuff these days. So it wasn't for lack of trying or interest, I think, until we got to the point of the last slide. But it took Jeremy, our creative director, coming up with this very simple, heartfelt, personal story that resonated with everyone on a level that was, it was interesting because while it took advantage and showcased all of these spatial computing elements that we wanted to highlight, what it really did was just kind of like pull the curtain back away from all of that and just help you experience the story of this girl and her grandma and this journey that they go on together. So it was an interesting kind of switch from doing things like Create and Dr. G where it was all about like pushing these technical boundaries and coming way, way back to the center of just How do we tell this personal story in a person's room and have that feel emotional, not think about the fact that you're wearing this crazy looking device and using this controller or using gestures and feeling like some space person, but just having this personal reaction to a very simple, basic story. I know that Jeremy's got some kids around him right now, but if he's listening, he probably ought to weigh in too.
[00:35:10.443] Anastasia Devana: But yeah, I mean, I will say that definitely the project I'm most proud of my entire career, because I think it was the culmination of everything that we learned as a team, as a studio, you know, over X many years that were there, you know, and yeah, the goal was to make people feel something, you know, with this thing. And I think we succeeded with that. And that's really, you know, to me, that's the only reason to make anything creative. is to affect somebody on an emotional level. Hey, I see Jeremy. Hi, Jeremy.
[00:35:39.447] Jeremy Vanhoozer: OK, so I'm both in full-time dad mode and learning this app. So this is going to be excellent. Trial by fire. Excellent entertainment. We believe in you. Thanks, Dave. Yeah, I would just back up. I mean, look, everybody's saying the true story here. We really got to a point with Last Light, you know, all through our content creation, there was this desire to tell a story because we felt and Roni felt as well that if we could get someone engaged into something they believed in, then the tech became secondary. And in reality, that was always the goal of the technology. I mean, you have the crowd that wants to have the tech specs and wants to understand, you know, how fast this thing runs and what it can process. But then the majority of people just want something to happen. And so whenever we would whiteboard out these ideas, we would put story content in a box and then we would either erase it or tuck it over into the corner because it was just so hard based on where the platform was, how much work it took to do well. Most of us came from other platforms. I come from a gaming background and, you know, narrative is tough. Making film is tough. Telling stories is tough. And so, on top of the technological challenges, like for our size team, for how fast we needed to move, and for as responsible as we wanted to be, because that was another thing with the later studios, we really tried to keep our budgets down, and that kind of kept us in particular boxes. So, as Dave pointed out, Create was really an exercise in getting people to enjoy something, and in the meantime, learn the technology. And so, with The Last Light, we finally went back to that on the pinboard because The goal was, look, if we can get someone to buy into a narrative that mattered, a character that's in their room going through something, that was a step for us. That was a step that we wanted to try to take. And we also knew that we couldn't do everything with one swipe. So, you know, I'm sure if you ask everyone who worked on Last Light, there were 10, 12, 15 features that we started with where it's like, OK, we could make it more interactive this way. We could really use the room more this way. And we really, again, tried to be responsible and just take a few swings to the Apple And number one on that list at all times was tell a story that people cared about. And so I also started making this when I was writing the script. It was actually very loosely started as a, you know, my, um, died recently. And I was like, okay, I would love to be able to somehow, not only would it be cathartic for me, but were some of the emotion that I felt, you know, in this experience and talk a little bit about loss. And so those first versions of the script were different. And then something amazing started happening where people on the team started coming up to me and saying, hey, this is so my story because and then they would reel off this amazing story about someone that they loved and the relationship that they had. So the last slide is extra special for me because it started as this one particular story that had a particular meaning and it became the team's story. And everyone has a contact point in this story. You know, everyone has different points of meaning. And so as we were putting this together, we began to realize it was working for us. And that was a big step. And then as we started putting on the technology. we started hitting a few points that were successful. We had a demo build that we started showing the studios, which was a huge moment for us because it was so different than anything in the building had ever seen. It was also a huge moment when we showed Roni. He only had a few comments. One was, don't make it a video game because, you know, hey, if people have to get in and start hitting buttons and, you know, it's us trying to be interactive, people are going to kind of disconnect from the story, which was great advice. And, you know, we stuck to that. But for him to see it for the first time, he had a huge emotional reaction to it. And then we started slowly showing people around Magic Leap, people from finance, people from marketing. We would just see, Hey, if you have some free time, come down to the studio. And I had the most surreal experience. I was in one of our little test rooms and we invited someone down and she came in and started to go through the experience. And what I had just lost her grandmother. And I'm of course watching her reactions. she's seeing but I know the experience by heart so I can go by audio cues and I know where she's at and she starts breaking down or sort of getting uncomfortable and she's crying and I'm like I start frantically looking around for tissue actually happened before and it was such a powerful experience I got to talk to her about it afterwards so I think at that point probably because I ran out and yelled to the team like, oh my God. But I think we were telling a compelling story. It was working well on the technology. And even though we were only taking a few calculated swings, we were hitting the marks that we wanted to. And so I know I'm rambling a lot now. I can obviously add to some of the how it came to be and maybe didn't come out as we wanted it. But I also want to give everybody else a chance to talk.
[00:40:21.920] Andre Elijah: Yeah, I mean, the one thing that I think I would love for any and all of you to get into is the cancelled release and then the story how it actually ended up out in the world because you know I saw the outcry on Twitter you know Anastasia was talking about it a bit as well and how she was disappointed it didn't come out and then magically it did and the responses were great so can you just speak to getting it out there in the world finally like who made the call was that like a roni thing that everyone just started bashing him with emails saying hey get this thing out get this thing out how did that happen
[00:40:53.648] Anastasia Devana: I know I've sent a few emails to a few of the C-level executives about it. But I know it was a concerted effort, I believe.
[00:41:04.333] Jeremy Vanhoozer: Yeah, I will say that Anastasia probably sent some of the heavier emails into the highest levels. But yeah look I'll be careful because you know there's obviously a lot of details and there was a lot that happened around this time and you know obviously there were the layoffs and Magic Leap which kind of changed the course for a lot of us but yeah I mean it was so surreal because the combination of that and the pandemic we were on such a high because we had hit these marks and the team was feeling amazing and we had literally gone from hey do we think we're going to be able to do this to oh my god i think we did this and then we were going to south by in like a week right like yeah like a week like weeks before the layout yeah we had been accepted into south by which was huge i remember announcing that to the team i mean just there was such a like we as a studio and as a team were finally finding our footing we had come from such a long path It was such a strange, you know, as you've heard people talk about, the studio had so many faces and it served so many roles and here we were just trying to be responsible citizens within Magic Leap and yet really try to do stuff that made us happy and really made an impact. So here we had hit this point where We made this story and it worked. And suddenly we were accepted into the South by and they wanted us to premiere there. And it was just, oh, my God, we run this run, the roller coaster going up the hill. Right. And so we were weeks away. And, you know, the pandemic was starting to happen. And was it going to be a big deal? And, you know, of course, looking back, it's oh, my gosh, what a ride that was. But so we kind of realized, OK, we may have a problem here. And so We were waiting for South by to kind of announce if they were going to cancel, because if they canceled, then maybe, you know, when we would pick it up again, we would still be in the festival. We didn't want to pull out of the festival. So there was this game of chicken that we were going through there. And I think every day that passed, we realized, look, this isn't looking good. And, you know, eventually I think we had to pull out because Magic Leap was like, we're not sending people anywhere. And then two days later, you know, South by canceled. So we sort of left that amazing high. Everybody went to their homes. We, you know, we were still working to finish up. certain aspects of the longer version. We had gotten into this flow of working from home. You know, it was still very odd with the pandemic kind of breaking out, but we were still working as a studio. Last Light was still there and we were still getting amazing reception. We just didn't know when exactly we were going to release it. Then, kind of out of nowhere, you get the fabled, Hey, there's an all hands meeting. And then of course, those were the couple of days we went through where, you know, the layoffs happened. Obviously there's lots of detail that I will try to avoid here, but it took us a little while to figure out like who was still there and who had actually gotten laid off. Pretty much all plans were derailed at that point. So there was this good week or so where we have no idea what's going on with the project. And I started sending emails. I think to Roni and other executives in the company saying, look, I just kind of want to know where we're going with Last Light and are we OK to maybe launch it now? You know, it was fairly packaged up. Poor Dave and Anastasia, as per every project, audio was really coming in towards the end because obviously they come in and really polish up the experience after most other disciplines are done. And so they were probably the ones that had the most ragged edges. And, I mean, I don't know, guys, if you want to talk a little bit really quickly about how you came in and tried to, like, finish this off even when we weren't on site.
[00:44:12.316] Anastasia Devana: Well, I think when, I mean, we kind of heard some rumors, right, about the event. So I think, like, they, like, was it, like, the night before we were there, just tried to finish the damn thing.
[00:44:24.695] Dave Shumway: Well, yeah, and for some context too, the version that we had prepared to show at South By was a festival version, like projects like this often are. And in order to get it submitted on time to get accepted, you know, this was like a three or four month old version. So there was like a lot of story and polish that we still wanted to put on the end of this that like the build for South By was fine. But the final, final thing, when all of a sudden we figured we might not have access to our computers tomorrow, it was insane. And I was working with our musicians over in the UK, and they were sending me files, and we were going back and forth, and we were all working from home, thinking we might not have another shot at this. So get everything checked in as soon as humanly possible, so that somebody that's left standing at the end of this inside, magically, could maybe hit that button and release the final thing. Because even if we got the South by thing, it wasn't necessarily the thing that we all wanted. And now that we felt like we might not have a chance, it was go, go, go, slam it in before we're fired.
[00:45:23.925] Jeremy Vanhoozer: I mean, yes, it was that ludicrous. It's funny now, yeah. Yeah, it's funny now. I can laugh about it now. Yeah, hindsight is an amazing thing. But it was really, I really can't emphasize enough how emotional this was. Because we had done all this work. We'd kind of stuck our neck out on this project that was really kind of out there. we had hit this high with South by and then that kind of got stopped and then all of a sudden it's like wait is this going to get shut down and so we just as people who cared about the project we were running around and because we wanted this thing to be as finished as possible and so obviously it happened you know everything kind of stopped and so like I said I immediately started sending emails and I know, as Anastasia has pointed out, people who were let go were sending emails. I was actually one of the few people in studios that were retained at Magic Leap, but in a very different position than I had in studios. And so as I was even trying to settle in and figure out what was going on, I was still trying to make sure that, OK, but the last light, what are we going to do with it? And people who had just been let go were emailing saying, hey, I'm really proud of this project. What's the plan? So after trying to pin some people down, you know, I got a fairly clear answer that look, this is a not going to happen and B it's not a priority. And again, I'm not, I have no interest in like throwing people under the bus. So it's, it doesn't really matter who said that, but what was pointed out to me was that, look, Magic Leap needs to pivot. And I understood that, you know, they made some big choices to pivot and they didn't want messaging to come out that, okay, Magic Leap's making these big moves to be a different company. And Oh, by the way, they're also going to leave some small little quirky narrative project. that doesn't seem to fit their new focus. But I think, you know, it was just so painful for me that, of course, I pushed back for a while. I was like, look, actually, I think this would be a goodwill. There was a lot of emails and texts that went back and forth. And I will say that there were several executives within the company who were relatively patient, but gave me a very solid answer in the end. And so I came to a point probably a couple of weeks in where I was like, look, I don't think this is going to launch. There doesn't seem to be a moment that this would come out and be good for magically. That was the message that got reinforced to me. It was one of those things where I had to disagree and commit because I had disagreed a lot. I had sent tons of emails. I think we had all sent emails. We all felt very sad about it, but it just didn't look like it was going to happen. But as you saw on Twitter and as you saw in all of our little voices popping up here and there, I don't think we ever fully bought into that, that it was a solid, solid thing. And I know there's a lot that happened in between here, but really how it got released was Magic Leap just didn't want a marketing pop that involved something very far off of their new mission. And so what happened was the SIGGRAPH show was coming up and they were kind of looking to see if there was anything that we wanted to talk about from Magic Leap. And I don't know, Dane, do you remember the details of how all this came together?
[00:48:16.650] Dave Shumway: I don't know, unfortunately.
[00:48:18.832] Anastasia Devana: Yeah, I mean... This was like part of the VR showcase, right? Yes. Like VRT or they called it or something.
[00:48:25.548] Jeremy Vanhoozer: Yeah, there was a VR theater at SIGGRAPH, which is usually, you know, you put on the VR device and it's almost this role of VR content. We were very outside of that realm. We were on a different piece of hardware. We weren't VR, but they were interested enough in this project because we cared about it. And it seemed, you know, it was narrative and it was different. And so they actually came forward. and provided almost marketing cover for the release. It's like, hey, we want to talk about this cool thing we've done. You know, there's a SIGGRAPH angle here of future technology and sort of narrative looking forward. And so they kind of got behind it. We were technically a part of the VR theater. So within the VR role, we played the trailer, which then pointed to where you could download it free in the Magic Leap store. And we kind of crafted this moment that was not very traditional. It didn't fit into even what it should have been in that VR role, but they were keen on like supporting it. And so for Magic Leap, that provided just enough of a bubble to say, oh, yeah, this would be why we would have a reason to release it. And so because it was part of the SIGGRAPH show and because they were supporting it and we finally I mean, literally this happened over a few days. So at that point, I was scrambling around. Steve Lucas, who I know is in here, was a wizard because everyone who worked on the project, almost everyone who worked on the project was gone. And so that included everyone who had built it, that included people who needed to put it in the pipeline for release. So between me running around with my hair on fire, making sure that I got approvals, which man, there's some stories about that. And Steve hitting buttons and flipping switches, making sure that this thing was ready to go. There was a few days there where who knew what was going to happen. But in the end, we ended up getting to release it, putting it on the store. And I think it was one of the most cathartic experiences that I've had as far as a piece of content that I've worked on. And it was so awesome just to see the team respond. to see the overall response to the piece. Yeah, man, I'm sweating walking around talking about it. That's how much we went through to get this thing out. But I'm just really glad that it went out. I'm glad that somehow, some way, we found a way to show it to people. And I think the biggest regret I have is that that team, many of whom are on this call, are just so freakishly talented. And we were just finding our stride. And I think the timing of everything was just really unfortunate. We were also very fortunate to have something like Glasslight come out and at least give a hint as to where we were going because the projects that we were looking at going forward were more of that and more interactive and more AR. So anyway, that is our rambling tale.
[00:50:55.799] Anastasia Devana: So, you know, if anybody wants to start a new XR content team, half of us are here.
[00:51:04.358] Andre Elijah: That's really awesome. I'm now using The Last Light as a reference for, you know, projects that I'm pitching and discussing in the narrative realm. And it's probably the only piece of content that stands as the reason why I will never sell my Magic Leap. So I've been clearing out all my older HMDs and the Magic Leap is staying just because of that. So thank you for getting it out ultimately. That's awesome.
[00:51:27.835] Dave Shumway: Oh man. Yeah, that means a lot for sure.
[00:51:31.492] Andre Elijah: One thing I want to touch on before we open this up to Q&A, I know Kent wants to jump in with a question. Steve mentioned earlier that when he was an investor, he got a few hours with Rony, and then in the time that he was an employee, he had magically really had no communication with Roni. I'm curious, what was it like working for Slash with him? If you can speak to it at all, even in general terms, obviously I don't want the nitty-gritty. You know, from the outside, on Twitter, you know, obviously he's got Twitter fingers, he says a lot. You know, a lot of it probably didn't help the PR situation with Magic Leap. in terms of the posturing of what the device was and where it could be versus the reality. We've heard the rumors of how many units actually moved, and we've heard his grand ideals and plans for the company. Was working with him cool? Was it not ideal? Or did you guys really not interact with him much at all outside of getting a sign off here and there? What was that like?
[00:52:40.218] Jeremy Vanhoozer: I'm waiting to see who's going to talk here.
[00:52:42.439] Anastasia Devana: Yeah, Anastasia unmuted first, so... Well, it's a... I've got to be careful here. So I'll say that, you know, what you see on Twitter, I think, is a very accurate representation of Roni's personality. No, that's it. That's all I wanted to say. I mean, I think what you see on Twitter, in my experiences, pretty much, that's Roni.
[00:53:11.840] Dave Shumway: I mean, what I would say as well is Magic Leap now has this sort of known information of being a pretty big company, and there were a lot of people up before the event. And there still are a lot of people. In the way that a lot of us saw Rony operate, it was still, you know, very small, scrappy startup CEO mode, where most of the time he's out raising money. And I mean, that's not really a unique thing for a CEO at a startup. You know, their job is to be the dreamers and the sellers of dreams. And so to not see him for any length of time and then to be able to show him something for a pop and get a nice, cool reaction, it was always really fun. And he was always a really huge fan of studios. But as far as day-to-day interactions go, I don't think that it's probably too dissimilar from, you know, a CEO and a tech startup looking for that next big investment.
[00:54:03.324] Jeremy Vanhoozer: Yeah, I mean, obviously, I will tread very lightly here. interesting and unique perspective, probably because at the time, Ant Williams, who was running studios, is good friends with Rony. And sometimes we would get these little impromptu meetings to talk about creative or, you know, I also had an opportunity to pitch, you know, creative things to Rony and he gave me tons of feedback. I would say that, you know, I mean, I've had a meeting with Rony at IHOP. I've had a meeting with Rony at a vegan Indian restaurant. I mean, it's a really it's a really interesting food related meetings with Rony. But, uh, I think for me, it was interesting because Roni didn't really have a lot of creative bounds. And in some ways that was amazing. In other ways, you know, I think it was also terrifying because, you know, you would work really, really hard for months to get something to work on device. And you're like, Oh my God, you looked around at the team. Like, this is a miracle. We did it. And then Roni would come in and be like, can I get like 10,000 more of those? And know in an email. And so I think, you know, his boundless optimism and, you know, his, his just like, yeah, let's make it bigger, better. Like, I think you need that. I think it was an interesting role for him to play the CEO and the sort of creative Willy Wonka character. And I think to be fair to Roni, I think he didn't really have the counterpart. You know, he didn't have the sort of partner at Magic Leap that was able to kind of help him put things in bottles. And so I think, you know, at Studios, I'd say we had a pretty interesting view of Roni because we were trying to push the creative boundaries. And if I think I would summarize it this way, you have a group of creatives that's actually trying to add the responsibility to the creativity. That shouldn't really be right. I mean, the CEO should sort of give you some boundaries and then you are a bunch of crazy creatives that go out and work on that. I think for me, it's such a I have a very conflicted vision and view of how to even answer this question, because there were definitely good supportive moments. And then there were other moments where I wish things had gone completely differently. But I think that, you know, I guess you could say that for a lot of these situations. Hey, Tim.
[00:56:08.321] Tim Stutts: Hey, Andre. How are you, man?
[00:56:10.143] Andre Elijah: Good. How are you, dude?
[00:56:11.864] Tim Stutts: Yeah, I decided to go ahead and raise my hand.
[00:56:13.725] Andre Elijah: So. Awesome. Glad to have you up here.
[00:56:16.305] Tim Stutts: Sure, sure. So Tim Stutz here, lead designer on LumenOS, worked on text input, settings, object recognition, a lot of stuff on the platform to support great experiences like Last Light. So I can talk a little bit about my experience and my story going in. I have many, many good things to say. I joined Magic Leap really early on, 2016. I guess some of us have been there even longer, but I know that when we were at our peak capacity, I was among the early 50% at one point of total employees. I remember that. And I got to work on things like text input for the OS. So this is like a system level keyboard that you use to log into the device, enter your IP addresses, mundane stuff like that. But it was a lot of fun and started with prototyping in Unity and then eventually working with our software team. which, as you know, decided to create a whole new operating system called LumenOS. That's basically what my team, the user experience team for the OS, dealt with for the years that I was there. It was interesting seeing that come together. I remember As a team we'd like prototype different keyboard concepts for the operating system and we kind of made a decision on which ones to move forward with and we ran those through a user trial and then we actually got one through to Rony in the winter of 2016 and he approved it for use on the device and that was really exciting but at the time it was only a Unity prototype. And so in the following year, I worked with the team to spec it out for the operating system. And that meant basically working with something that was very rudimentary at first. Like in the early days of LuminOS, it was only geometric primitives. And I remember when text fields and buttons came live. And so the keyboard kind of grew with the operating system. You know, fast forward, like, over the years, it was just a very wild ride, but the UX team under James Powerly, we had a great run going up to the launch of Creators Edition. We knew, like, as people working on the OS layer, arguably our job wasn't to wow. We were just there to create a supportive platform, but we had some of the core apps like gallery and capture and what's now called C3. This is something we worked on that was native LuminOS and we had some interesting challenges. I mean we didn't have like the capabilities of Unity or Unreal so we were kind of flying by the seat of our pants trying to figure out how to make things look and function well in a new framework but really happy there with my experience overall around the time last year I guess last year in a day wow crazy it was a hard time for us as a team we lost like most of the people on our team were were impacted and that and that was really quite a tough time but my colleagues and I have stayed in touch and I've also stayed in touch with some of you on this call and everyone's doing interesting work now and I just had like a anniversary get together last night with some friends at a local pub and yeah I have a lot of hopes for ML2. I think it's going to be successful in the enterprise space and I was sad to see the leaper go from the website but I think enterprise is the right direction moving forward for the immediate future. So that's all I have to say.
[01:00:02.926] Anastasia Devana: And did Tim and I have a fun experience together working on the songs for the lyrical thing?
[01:00:09.135] Tim Stutts: Do you want to talk about that actually for just a few? I mean you can if you want.
[01:00:13.347] Anastasia Devana: I think it might be interesting a little bit. So a little-known fact, but the main sound designer we worked with to create content for the user interface sounds for the Magic Leap OS was Ben Burtt. I'm not sure who he is. He was the main sound designer for the original Star Wars trilogy and some of the You know, other films as well. But of the Star Wars trilogy came. So Tim and I and Brad Shipes, our technical sound designer, we actually took a trip to the Skywalker Ranch and to meet with Ben and to talk to them about the project and everything. So that was fun. And one of the highlights about that, so we have one of the kind of like internal demos. So Magic Leap had like internal demos that were shown maybe to some, you know, visitors, but never got released publicly. And those are really, really cool. They were on the WD3, which is a wearable device tree, not the actual final shipping form factor. But one of those was this laser sword. TM. So one of my favorite experiences from Magic Leap is Brett, the technical sound designer, who got to actually create this laser sword TM on Magic Leap. And it was a really awesome, kind of like mind-blowing audio demo. when he did a 40-minute slide presentation to Ben Bird about how he created the sound of the laser sword on Magic Leap. I thought it was hilarious. I was like, that's it, Brett. This is your career highlight. It's all downhill from here.
[01:01:47.298] Andre Elijah: That's really cool. I wish I could go to Skywalker Ranch. A buddy of my dad's went there to appraise the property. So that's the closest I've been. I was checking out the inside photo.
[01:02:01.739] Tim Stutts: Yeah, it was a lot of fun. In the beginning, before we had a relationship with Sonic Arts, I was more involved as an IC on the sound side. I was creating temporary sounds because we didn't really have a solution there at one point. We were kind of a little bit limbo. I just think one of the highlights for me, one of the things being the virtual keyboard, which I like to say is the What I'd describe as the least crappy virtual keyboard, because all virtual keyboards are no fun. No one wants to type on their Apple TV remote. But one of my highlights was seeing the temporary sound design I'd done to the keyboard, and have Ben Burke come in. and do it proper justice. So watching a video back of the keyboard typing with the guy who did the sound effects to R2-D2 providing sound design on the keyboard was a really special moment, a highlight for me.
[01:02:54.360] Andre Elijah: That's really sweet. I'm glad that you joined us, Tim, because that story is awesome.
[01:02:59.264] Tim Stutts: Yeah, man.
[01:03:01.313] Andre Elijah: I wanted to open this up to Q&A. Kent's been in our DMs saying that he wants to get up here, so I'm letting him up. So Kent, where are you? There we go. Oh, there he is. Cool.
[01:03:12.557] Kent Bye: Hello. Thanks for hosting this conversation. I have a comment and then a question for the panel. So the comment is that Paul was in here, but I think he may have left. But back in 2015 at GDC, he wanted to come onto my podcast, and at that point, I think Magically could only really talk to maybe two or three other press at the time. And so I was able to do an interview with him to get a little bit more inside scoop as to what was happening from a developer perspective. But then after that, it was basically like three years of being completely shut out with having any sort of meaningful relationship with Magically at all, up until LeapCon. So I show up at LeapCon, and I'm there to do interviews. and, like a lot of people, show up on that first day and we thought that we're gonna have the keynote, it's gonna start, and then we all learn, like, okay, it's not happening, and I was there just kind of milling about, and then the PR people come up to me, and they're like, I'm sorry, but you can't be recording here, you can't be in the hallways talking to people, and I'm like, are you kidding me? Like, I'm pressed, like, I'm here to be talking to people, and I was able to thankfully convince them to allow me to kind of mill about in the hallway to talk to people, but they originally didn't want me to be there at all, and I'm like, These people don't know who I am and they're there to get rid of me. And then I go to the opening night party and I have my press badge on. And I'm about to walk in, and someone from Batchelor comes up and says, I'm sorry, but you can't come into this party. And I'm like, what? I'm covering this community. This is how I understand what's happening, and you're going to prevent me from going into this party where I'm going to be able to talk to the developers. And so what I ended up doing was standing outside as this protest. And it ended up having lots of different Magic Leap employees come and have almost like an internal battle where they went up the chain until eventually, I think it might have been the director of engineering was able to kind of override the PR person that was in charge at the time. And then I was able to get in. But I think the reason was that they didn't want to have me overhear any conversations from any Magic Leap employees. And so I guess the question I would pose would be that there seemed to be like this over-emphasis of secrecy over a number of years, but in looking at what Oculus had done with having kind of like an open ecosystem with the sharing and getting stuff into the hands of developers very early. There seemed to be almost like, in order to have access to the Magic Leap, you had to have it under lock and key, you had to do all these additional things, and there seemed to be a lot of additional hoops that were really focused around the secrecy, and kind of like Apple-esque, like let's reveal in a big moment, when really, it seemed like that that secrecy was harming the cultivation of the ecosystem. So, I would just love to hear any comments from inside about that tension between how secretive everything was versus whether or not that was beneficial from a PR perspective or whether or not ultimately that level of secrecy went way too far and ultimately made it more difficult to really cultivate a viable ecosystem.
[01:06:20.347] Steve Lukas: I'm going to speak just because from what I can see, and anyone here can correct me if I'm wrong, but all those policies were things that none of us on the panel had anything to do with. And nor was it anything that we were involved in or was communicated to us. So I'd like for you to get your answers, but I don't think that's something that any of us were privy to. Because you look me in the eye when you're in line and you're like, I can't get in, Steve. And I didn't even know where to go. I mean, I had only been at the company for a month. But I had no idea whose call it was or where it came from. And there was so much going on. I wanted to help. I just didn't know where to turn. So that's probably about all I can say that isn't just going to be more words for you, Kent. I mean, we met up. And I showed you stuff during the conference. And it was not what I expected either. Yeah, that's about all I can provide for you, buddy.
[01:07:18.682] Dave Shumway: I mean, I guess the only thing that I can say, and I think all of us were probably involved in LeapCon to some extent, but LeapCon maybe functions as like a microcosm of the larger story, which is just magically trying to do everything at once and to be everything all the time. And that's really, really hard to do. And that's not an excuse for why whatever happened at that singular event at LeapCon, Speaking more broadly, owning the hardware and the software stack, and the store, and this and that, and putting on this developer conference for the first time, and trying to figure out how to launch it, who the customer is. I think maybe some other larger companies take smaller shots. Maybe Oculus isn't the one. But definitely, from the inside, you asked about our insight and perspective. There was definitely that tension kind of all the time of wearing so many hats that it was hard to push forward in any one direction. And maybe that came to a head at LeetCon and what you were feeling.
[01:08:18.753] Andre Elijah: I'm going to let up future meme to ask their question.
[01:08:23.140] Nick Marks: Hey, yeah, I just wanted to comment. Hey, Steve, it's me, Nick, from Game Gen. Yeah, I've been loving listening to all this stuff. This thing started off with such an interesting comment that Andre had, and I think it speaks to a lot of the problems. And I just, first of all, want to say I love Magic Leap. I drank the Kool-Aid. I still work on stuff inspired by them. But, you know, it's this interesting thing that Ken just brought up, these layers of infrastructure. that happened and yeah Andre you brought up a moment that I'm like I need to say that one just because it's like the antithesis of where stuff could have gone wrong you know you talked about Steve showing you that amazing demo of his multiplayer on like Android or iPhone linked up with Magic Leap and yeah I saw that too like I went to the first Magic Leap hackathon And Steve was one of the first reps I met from there. And it was like, Steve just blew my mind. He was like, let me show you Space Sheet. I'm like, cool. Let me show you multiplayer. I was like, cool. Oh, my God. Multiplayer is going to be the most amazing thing, like multiplayer multi-device. Like, that was it. And so I bought like three Magic Leaps and worked really hard in getting my own multiplayer demo. And I focused on it from education, right? So, you know, fast forward to like, I don't know, a year and a half later and my team won the Magic Leap Hackathon for, I don't know, it was IBM or Intel or whatever.
[01:09:57.193] Steve Lukas: Or it was AT&T, AT&T Shape, FYI.
[01:10:01.777] Nick Marks: Yeah, AT&T Shape. Thanks. Thanks. Thanks. This is pre-COVID times. I've lost those memories. But yeah, fast forward to Rony and the CEO for AT&T come over and I'm showing them my great idea of like this spatial classroom for education. And they're like, holy crap, you can like connect an Android and have multiplayer experiences. And I'm like, And it was so depressing to me because that was Steve, like Steve came up with that thing. And it was like they were so impressed by technology that like we made ourselves uniquely. But it was, you know, Steve was the one that did it way back years before. Right. And so I just feel like there was a fundamental like if I'm throwing out a postmortem there and, you know, and I guess I'm not employed by magically so I can throw people under the bus is, yeah, I think what Kent just said, there's so much infrastructure that had that that I think people internally couldn't get their ideas out and get them in front of the right people. Because, yeah, a lot of this, I mean, not to take anything away from anything anybody's done with Hollywood people and narrative, but like just core multiplayer game development stuff. just didn't happen and amazing ideas like Steve's stuff didn't even make it to Rony's eyeballs. And so yeah, I think there was just issues there that I wish had been solved. And yeah, I just wanted to chime up here and say what a genius Steve is. So I'll shut up.
[01:11:37.282] Steve Lukas: I don't know how to respond to that. Thank you, Nick. I really appreciate that. Steve, I hope you pay that man. I hope you pay him well. Well, we'll see once I get a job again. I'll see what I can do. But yeah, I mean, this does, I did say earlier that as an investor, I got over three hours of time with Rony on the other side of his desk. And once I was in, I did try very hard to present him with what I had. And I know Allie Heston, who is also part of the Spaceship team, was able to show him during her interview. So I thought that was great. But yeah, they had different priorities. And that's all I'll say about that.
[01:12:15.526] Andre Elijah: So speaking of priorities, I have to ask a question. And you're probably going to say, can't comment. But what the hell happened with the grants? The Magically Grants, or I don't know what the official name for it was, the creator, something or other. You know, you guys funded a first wave of content, I think. Maybe Within was in there, Felix and Paul was in there, and a few games, a few. I think Spatial was in there as well. What happened there? Was it an effort on the side of executives to just idea farm and figure out what the fuck people even want to do with the devices? Or was there an intention to really kind of fuel the creator group that we were talking about that was at LeapCon and all those indies? Like, what was the impetus behind that and what did it turn into? Because I know when the grants were announced, every indie went crazy and submitted pitches. And then when the actual finals were announced, there was a bit of a revolt everywhere. Can you speak to that at all?
[01:13:13.191] Steve Lukas: I think I'm the only one on the panel that can say anything about it, and I'm gonna keep it limited for lots of reasons, but you've heard what I said earlier in the talk about how I feel about DevRel, how I feel about embracing the indies, and the intent was absolutely there as the program was created to embrace all of these indies who had been sparking the world with the excitement of Magic Leap. And like I said, showing you the bow and arrow, the whole point was to pretty much infect everyone with the same enthusiasm that I had and let that grow in scale. other decision makers got involved and my level of influence was not as high as I would have liked and I can't say much more. So, I felt your pain though because I wanted the same things you wanted, Andre. And in future roles, that'll be a lot more apparent.
[01:14:05.977] Andre Elijah: For sure. I appreciate you commenting on it, dude. Is there anyone else in the audience that has another question? I think we can do this probably like another 15 or so minutes and then wrap it up because I have work to do, but decided to do this because the timing was perfect with the one year after layoffs. Hold on, we got someone coming up. We got Nicole Lazaro.
[01:14:25.487] Nicole Lazaro: Hey there, greetings. This has been an amazing hour or so. Thank you everyone for the candor. I was curious about the future of AR, since the folks in the room are a big brain trust. You have played and designed stuff that us on the outside haven't. I was curious, what do you see as the future? I wanted to talk about the entertainment side. What do you think is going to be happening? Where's going to be the deepest fun stuff that we're going to be playing? I'm Nicole Lazaro. You can look up my bio. I run Xeo Design, Xeo Play. I created Aladdin's Cave of Wonders at the AT&T Hackathon. which is, you know, you walk a spatial, you know, we put a big room full of trees and you could gather gems with your hands and stuff like that. It was just amazing. And I really, I really love the space. But I was curious, like what you folks think are going to be the entertainment applications in the future? Thanks.
[01:15:19.998] Jeremy Vanhoozer: I can maybe say a few words here. Yeah, it's really interesting because going from studios at Magic Leap where, you know, we were steeped, I mean, we lived and breathed this stuff and we were breaking into these same questions every day to stepping outside of this realm, you know, that's thinking about AR or XR or whatever, whatever two letters you want to pair with it at any given day. You know, I moved back into the more traditional game space for now. which is also really cool. But when you turn around and start talking about things like AR, it's still such a small window, right? I mean, being in the space, you're like, OK, this is all I think about stepping out of it. It's still relatively small. Now, all of us know the space in general, and we know the big players are getting ready to come in. And yeah, we get that. So it is interesting kind of looking at how I did think about it and how I'm thinking about it now. And one of the things that really strikes me being outside of the bubble is how normalized it still needs to be. And so I think, you know, whether that's Apple or Facebook or, you know, or a surprise person that pops up with a genius device, taking down the life tax of putting on a device or making the speed bump as low as possible just to getting into the space, I think is still such a huge thing that I think before we see bigger entertainment experiences, even the sort of co-located physical and AR experiences, I think AR itself's got a few steps to go. And by that, I mean the usual suspects, meaning, OK, these small experiences that get people to do the thing that break past the hurdle. So, for instance, can my run be plussed by an AR overlay? Yes, it can. And someone's going to do that really well. Can we add a little bit of gamification or storytelling to something that you do on a daily basis with very low barrier to entry? Yes. So I think as someone who builds things like my mind has gone there more now that I've stepped out of sort of the saturated AR, XR bubble and thinking about those small experiences to get us to the big experiences. Having said that, I would also say things like COVID has really slowed down like those shared spaces, big public spaces with AR overlays. I think they're going to come back. I think it's going to be amazing. But the thing that really struck me when I was in studios is the fact that VR is I can be in my room and I can put on the device and I can be anywhere at all. You know, I can be immersed in this huge cathedral or wherever I want to go. To me, AR was the most interesting because I could bring stories to my space and I could include my space in those stories. To me, that's what we were sort of pushing on is how is your room a character in the story? How is the space you happen to be in a participant? in that bit of entertainment. How can I make something feel more intimate? You know, scale plays a lot with that. You know, your relationship to character space, the level of interaction, you know, gestures versus control, all of that stuff is going to play a role. So I know I just did a lot of babbling, but I am big on getting the small things done well to get people to buy into the space and then sort of expand from there. So I think there's going to be several steps before we go big, big entertainment.
[01:18:18.627] Anastasia Devana: Can I say the opposite? I think I actually say whatever you want. The opposite.
[01:18:22.221] Jeremy Vanhoozer: I actually I hope you say the opposite. I don't feel like we're back in the studio. And this is awesome.
[01:18:26.522] Anastasia Devana: Yes. No, but I think to the point that you made about the actual look, you know, of the devices or whatever, not being socially acceptable, that's probably going to be for a while. You know, of course, once Apple comes out with it, it's going to be the coolest thing ever. But to that point, I think location based like once people are allowed and feel secure enough to put shared devices on their face and go be in a closed space with other people again. I think that kind of takes away the social aspect. And actually, like a couple of internal projects that Magic Leap did that were kind of like larger scale location-based, they were super effective, right? So I think there's definite immediate kind of potential there. And then the other thing is I think either Steve or Andre were saying that. experiencing and like interacting with the content together with other people, like co-located. I think that's another sort of thing that's gonna be, you know, very magical and exciting. Like not necessarily location-based, but just local kind of multi-user, multiplayer type thing.
[01:19:34.266] Javier Davalos: So, hi. Hey, Javier. We've met, we love you guys. So it's more of a comment you were asking about the grants program and Steve couldn't answer. But I'll tell you my experience because my company, Overlay, we were part of it. And you know, I tell you my feelings. My feelings are that it was gaining momentum. There was like really interesting stuff coming out. But then, you know, I feel like with the layoffs, you know, I felt kind of like laid off myself. Because I felt like a, I really felt like a leaper, you know? Especially Steve and you know all the DevRel team, you guys made me feel really like part of you, you know?
[01:20:16.764] Dave Shumway: I was pretty sad.
[01:20:17.345] Javier Davalos: One of us, one of us, one of us. One of you guys, yeah. So, but I tell you, yeah, I felt laid off, but I don't resent anything that happened, you know? I am the opposite, I am like so thankful for everything that you guys did. You guys really basically started up my company and now things are looking good, you know? So, you know, I just want to thank you, you know, Steve and all of you guys for what you did. And I just want to say that I don't resent anything that happened with the Grown-Us program because I don't believe at all that it was your choice, right? Like, I don't believe you choose to, like, move out of entertainment and jump to enterprise. It was just, like, something that had to happen, so. That's all. Love you all.
[01:21:04.568] Andre Elijah: Thanks for joining, Javier.
[01:21:05.829] Steve Lukas: Yeah, thank you, Javier. It's always great to see how passionate you are. I want to see that tilt brush inside Figman like you showed, and I know we've got some infrastructure to correct on the inside to be able to unlock you. So I'll still keep pinging my guys that are left behind, and you keep doing the same, and we'll get there, man.
[01:21:24.260] Andre Elijah: So I just wanted to let up Joseph, he had a question, and then after that, I guess we'll do closing comments if any of you want to say a final thing, and then we'll wrap this thing up. So Joseph, your show.
[01:21:35.424] Questioner 1: I had a question. As the company was working with Sigur Rós, I'm just curious, like, what was the band's sort of perspective on composing for that space? And I'm just curious if there were any, like, sort of interesting perspectives that came up during the process.
[01:21:53.910] Dave Shumway: Yeah, Anastasia, you were probably closest to that, for sure.
[01:21:56.832] Anastasia Devana: Oh, gosh. Yeah, so I worked on some of the early version of Tenandi, and I went to Iceland, I met the band. But unfortunately, like, I can't speak to the whole process, because, you know, after the first couple of prototypes, there was another team working on it. But I just know that, you know, they were very excited from the beginning, at least. I don't know how they felt towards the end. But no, I think the project turned out, like, really awesome. I think it's one of the best XR things that I've seen and experienced. So yeah, they were excited to do this to kind of push the boundaries and experiment. Yeah, I don't know. Sorry. I don't, I don't think they have any more comments on that. There is talk on YouTube from LeapCon and people who were working directly with Super Ross. It was a spatial audio LeapCon panel. So there's a little bit more takeaways there too. If you want to look for that, you might hear a little bit more insight.
[01:22:51.709] Dave Shumway: Yeah, just quickly, one of the things I remember learning from that, I didn't get to work on the project very much, but one of the things that I helped out with was the initial press demos when they brought Rolling Stone and some other people in to see that for the first time. And one of the things that we're trying to figure out is whether to use headphones, like earbuds or something. or just let it play through the speakers of the device. And if you've used any kind of HMD that doesn't have speakers that drop down, you know that they're pretty small and kind of tinny, not a whole lot of low end. And that's just kind of the way that physics works. But we also wanted to make sure that the press were having both like a comfortable experience and a good sounding experience. So we went through like a bunch of different headphones and earbuds and all different like configurations. And we did kind of a similar thing with South By for Last Light in preparing, you know, how should people listen to this? And one of the interesting things for me, at least as an audio person, was, yeah, I missed a lot of some of those bass frequencies when we just listened to it through the device itself. But it had this effect of bringing the sound into your space. And I think one of Jeremy's comments of like VR can transport you somewhere else. And AR with Magic Leap and others brings the content into your space. So the less friction in terms of what I'm putting on my face or sticking in my ears, I actually bought into that experience in a much more profound way. So, after listening to the Sigur Rós piece and going through it, you know, a dozen times and a dozen different sets of headphones, I just went with the default kind of magically built-in speakers and had just a completely different experience. And there were some things, you know, audio file-wise that maybe you missed, but the feeling of being connected to the content was something that was very surprising to me, even having worked with the device for, you know, a couple of years by that point.
[01:24:37.263] Andre Elijah: Thanks for that, guys. That's really awesome. We're at 145 Eastern Time. I'm not doing the math to figure out what it is for all of you in Pacific. I just wanted to know if any of you had any last comments. After being up here for what an hour and a half hour 45 minutes kind of reflecting on your time at the company Is there just anything you know that you kind of want to leave us with? Obviously, there's a mix of the good and the bad but clearly there's you know A lot of great memories and a lot of cool shit that all of you had worked on at the company So just want to see if there's any last comments Yeah, I'll um, I haven't been here the whole show.
[01:25:10.091] Jeremy Vanhoozer: So maybe I have a little bit more wind one thing I did want to stress, you know, and it's just cool to hear everybody talking again because I You know, no matter how it ended, I mean, you can ask anyone. I think this, I would describe the Magic Leap experience as a rollercoaster. But I still look back on it as one of the most creatively engaging times of my career. I met people who I will friend for the rest of my life. I think one of the coolest things is things that were impossible when I started at Magic Leap were not when I left. And that is pretty awesome. And also, I can tell you that just from that experience, You know, I have a diverse background in creative and art, but, you know, jumping into AR and kind of devoting my brain and heart to that for several years, I am on fire for the space. I mean, I have 100 percent confidence that AR and XR and knowing what we were working on, even just the looking forward part of studios, you know, the creating characters that would live in your space, you know, sharing rooms with digital characters that reacted to your space. the co-located experiences, you know, building physical sets with an AR overlay, like there's some amazing stuff, right, that is going to happen. And so I think for me, I mean, granted, there's a whole book's worth of conversation about Magic Leap. And sure, I think for most of us, there's lots of decisions that we would have loved to see go different ways. But I think if you sit us all down like this and get us talking, there's just such an amazing group of people that were attracted to this quirky company that somehow ended up in the swamps of Florida. It felt like the strangest summer camp with like, you know, a NASA level crew to really put some really phenomenal stuff together. So I would say that no matter what I do in the future, I mean, this will be a highlight. I mean, as you can tell by the people on this panel, just some really amazing hearts and minds that worked on this. And, you know, I will be eternally thankful for the opportunity to work with this crew, so I just wanted to make sure I said that.
[01:27:00.709] Tim Stutts: I'll go ahead and put some closing remarks in to echo Jeremy's incredible opportunity. For the team I was on, on the operating system side, just the fact that like our day-to-day interactions could be with marketing, hardware, electrical, industrial, engineering, software, you name it, was pretty magical. I mean like some days were spent going over and talking to the guy who was working on the LED for the wearable. And then in the same day, I might wander over to someone working on a new form factor for a new piece of hardware. And it wasn't a huge company, but it was big enough that there was a lot going on under this giant circus tent in Plantation, Florida. And you could just walk out into the aisle and find someone working on something new. And every other company I've had, has felt a little bit more siloed in that respect, that it's not as hands-on. You don't get to touch on nearly as many things. So yeah, it was a really special experience for our team to have that hands-on. I mean, our team got to work on haptic patterns for the control and LED patterns for all the hardware and talk to the power management guy about how often a notification should come up on your screen that says your battery is 5%. you know that kind of those kind of discussions and then have like our own content like with c3 and the launcher like the floating islands that you'd log into i mean just such a variety and richness of opportunities and i'm very thankful for that
[1:28:36.787] Andre Elijah: Thanks Tim. Anesthesia
[01:28:41.494] Anastasia Devana: well first of all i just want to just pretend I said everything that Jeremy said and Tim. I just want to sign under everything they said. I couldn't have said it better myself. I feel exactly the same way. And just one other thing to add is, I don't know if people were expecting some dirt, dirty laundry or something disgruntling to happen in this panel, but the sad thing is even people who were affected by the event are still the biggest advocates and champions for the company. And, you know, even now looking at the ecosystem and what's out there in terms of hardware, software content, I think Magic Leap One even is still all around still the best untethered, mixed reality, spatial computing device. And I think some of the content that came out, like creative content, is some of the best I've seen, you know, in the field. You know, and talking to people who are developing, like the SDK, I guess, was pretty good. So, you know, Magic Leap is not dead, luckily, you know. And I do wish them the best. I'm looking actually forward to ML2. It should be pretty sweet.
[01:29:44.370] Joe Gabriel: One quick thought. I worked on the Deja team with Steve. Hey, thanks for putting this on, everybody. So happy to see some new faces in here and some familiar ones as well. Just at one point or another, all of us have at one point been looking from the outside in at some company that we're just really interested in, really curious about. And what's amazing about AR, VR, XR, spatial computing, you know, is that it's still just the early days, you know, so I'm really happy just seeing all the different developers and creators here, or people trying to break into the industry, and just a word of encouragement that, like, finding ways to just do something, to get creative, to make something, to pop into these clubhouse meetings and get networked, like, it really is just, there is no silver bullet. AR is freaking hard, you know, on every single level, but the industry needs your expertise and your interest and your passion, so. Just, yeah, like this isn't the end of anything. It really is just one door closes, another one opens. So, yeah, excited to see where everyone else kind of takes things. Thanks for being here.
[01:30:48.211] Andre Elijah: Thanks for that, Joe. Dave?
[01:30:49.732] Dave Shumway: Yeah, it's definitely easier to say this now than it was, you know, a year ago. But yeah, just to co-sign everything that has been said, some of the greatest working experiences and personal relationships, for sure. And to know that those people, whether they're still at Magic Leap working on ML2, or whether they've gone into every possible big tech AR or AR adjacent company you can think of. It fills me with confidence in the medium going forwards, I think. We all hit that wall pretty hard, that it's further off than we thought and that we wanted it to be. But we learned a heck of a lot in a really short, respectively, period of time. And those people are now continuing the fight at Magic Leap and they're at Apple and Facebook and everywhere else in between leading that charge. So now we get to sit back for a bit and see what happens next, which is not such a bad thing.
[01:31:44.187] Andre Elijah: Thanks for that, Dave. And Steve, I'm gonna give it to you, dude, to end us off. Obviously, you announced yesterday that you're out.
[01:31:52.474] Steve Lukas: How much time y'all got?
[01:31:55.496] Andre Elijah: We got some time to shed some tears, bro. You're the face of it for a lot of devs in the community. You're the one that was holding it together.
[01:32:01.962] Jeremy Vanhoozer: We got popcorn, Steve. We got popcorn.
[01:32:03.323] Steve Lukas: Okay. All right. So I got a lot to say here. And, you know, it's... It's one of those things, and I know we've got some journalists in the crowd, and I've seen the headlines, right? And this is speculation, but with a lot of the negative press that had gone out about Magic Leap, whether or not it was justified, I'm sure it made the marketing and PR team a bit gun-shy. And there's a lot of focus on things that Magic Leap maybe could have done better. But the truth is, I said it earlier on this call, and it's the sentiment that I have, what if we didn't have a Magic Leap? All of this that was created at this company, still you cannot see on a competitor headset. So we all got to try out the future. This is why I left my job at Qualcomm Ventures for a year and a half to do my startup, to just really deeply embed myself into this and try it on and try to understand where this thing is going by having it on not just at a demo, not just at a conference, but to have it on all day or for hours at a time and see where is this going. And what I've learned is that the longer you're in it, the more you start to understand where it's going and where it can be used. Now, I still see people who are getting into the industry because they're excited, and that's good. And then they say, all right, how does this replace the phone? I'm like, ah, I see you're still a baby in this industry. Because after a time, you realize it's not. It's not going to replace the phone. Or it's going to replace it as much as your phone has replaced your computer. We still use laptops. We use a phone for some things that are better. We use laptops for things that are better for that. And we'll use headsets for specific things that are much better in headset than on your phone or on your computer. And the more time you spend, the more you can identify and understand what those are. And Magic Leap coming in when it did was a great accelerant for everybody that worked there because we all immersed ourselves in this world for years and then we came back to the present and like okay let's pave the way for what's next based off everything we learned that so many still haven't even tried so that's why i believe magically needed to happen and the whole thing is in this industry there is so much ground to cover i say this is like the eighth continent so much land we have to figure it out and to figure it out You got to get in there and try everything. The beauty of this industry is that you can do anything in the headset, right? There's possibilities are endless. The curse is with the possibilities being endless, then it's hard to get a focus and say, let's go do this thing really well. You know, we talked about with entertainment experiences, studios, like let's do that really well. And obviously Last Light, we know how that went. Awesome. But then It's a lot of money and a lot of resources and a lot of time to try everything and iterate. And my personal example of that is that I kept thinking remote multiplayer matters a lot, even with this headset. And before I joined, I did basic Photon stuff and connected myself with a guy who had just been on Twitter. His name is Andres. You may have heard of him. And I said, hey, man, you want to try something? Let's just try it. And he's like, OK. I'm like, all right. Turn on the MPK, or open the app, and then click join. And then next thing you know, I saw this ghostly figure in my house. I was like, oh my God, this is amazing. And I'm like, turn around, turn around. Now turn your head, tilt down, because of course we didn't, this is all early testing. And then he's like, whoa, you're in my closet. I'm like, oh my God. And I suddenly got more than this audio app can give me, because I could see his attention span. I could see where he's facing. I could see him moving around. And then I started getting these additional visual cues in our conversation that I don't get in an audio app. And I said, this is amazing. And where am I going with this? Then I came to Magicweave. And I saw on a Slack channel one team member said, multiplayer only matters for co-located content. So that's it. It doesn't matter for remote. And I was like, what? And then he said, although I haven't actually tested this thesis, I'm like, oh. Well, I have. Because they're all focused on co-located, which is also very important. But remote, especially for a device at this cost, was really important, and once we started doing meetings and get-togethers in Headset remotely from our own homes, then it was like, I can have a party in my house, which I did multiple times, and then when the party's over, I don't even have to go home. I'm already here, and I don't have to kick anybody out. It was something that very few of us have ever experienced. And now, where does that take us with meetings? And my team, God, trying not to get emotional now, but my DevRel team that I just left yesterday, they scheduled a last meeting. And that last meeting, they said, hey, let's do it in Spaceship. I'm like, oh, OK. Sure. I mean, that's because they know that I care about that product. So let's get in. And next thing you know, they had spent the previous night figuring out the app. drew a farewell piece together with balloons and signs and figures and a fortnight battle bus and and sayings that i told them as anecdotes and it was all around my house and so were they i said this is kovid we're a remote team we don't get goodbyes but because of this device i get a goodbye and it was so emotional for me and i've recorded i'm gonna i'll be posting some of this on twitter i said this you gotta be in this headset and understand it and magically enable that to happen and the more you're in this space working on this type of content the more you can identify where this works and where this doesn't And it's about finding where this tool can fit. It's not looking at a use case and saying, let's go target that. Well, the tool kind of works, and it's OK. And it's like using a hand wrench instead of a socket wrench on a hex bolt. And you're like, I can kind of make it work, but it's not great. So you know what? Let's go find the right job for this tool. And that's what I encourage anyone working in this space to do, is that where we are right now, identify. Don't just say, hey, we do mail and calendar on our phones all the time. So let's go do that in AR. It's like, well, what? Do we even want that experience? Is that really going to make it better? Or is the phone and the computer still going to be the best device for those experiences? And there are other ones, like co-presence. And like meetings where, instant meetings, not meetings where you have to set up for like 20 minutes to get started because you put that on a CEO and they're never gonna wanna do it again, right? But you gotta find out how to reduce the friction points and make this something that we're actually gonna use and where it makes things better. And one last comment on that. So Ali Heston was, again, the designer on Spatiate. And we were working on the UX. And she was ready to send me sketches and designs drawn in 2D on a computer, and we would go back and forth. She would send them to me, and then I would make edits, and then I would send them back. I'd go over what my concepts and thoughts were, and we were going to do that. But instead, I said, let's jump in Space Shade. So it was me, Allie, and Andres. It's basically a Tilt Brush clone with multiplayer, right? And the three of us are in this room, the Spatiate room, and I start drawing out, I'm like, okay, here's my thoughts on the UI. And then Ali grabs a button, moves it, it's like, actually I think load should be here and you should be here. I'm like, okay, well let's move these buttons over here. And let's put these additional UI elements over here. And like, oh wait, guess what? I'm in the headset, the target device, and I have a limited field of view so I can see what are the right button sizes. If I did this on the computer, and then exported and built, like, oh, too big. Oh, try again. The iteration process would take so long. But by doing it all in headset, instead of weeks of back and forth, we finished the UI in 45 minutes as a team. And that UI is the same UI that is in the app today. So there are ways that this headset will massively reduce costs and improve efficiency. And maybe not this headset, but this type of headset, which, again, is the only one of its kind right now. So these are the things that I don't think people talk about enough. They talk about things they don't like. And this is the future of computing, not necessarily all computing, but a specific set of computing that's going to make our lives so much better in the future. When is that future? That's variable. That's up to us. Whoever wants to get in and work on it, try to accelerate that, to bring that future here closer, that's what I care about. That's what I'm fighting for. So with that, I think I've taken up enough air. And I'm going to say, thanks for all for coming.
[01:41:12.049] Andre Elijah: Thanks, people. Talk to you soon.
[01:41:14.825] Kent Bye: So that was Andre Elijah, who was the moderator of this conversation. And Alessio N'Goma was also a moderator, but I don't think he actually spoke too much throughout the course of this conversation. There's Anastasio Davana, who is audio director at Magic Leap. Steve Lucas, head of developer relations and engineering at Magic Leap. Paul Reynolds, who is a senior director, SDK and applications. Dave Shumway, who is a lead audio designer and composer. Jeremy Vanhooser, who is a VP of creative content. Tim Stutz, who is a lead interaction designer, as well as Joe Gabriel, who is a developer relations community and program manager. I have a number of different takeaways about this conversation. First of all, I guess I was struck by how exalted of an experience a lot of the former employees felt. They really loved working there. They loved the other people that were working there. challenging environment and hard, but also a lot of creative challenges and really just like this boundless opportunity to really push the future of the medium forward. And this dialectic between trying to create the technology, but also then creating the software and how those were related to each other. Usually the hardware sets the limitations for what you could do with the software. But this was a situation where they were saying, hey, whatever you want to build, we can build the technology around that. And I think that actually doesn't fit with how most of the creative process works. And I think that's an interesting point that Paul rounds was saying is that usually you are starting with some sort of constraints and limitations, and you do the best you can to kind of push the limitations from the software perspective, but usually there's a pretty hard bound. So I guess When you think about the future of augmented reality, there's still a lot of how this is all going to play out that we don't quite know what the future is going to be. We haven't had any major augmented reality head-mounted display that's launched at a consumer scale yet. We have the HoloLens, and we've had a number of different failed iterations there, everything from the Meta and the Daiquiri. Blipper as well as ODG, you know a number of these different companies that were trying to either at the software or the hardware level try to make a push into the enterprise space but the Microsoft HoloLens 2 is really the biggest competitor to say Magic Leap for what they've been doing and the buzz here was that there's a Magic Leap 2 that's coming at some point and But I guess one of the things that I took away from listening to this was that, you know, I was hoping to maybe hear some sense of like, OK, what went wrong? What was like the thing that someone is going to be able to really pinpoint? It seemed like that they're just spread really thin in terms of just doing a whole lots of different things in terms of operating systems and making a browser and the magic reverse air cloud and the hardware and the optics and the content. You know, there's just a lot of ways in which they're trying to innovate on all these different dimensions all at the same time, because it's a new computing platform, then they have to, you know, build all this stuff in order to create this as an ecosystem. So that was certainly a challenge to be all these different things, you know, working there, it'd be really exciting to see the variety of these different projects. But there was a little bit of a moving target in terms of the hardware's evolving, and then Anybody who's making any XR experiences, even if you update the Unity from builds to builds, it has lots of different things that aren't always fully backwards compatible. Things break and mysterious bugs within the context of Unity. So even with all the more stabilized consumer VR hardware that we have now, there's still these issues that XR developers have to deal with. So that on top of having all the different iterations for the hardware and having to update the whole pipeline. So I could see how there's certainly a lot of challenges that are involved there. And that Rony, he was certainly someone who was motivated by a lot of the creativity and wanted to have a lot of creative input. And that, you know, there's a lot of the story that Rony would have to create in order to basically sell this technology that hadn't really been created yet, especially in the early days of when Magic Leap was first starting. And so he's having to gin up this support for an idea. And then I think that's part of that marketing hype was to sell a vision of what this could be before it even exists. I think trying to close that gap was the existential challenge of Magic Leap, in terms of maybe overpromising or building up too much of that story. Then, when it came to actually delivering, it was not really matching what they were promising. But at the same time, even if it was the perfect version of what they had promised, just the market fit of whether something like this needed to start an enterprise, but there were a number of other different AR hardware companies that were also starting an enterprise and weren't necessarily making much headway. We have both Facebook and Apple that are potentially working on different aspects of the glasses. We have the Snap that has done the spectacles. What is going to be the key thing in the application that is going to really make this take off? I think that's up in the air. We've had the Google Glass and Google, and they're also working on different aspects of the enterprise aspect. You have a lot of companies that are thinking about this, but what is the thing that's actually going to take root? I don't know if anybody quite knows the answer, but I think just listening to some of these different conversations, one of the things that was striking to me was hearing the create and how they were putting these different objects together, and those objects were interacting with each other. That reminds me of the future of spatial computing, when you have these applications that are somehow interacting with each other within the context of a spatial environment, which is not something that we experience too much of in our Windows context, because they're kind of like in these self-contained containers, and they're not really interacting much with each other. But in spatial computing, there has the potential to be able to have a lot more interactivity. And I think if you look at the Create app and what they were doing with the different types of interactions, like the UFO interacting with the dinosaurs or something like that. And you would have those two spatial objects in the same spatial context. And then there's these different interactive components that are happening between those objects. Just that, I think, is actually a very compelling idea to think about where this may go in the future. And I think the Pluto VR has been experimenting with this multi-app ecosystems and what the future of spatial computing is going to bring. I think it was Jeremy Van Hooser who was saying that there was a lot of creative input that was coming from Roni, but perhaps Magic Leap as a company would have been served if there was a producer that was really putting some constraints and limitations and trying to bottle up a lot of that boundless optimism and boundless creativity into something that was constrained by the budgets and the time and the overall goal for where they wanted to take this. In that realm of that dreamy, visionary state, it ended up being a lot of iterations and always trying to innovate and do more and more and more. And I think in that, you see perhaps a pattern of the lack of shipping things or just this frustration with some of the creative team of doing all that they could to make something happen and then being requested to do even more. That was sort of an impression that I got and how that sort of ripples out to the cultural DNA of the entire company. In order to really do a full retrospective, I think we'd have to have not only a discussion with some of the key strategic leaders within the company, but also have more access to the data for what was happening. There's a lot of things here. They were trying to hit the consumer entertainment market. Maybe that's just not really fully baked. We need to really develop a larger ecosystem when it comes to virtual reality technologies and people going from what they have experienced from spatial computing within VR. need to have some sort of human-computer interaction or some metaphors or some experiences that people would want to do in this augmented reality environment. The killer app for AR, I have still just a hard time imagining what is going to be and what context is that going to be the thing that people want to put the heads on and deal with all the variety of different privacy and other issues when it comes to putting cameras on your faces and walking around in public. A lot of stuff that still has to be figured out. So we'll see if it remains within the enterprise marketplace or if the social apps, I think it was what Steve Lucas was talking about a lot. I actually followed up with Steve after this conversation just because there was things that I was going back and forth with him on Twitter, and then he just wanted to share more of his insights before he goes on to his next gig, just kind of unpacking different insights that he had from his journey. And I think in that we get to talk a little bit more about my impressions of Magic Leap and other stuff that the conversation didn't really go into here, which You know, it was a conversation moderated by Andre Elijah and really driven by his curiosities of what he wanted to hear about, which happened to be a lot of the last light and a lot of the creative aspects there. And I think there's actually some really interesting aspects in terms of that was a big project that was about to go to South by Southwest and then COVID hit. And then it was kind of put into disarray until the SIGGRAPH came and then they finally got everything together to be able to actually release it. But there was a long time there that it wasn't. going to be released just because it was against the marketing message for where the pivot for Magic Leap was headed, which was much more in this enterprise space and not the creative narrative pieces and experiments that they were doing there with The Last Light, which I still haven't had a chance to see because I don't actually have a Magic Leap headset, and I just haven't had a chance to have access to it or to be able to see it yet. So just a couple of final points is that there was a discussion around the grant program. I think at the time there's a lot of different independent creators that applied for it and the people that ended up getting a lot of them were pretty well established entities like within or Felix and Paul, you know, they don't have any lack of funding or resources. And so when they have this kind of indie grant program and they end up going to like these big creators that Didn't necessarily even make as much sense as to try to like really foster the ecosystem There was Steve Lucas saying that you know there's other decision makers that came in and made those executive decisions And you know he didn't have as much say in terms of how things went out, but I think When that happened, there were a lot of the indie creators that turned on Magic Leap in terms of trying to get at least some support to be able to build some of this content, and those that were making it just didn't have much financial resources. The Information published an article in December of 2019 saying that only 6,000 headsets were sold for Magic Leap. At that point, they had been released for well over a year. And, you know, I think they were aiming for like 100,000, and they only sold like 6000. So, you know, as I think about this, I see something like the approach that valve took, where they were just giving out a lot of the developer kits to a lot of different game developers. We have Oculus that had the Kickstarter, which was like $300 to be able to buy a VR headset. And the dev kits, that really just seeded the whole ecosystem. They could have potentially just invested a lot of the money that they would have spent on all the other things and just invested into getting headsets into the hands of the developers, but there's an issue of whether or not they were going to actually build anything. So, they chose to go the consumer route and to sell these headsets that were like $2,300 to $3,000 after you bought all the different accessories that you might need. And, you know, I think from my perspective, there was certainly a lot of the secrecy and from the different employees that I was talking to at LeapCon, there was this culture of secrecy that I think a lot of the company was trying to overcome. And I think I was dealing with certain aspects of that during LeapCon. And afterwards, Joe Gabriel said that, you know, there's three different phases that at least he saw from the developer ecosystem growth, where the first phase was the pre launch and like super hyper secretive and ultra limited hardware. And then the second phase was a little bit less secretive and onsite workshops, and, you know, they would bring the hardware and then The third phase was much less secretive and the discord and online workshops, dev jams, and giving out a lot more hardware. So I think there are certainly different phases of the ecosystem. And there was certainly a lot of that secrecy that, at least from my perspective, when I was at leap con, there was still kind of overcoming different aspects of that culture of secrecy. Well, there's certainly a lot more aspects of Magic Leap's story that I think is yet to be told. The thing that I appreciated from this conversation was that you have a certain media narrative that is casting what the story of Magic Leap is. But yet, just to hear from the actual developers and the people on the ground in the trenches actually making the technology happen, that really the people that are in charge of a lot of those marketing stories is a small percentage of the company. They don't have much input or transparency in terms of how those messagings were made and what type of stories were being put out. It just seems like there were a number of different potential missteps, but to really isolate them and do the Monday morning quarterback and to say they could have, would have, should have done this thing or that thing or another thing, certainly there's a lot of money that was spent over $2.6 billion and potentially even more invested into the ecosystem and then how wisely that money was spent, I think that's certainly up for debate. Certainly, if they were trying to get more leverage, then they could have made other decisions. But, you know, that's kind of like the type of speculation that doesn't really make any difference at this point. I mean, it is what it is. And it might have just been an issue of timing, even if they did everything perfectly, that the overall market just wasn't ready to be able to receive this expensive augmented reality head-mounted display device. So, again, this was a conversation that was happening in Clubhouse. Thanks for Andre Elijah for hosting the discussion. It's part of his show that he does every other week or so called the No BS ARVR Reality. So this was the Magic Leap edition of that show. And yeah, it just also gives you just a flavor of the Clubhouse vibe of where the conversation goes and as people come in and then jumping from topic to topic and giving a little bit of a slice from the developer's perspective and the people that were on the ground, just, you know, from them, they really had a great experience there. And, you know, they learned a lot. And a lot of people that were at Magic Leap are now across all the entire industry from Apple to Facebook, to Snap, to Niantic, Google, Tilt 5, to all of these companies that are out there still pushing forward the augmented reality medium. pretty much spread out across the entire industry. All the major places where the headsets are still going to be developed, they're out there making that happen. And there's certainly a lot of prototype stuff that was happening there at Magic Leap. And I still have a lot of interviews from the Magic Leap LeapCon that I haven't quite dug into. And it's probably worth digging into at some point, especially as we start to get closer to these other augmented reality headsets that are going to be starting to come out here at some point. So, that's all that I have for today, and I just wanted to thank you for listening to this special edition of the Voices of VR podcast, and if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listed supporter podcast, and I do rely upon donations from people like yourself in order to continue to bring you this coverage. So, you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
