Andre Elijah is an independent VR developer who recently launched Viewport Interactive working on branded experiences for car companies and beyond. We talk about the Elixir Quest Handracking demo that he tried at Oculus Connect 6, and how it shifted his sense of embodiment by having different representations of his hands. Elijah and I have also had a number of debates on Twitter over the years, and so we had a chance to come to a place of agreement between his desires for bootstrapping the industry and not unnecessarily slowing down adoption of VR while I’ve been advocating for deeper discussions around the ethical frameworks to be able to navigate the many moral dilemmas around privacy that are introduced with biometric data and the ability to capture a digital representation of the world around us. We explore the tensions between his economics of his pragmatism and the ethics of my idealism, and we were able to bury the hatchet and come to a place of understanding from our previous sniping at each other on social media.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. So, continuing on in my series of covering some of the highlights from Oculus Connect 6, today's conversation is with André Elijah, who's the co-founder of Viewpoint Interactive. Andre's been a VR developer for about three years now. And so he's somebody who I've actually had these different interactions online on Twitter. We've had like these sparring battles, usually around Kim defending Facebook around data collection or privacy. And it always kind of ends in this contentious agree to disagree kind of setup, but it doesn't feel like we've fully understood each other. So I really wanted to just have a conversation with Andre to understand him and where he's coming from and what he's doing in the realm of being an independent developer within the VR industry. And we kind of hash it out and we just talk about our different perspectives. And, you know, he's really coming from that perspective of a indie VR dev who wants to see the industry grow and survive. And just seeing how all the different companies that are out there have their issues, especially when it comes to XR and. In a lot of ways, Oculus is the leader in terms of what they're able to do and both the quality of the product they have, but also the ecosystem that they're cultivating and their relationship to the independent developers. So from his perspective, when he sees other people criticizing Facebook or Oculus in some ways, then he feels like he's a little defensive and also wants to see the overall industry grow and not to arbitrarily slow it down by being bogged down into thinking too much about issues around ethics and privacy. So we kind of talk about the variety of different perspectives as well as some of his highlights from Aqua's connect six. Uh, he actually had a chance to try out the hand tracking demo where you transform your body. The only experience that I got to see after this in terms of the hand tracking was the insurance demo, which essentially you're kind of walking around and pushing things. You're not actually grabbing or seeing any interesting embodiment aspects that he was able to experience. So he talks about his experience of that as well. So. We're covering all that and more on today's episode of the Voices of VR podcast. So this interview with Andre happened on Thursday, September 26th, 2019, at the Oculus Connect 6 conference in San Jose, California. So with that, let's go ahead and dive right in.
[00:02:23.968] Andre Elijah: My name is Andre Elijah. I am the co-founder of Viewport Interactive. I've been around for a few years. And Viewport's a brand new company that we announced this week at OC6 with the help of Unreal.
[00:02:35.032] Kent Bye: Great. So what are you doing with your company?
[00:02:37.529] Andre Elijah: We're doing a lot of work with brand integration, marketing projects, particularly automotive. So we have some projects coming down the line for the Toronto and Detroit auto shows, helping companies there kind of reach customers using AR and VR. And then we're also working on a couple of original projects right now, one of which is an interactive XR documentary called The Innocence in the Fire about climate change and global warming. So that'll be coming in the next year.
[00:03:02.931] Kent Bye: So maybe you could give me a bit more context as to your background and your journey into VR.
[00:03:08.032] Andre Elijah: Yeah, sure. So I've been basically a lifelong nerd. Way back in the day, I worked in the corporate world building high-performance computing systems, disaster recovery systems, and high-frequency trading systems for traders. And I went from that into working in the film industry, specialized in the RED camera systems. I had the first couple that launched in Canada. and worked for a bunch of people, Jerry Bruckheimer, Beyonce, you name it I worked with them, did a lot of work out of China as well in documentary film and then I decided I needed a change and got into video games and then from video games I ended up in architectural visualization because I like making pretty images and from there that turned one thing into another and now I've been in VR for three, four years grinding it out every day.
[00:03:54.127] Kent Bye: So what type of projects have been keeping busy with because it sounds like you just announced a new company But yet you've sounds like you've been in VR for a while So what have you been doing for the last three years in VR?
[00:04:03.374] Andre Elijah: So a lot of companies have kind of outsourced their R&D when it comes to immersive technologies to us Not a lot of companies have the wherewithal to figure out how to navigate the waters They have a lot of content either, you know, traditional 2d content or even some 3d work They don't know how to get it into VR, what to do with it. So they typically call us. So whenever there's a new piece of kit, we generally get it in-house and we start prototyping. So we do a lot of the not-so-sexy stuff that needs to get done before a company goes out and announces that they're in bed with VR and all the big companies.
[00:04:36.285] Kent Bye: And you said that you're working with Detroit and Toronto Auto Show. So maybe you could give a bit more context as to what that is and what you're doing with that.
[00:04:44.527] Andre Elijah: Yeah, so we've got a couple activations that are coming up, both AR and VR, showing off some of the new car models that are coming out. So NDA to hell, and I can't really talk about everything. But you'll have interactive car driving simulators, virtual drive tests. customizing cars on the fly for maybe some new financing options, that sort of thing. So that just started in the last couple of weeks, and it's really exciting. But yes, NDAs are a thing.
[00:05:09.599] Kent Bye: So with marketing or with these activations, what are you telling the companies that this is going to do? And how do you measure success to see that it's actually doing that?
[00:05:20.628] Andre Elijah: So I think the first thing we tell companies is that they're doing this now to plan for the future. They're laying the foundation and the groundwork for all their work moving forward in that they need internally at a company, they need the processes and they need to have the knowledge to figure out even how to get started on all this stuff. So we start now for a future of two to three years from now when everyone will have a headset or at least they'll have foundational knowledge of what VR is because a lot of people aren't aware. And so really what we want to do is we want to just create experiences that wow their consumers, wow customers, and have people talking. So as long as people are sharing pictures, they're sharing a hashtag, and they're engaging in the brand in a cool way, and they walk away saying, oh my god, that was really cool, then we've done our job, and then the brand is happy.
[00:06:03.579] Kent Bye: I see. So it's got more of an anecdotal, immersive engagement. People stand in line. They want to have an experience. And they have the experience. And then just to have people have that, it's enough to have an impression of what they're working on.
[00:06:15.603] Andre Elijah: Yeah, for now. Immersive tech isn't everywhere yet. And people aren't really aware of it. So we're pretty cognizant of the fact that we need to do the groundwork and pave the road to be able to get there in a few years. But right now, yeah, it's about having people talk about it, tell their friends that something was really, really cool. and then maybe be open to engaging even more later on when they're comfortable.
[00:06:37.834] Kent Bye: Yeah, at Augmented World Expo, I saw, for the first time, in real, which I want to clarify is not unreal, is in real with an in real, which is totally confusing when everyone says in real. I don't know if they're talking about unreal or in real. But maybe you could talk a bit about what you were doing that in real over the last couple of days.
[00:06:56.452] Andre Elijah: Yeah, so we're actually going to be partnering up with Unreal for a bunch of our projects. And it's funny to note that we actually use Unreal for all of our projects. So we're using Unreal with Unreal. And yes, it gets very confusing very quickly. But they honestly have an amazing form factor. The glasses are great. They're really comfortable. And ultimately, you don't look like a dork when you're wearing them, which is great for social acceptance. So we want to put that to the test, especially with the activations and have people feel way more comfortable putting these things on their faces to engage with the digital content and the brand. So it's a really great partnership. We're really looking forward to it.
[00:07:28.564] Kent Bye: Yeah, I heard that they're based out of China and that there could be some intellectual property lawsuits. And is Unreal going to be available either in the United States or Canada to be sold? Or do you feel like there's potential litigation that may make it difficult for them to distribute what they're doing?
[00:07:46.290] Andre Elijah: I've been told not to worry about it, and I'm going to trust them on that. All I know is that, you know, they're making it accessible to developers, which is really great. A lot of hardware companies kind of have a reluctance to engage with independent developers such as myself. And the fact that they're willing to support us, we've got a direct line of communication for support. The fact we have access to the hardware is really, really great. And they are only rolling out to developers right now. So we're in September. I think towards the end of the month, it starts shipping to developers. And what their mass consumer plans are going to be, we'll see. But for right now, I've got a really great line of communication with them. They're really open to taking feedback and engaging with us. And that's what we need.
[00:08:24.930] Kent Bye: So what's been some of your takeaways or impressions being here at Oculus Connect 6?
[00:08:29.992] Andre Elijah: There's a lot of excitement. There's a lot of excitement for finally growing the user base. And I think that the announcement of, the link for Oculus Quest is going to change everything. Because as opposed to consumers having their wallets and their attention divided between multiple headsets, we now, it's pretty clear that we're going to have one going forward that does double duty. And I think that'll be really great for boosting numbers. I think the hand tracking feature is going to be amazing. Had a play with that yesterday, and even in its really early state, it was really convincing. And I know that, you know, when we're dealing with consumers that aren't necessarily hardcore gamers or not tech literate, a controller is really scary to them. All the buttons, the joystick, and everything you do, it's one-to-one, so everything moves and it freaks them out. So being able to just engage with virtual environments naturally with people's hands is going to change everything. And I think that'll be the next step to acceptance and mass market acceptance. First was cutting the cord. We got that with Quest. Next step is the hand tracking. And so from there, it's, you know, maybe all-day battery life and some further clarity. So we're getting there really quickly. And Quest launched four months ago, and we're already getting all these features. So it's really exciting.
[00:09:43.198] Kent Bye: Yeah, I know. It's really quite remarkable, because this is my sixth Oculus Connect, and so I've been each year.
[00:09:48.661] Andre Elijah: You're one of those. You've missed every one. Yeah. Yeah, that's awesome.
[00:09:52.377] Kent Bye: So just to see the progression of what's shown each year and just to go back from 2013 when the DK1 came out and to see where we're at now is pretty remarkable. And to think about where things are going to be in another five or six years, it's like just to see where that's going. But for me, I haven't had a chance to do the demo yet, but I can imagine that having done the Leap Motion and being able to interact with virtual objects, I know that You know, there's a lot of embodied cognition concepts and principles so that actually being able to manipulate things with your hands, I think, is going to actually open up and allow us to do much more sophisticated thinking and interactions and just kind of think out loud and being able to have the full dexterity of our fingers. I don't know. I feel like there's going to be a lot of unexpected new possibilities that are going to be made available once we're able to have that full flexibility with our hands and just to really deepen the level of presence that we have.
[00:10:43.177] Andre Elijah: Yeah, for sure. I mean, even in the demo, so I did, there's two demos here at OC, and the one that I did was this, you know, little witch cauldron game. And because of a variety of spells, your hands kept on changing between different types. So there's a robot claw, there was a weird monster hand that had like Wolverine's claws, and there was another one that was like octopus tentacles. and just even seeing the physics on the hand when I'm moving my fingers and how it just manipulated everything, it was kind of amazing. And for a second, I actually thought that I was a monster with octopus hands. And so, you know, I've had all the headsets, I've had all the controllers, and everyone always talks about, you know, embodying a character or embodying a person and, like, a sense of presence. And I'm always cognizant of the fact that, hey, I'm still a VR dev, I do this stuff all day every day, you know, so I'm aware that I'm in it. But seeing just kind of how it reacted to my hand and not a controller, was really amazing and I can't wait to play that on an ongoing basis and then create stuff that takes advantage of that as well.
[00:11:36.475] Kent Bye: Yeah, because I know that Facebook just acquired Control Labs, which was starting to do EMG, so looking at electromyography, your muscles, being able to isolate that down into individual motor neurons. And so even though you have your hands, you're able to potentially project out into other distorted representations that you can see as you're moving your hand. You see that proprioception of what you feel, but also what you're seeing. Your brain apparently just believes that it's real. But then to take the next level of actually having EMG to be able to then project out to using your motor neurons to be able to control all sorts of robots or other levels of embodiment, I think things are going to get really weird. We're going to sort of flip in and out of different embodiments. But I feel like it's going to be something that our brains are going to be able to handle. And I think people are going to enjoy this process of kind of embodying these different entities that could then allow them to perhaps express themselves in different ways and sort of tap into different levels of their own cognition.
[00:12:31.943] Andre Elijah: Yeah, I mean, and I think, you know, because all these technologies and all these abilities are coming at the same time, you know, embodiment, you know, okay, great, we had hand tracking, but we were still tethered. There's still be something pulling you out of the experience. But the fact that things are moving so rapidly, we're getting a confluence of all these different capabilities and a really light and shiny package that just you can move around freely in. and it hand tracks and it does all this stuff, it's great. It's the perfect timing for all of it. And I think that all this stuff is going to be the key to having the weird experiences that become the compelling stuff. If you had told me five years ago that the killer VR app would be swinging around lightsabers to musical blocks, I would have been like, you're on drugs. And I think just the fact that we're getting all these capabilities, we don't know what the next Beat Saber-esque success will be because of hand tracking, and it will be enabled because of that. It's kind of like the whole thing of, When Apple invented the iPhone, we didn't know that the coolest thing about it would be ordering car door to door, right? So what are these capabilities going to enable us to provide to people in two, three years, or even in six months when it goes live? So we'll see.
[00:13:35.962] Kent Bye: So what do you personally want to experience in VR?
[00:13:39.437] Andre Elijah: I want to be entertained. I want to be taken out of the real world and everything that's going on in it, and I just want to forget about it for a couple hours at a time, or even a day at a time if we can. I've been a VR guy since the beginning, more so than AR, because I like being able to escape the world, I like the escapism, and I just want to go into a crazy scenario that I couldn't otherwise be part of. We're getting there. We're getting there really quickly. And I think that, you know, if you look around OC, everyone's working on something really strange and wonderful. And it's honestly, like, the most inspiring show that I go to every year. And it's the one I can't miss.
[00:14:16.139] Kent Bye: Is there any experiences that you've done that's, like, one of your favorite experiences? A point, too, that says, like, this is something that you was either a turning point for yourself or ones that you come back to as to, like, having a new insight or ones that you just really enjoy?
[00:14:30.862] Andre Elijah: I mean, there is one thing that I built. So I don't like swimming. I never could swim. And a few years ago, I started getting popular because I ended up building Drake's house in VR and Unreal. And I built out the whole swimming pool and everything. And I hadn't been in a pool in 10 years. And so I was wearing my Vive. And one day, I built this whole thing. I was like, do I take a dip? Do I go walk in further and further and further in? And I did. It was awkward, you know, you had the water moving and everything, and I started submerging myself. So it was really awkward, but I realized, wait, I'm not, like, dead. That was amazing. And so since then, yes, I've now gone into pools. But I mean, there's things like that. You know, my buddy Blair that made Technolust, I've been messing around with the lo-fi stuff for a little while now. I'm a Blade Runner fan. I love cyberpunk. So being in that, like, weird, dark, dystopian world is absolutely amazing. And there's, you know, everyone's just working on some really weird stuff, and it's a lot of fun to check it out, so.
[00:15:28.211] Kent Bye: How did it come about that you recreated Drake's house in VR?
[00:15:32.173] Andre Elijah: The BBC had leaked the floor plans. And I was just starting my last company, Opiates, at the time. And I kind of wanted a calling card. And so the floor plans leaked. And I realized, hey, I'm never going to afford a house that's $70 million. So why don't I just build it and walk through it in VR? And so I built it in Unreal. It was the first thing I'd ever modeled and figured it all out. And here I am three years later still doing this stuff.
[00:15:56.327] Kent Bye: Did you ever hear from Drake about it?
[00:15:57.588] Andre Elijah: Yeah, there were discussions about it. There were discussions.
[00:16:00.690] Kent Bye: Well, because I feel like there's a certain element of there is, I don't know, like ethical or privacy implications there around, like, I don't know if I would want people to be modeling my home in VR. I mean, there's a lot of security implications there, so. Is that experience still available, or did it sort of get mime-holed?
[00:16:17.681] Andre Elijah: So I made a video of the experience. I never actually released it. It's only friends, family, that sort of thing, and a couple of shows I showed it. But it was mostly just a video fly-through that was done. But I think that the plans that were released were really early, so I know some things have changed about the house. I don't think it was in response to me, but security's tight at that place anyway. It's a fortress.
[00:16:36.897] Kent Bye: OK. Well, I'm curious, what are some of the either biggest open questions that you feel are kind of driving your work, or open problems you're trying to solve?
[00:16:48.305] Andre Elijah: I think the problem we're trying to solve is adoption and doing it in a way that doesn't poison the well. I've got a really big pet peeve about branded projects and just kind of how basic they are and how they can be done so much better. And in my mind, I'm pointed to Jumanji VR. Just Google Jumanji VR on Reddit and you can see what everyone was saying about that. I really believe that there's a space for the big companies, the platform holders to fund and to help the indies. I think that the indies are in a really With the work that they're doing and the gameplay mechanics that they're all exploring, I think they're in prime position to kind of go mainstream. If, you know, a brand came to them and said, hey, let's reskin your game with, you know, a major property. First one comes to mind are the Windlands guys. And, you know, that's a Spider-Man game, right? Like, point blank. So I would really like to see and kind of do what I can to kind of marry the indie scene with the big brands and create experiences that are awe-inspiring and ultimately push a kid to say, mommy, mommy, mommy, I want a quest. And I think we have a long way to go. We're all nerds at this conference. And we're getting off on hand tracking and wireless tethering and all sorts of stuff like that. But we need mainstream to come in. And to do that, it needs to be a partnership between the big companies and the indie people that are devoting their lives to building this stuff. And so I want to see that in the future, in the next year or so, be taken way more seriously.
[00:18:15.219] Kent Bye: Well, I think one of my impressions with what's happened with the VR industry is that early on, first couple of years, it was a new technology. There was a lot of excitement from different digital design agencies to do an activation with VR. But then they'd create these experiences and then be either on the Google Cardboard or Gear VR. But yet, the access to the technology was such that it wasn't an economy of scale where they can actually justify the amount of cost it took to develop it. And then it kind of fizzled out in terms of the excitement around doing branding and advertising. But there seems to be some of those experiences that have persisted. And so it seemed to be a lot of movie properties to take existing IP and then try to create some sort of experience. But what do you think is the secret for what makes a good branded experience? Or what is it the ones that are still around, why are they still around?
[00:19:05.897] Andre Elijah: I think passion from the developers. A lot of these projects that, you know, we're talking about are kind of phoned in. I have no problem saying that. And it hurts as a fan of these properties when I go, you know, day one and it's on Steam or it's on Viport or whatever and I go get it and then I load it up. It's like, oh, there's a huge disconnect between me imagining myself in that world and as that superhero or whatever and then what I'm being given. And I think, you know, at this point, Unreal, Unity, they're gorgeous. The capabilities are all there. There's no excuse for ugly graphics on a branded project. And if a company's spending millions of dollars, they should get the most out of that. So I think people are becoming more and more cognizant of that. I think there's a little bit more trepidation when a branded IP comes out and it's like, oh, cool, maybe I won't download that right away because it might be a letdown. And I think that, you know, it's going to push agencies and all that to bring their A game. And now with VR becoming more prevalent and Quest becoming a thing, and it's part of the discussions again, you know, the early days of It's not cool anymore just to have something running in VR. You've got to have something to do in VR and a way to keep people engaged and coming back and putting on the headset. And I think now that there's a hardcore push, we're starting to get a sense of the numbers. $100 million spent on the Oculus platform alone, not including Steam, not including Viveport. There's money there. So now everyone's going to have to start bringing the A game. So it'll get good.
[00:20:26.928] Kent Bye: Well, I know that we've had a number of interactions over the years on Twitter, and some of them have been a little bit more contrarian or antagonistic. And my impression of that is just sort of like, I don't know if it's around privacy, or if I'm too critical around Facebook, or I don't know. How would you summarize that?
[00:20:42.616] Andre Elijah: Listen, we all know at the end of the day that Oculus is a Facebook company. Facebook is a data company. They're tracking everything. We know that. You want to, in my opinion, You want to protect the rights and the privacy of the masses, the public, and that's cool. I see an industry that's had a few false starts and needs a giant backer that is willing to go all in to get it to the next level. A lot of people talk about ethics in XR, VR, AR. They talk about privacy in it. I just want to get to a point where we actually have a viable business and then we can make the rules and the foundations that'll guide everything. I see a lot of devs that are putting their lives on the line to make this thing happen. And if there's another false start, what happens to them? That's my concern. I talk to them every day. I consider them to be among my best friends. So I want there to be an industry. I want the stuff to go wide. And yeah, there's going to be issues along the way. There's going to be bumps in the road. But I want to get to, I don't want Oculus to be on that stage saying there's $100 million in content. I want them to be on that stage saying there's 100 million users. And I think that we have to gloss over some of the finer details to get to that. But I do believe that we have to get there. And Facebook thus far has been the best shot at getting there. You know, Google popped into VR and then they left. And they told people to engage with their platform and they were left hanging. Right? We see HTC doing their thing. You know, some of it's really great, some of it's a little bit lacking. You know, Facebook seems to be the one that, you know, is putting out polished products, a full platform and trying to get people paid. And I really appreciate that. And I think, you know, before we start dragging them for, oh, my God, you're a Facebook company. Oh, my God, what are you doing? You're tracking the cameras. Are you tracking this, that or the other? maybe we should just say thanks for giving all the devs free headsets. Thank you for engaging us. Thank you for, I don't think enough is said about Oculus Start and Oculus Launchpad and bringing out all these people, flying them, paying for their hotel, giving them hardware, giving them face-to-face time with legends like Robin Hunicke and Jason Rubin. But it's always like, hey, let's drag Facebook because they're the data company. So that's my take. I see you smiling and laughing. But that's just my take. I want there to be a market. I want it to be a viable thing. And honestly, if there's another false start, when are we getting this shot again? That's my take on it.
[00:23:00.173] Kent Bye: Well, I have a lot of thoughts. But I wanted to first ask, because it sounds like you were just part of the launchpad, right?
[00:23:04.862] Andre Elijah: Yeah, Launchpad was great. I came in last Saturday, I'm flying out tonight, so I think that's five nights they put me up, and it was great. Having that level of access to Facebook and Oculus staffers is amazing. Being able to go out with them at night and bond until two in the morning is absolutely amazing. And I'm from Canada. I'm from Toronto, right? So having that level of interaction with a bunch of the staff is insane, right? We don't get that. I'm not in the Bay Area. I come out once a year, twice a year, that sort of thing. So to be able to do that is great. I'm so thankful to Facebook. So maybe that's why I'm a little bit defensive on their behalf. I don't know.
[00:23:39.863] Kent Bye: Well, so to get back to this dialectic, because I think you're right in terms of what Facebook is doing, that we live in a market economy, and that Facebook is innovating, and they're taking a lot of risks, and they're investing money to create this entirely new platform. But there's also been HTC and Valve that has really created this dialectic that, without that market pressure and competition from Valve, we may be still sitting down VR with controllers.
[00:24:05.267] Andre Elijah: Yeah, no, completely agree. But in terms of the level, you know, I'm thinking from the level of support that the devs are receiving, right? Valve is great, they have an amazing platform, it's done very well for a lot of people. But in terms of, you know, being flown out, put up, access to people, access to hardware repeatedly. Like, I remember before Start was even a thing, you know, Oculus had a thing on their page, sign up for dev kits. How big is your team? Okay, we will now ship you a bunch of dev kits. I don't know how many headsets I've received over the years from Oculus. Support tickets, the whole nine, they're really great. So that's where I'm coming from on all this.
[00:24:40.232] Kent Bye: Yeah, so as an independent journalist who does podcasts, I am on a bleeding edge of interfacing with a whole PR relationship that is what I see as kind of old models that I sort of challenge. I feel like being the first podcaster as a primary mode, then I sort of run into a lot of things. And so over the years, I've been coming for six years now, I've dealt with different levels of disconnect. Each year there's something about the experience that makes me angry. And then I remember August Connect 3, I got really angry about a number of things and I sort of went off. But I think there's another element, which is that as an independent journalist, I'm trying to find ways to support and find my own business, right? So in my own lack of business skills sometimes get projected onto the companies that are doing the business in a way that's really what I feel is that there's a lot of ethics that the way that things have gone down with even the story for how Facebook started with seizing data and starting to do like hot or not type of stuff. I mean, it's kind of gross, the history of the evolution of Facebook as an entity. And so there is this move fast and break things that has had unintended consequences into the society and the culture. for the ways that they've developed and evolved, but then without thinking about those design considerations, about those ethics, they've helping to accelerate genocide in Myanmar, or having election hacking with the United States. And so there's all these big, big issues that without looking at it holistically and having authentic conversations about it, this closed mindset that's insular, that doesn't have open conversations, creates these situations that have echo chambers that leads to ways in which that you're architecting a dystopic future. And I feel like there's a responsibility, a moral responsibility for these companies that are creating the platform of the future to be engaged in these ethical conversations that need to have all these perspectives heard. And even this conference where we're like, we're going to record everything in your home. We're going to read your thoughts and read your mind. We're going to have access to all this data. And my thought is like, wow, that's an asymmetrical power relationship and power dynamic that unless we start to address that now, We're going to be architecting a future that we don't really want to live in. So I think that there's as an idealist and less of a pragmatist, as a journalist, I'm trying to look at the larger concerns that maybe are not being looked at because they're not coming up yet in the day-to-day sort of pragmatism of running a business. But yet, unless we think about those things at the beginning, the time to think about it is now. The time to say is like there's a lot of ethical concerns that you as developers need to be aware of. Here's what we're doing to architect for privacy of the future. You know, they got like a $5 billion fine from the FTC from all these consent decree violations. And now they have to like meet all these mysterious obligations for doing that. They're trying to architect, but what are the responsibilities for you as an independent developer to maintain that same level of ethics? And to not have that in the keynote, and to have that we're just going to record everything and read your thoughts, it's like, to me, that's concerning, because that's like setting the culture from the top down, that they need to be setting the example to say, this is important, and we're going to talk about it, and we're going to have an honest, authentic conversation about something that's complicated and nuanced. And the fact that they're not doing that, I feel like, it leaves a sense of unsettledness. There's an emotional labor that happens with people who are like, wow, I kind of felt kind of gross about Facebook recording all of my location without really talking about the deeper ethical frameworks around that. And I feel like then it gets distributed out into the audience when they could be thinking about creating stuff. Now they have to be worried about whether or not they can trust Facebook with reading our thoughts or seizing all of our data of our space.
[00:28:14.497] Andre Elijah: I think the way I look at it is Facebook as an entity is made up of thousands of people, right? VR, as important as it will be in the future, it's not important right now. It's small. If they don't sell another Oculus Quest, is VR going to be a thing in a year? It's not, right? They need to continue that growth. They need to continue that curve. In talking to individual employees, they're focused on whatever their job mandate is. One team is just, hey, does the experience that we're looking at for the store meet the framerate requirements? That is their focus. That's it. Another team is looking at, okay, cool, we're doing engineering to figure out better ways of input. The stuff that you're talking about is at such a high level that it's not so much that it trickles down throughout the entire company because the individual people in the company aren't even dealing with those issues. From my perspective, yes, all that stuff is really important. I think, you know, as devs, it's really important, you know, if we're making a game that we want everyone to play, if we want to make the next Fortnite, let's not fuck it up by capturing everyone's information if we don't need it, right? If we're going to do something, let them know and let them opt in. That goes without saying. And I think in a lot of ways, Apple's kind of shown the way in terms of how we should be dealing with privacy moving forward. And yet, for every iPhone that's out there, there's still another 10 Androids where people are willing to give up their privacy for a cheaper device, free apps, whatever, whatever. So while I think all that's important, I think it'll be an ongoing conversation. I don't think that all the issues people are levying at Oculus, Oculus is such a small part and the team is such a small part of the overall company that really those are things that we should target, I don't know, at Mark. As opposed to, you know, if you're talking Oculus, if you're talking, you know, this big monolith Facebook, well, it's made up of a lot of people that don't have a say, they don't even see it in their day-to-day job and, you know, that's not within their purview.
[00:30:07.206] Kent Bye: Well, I would love to sit down with Mark and have this conversation with him. But I haven't been able to get that interview or to get other people that would be at the level to be able to talk about things. I mean, I have talked to people who have architected different levels of their privacy policy, and that's been great. But there's so many complicated issues with VR that It is like we really need this interdisciplinary approach that it does actually transcend any individual. And you have to start to think about things holistically. And that's been part of my challenge is that if I talk to a program manager who's working on something that's so low level, then if I ask about, like, what about the third party doctrine and how is this going to impact, like, the data collection and potentially, you know, erode our rights to privacy in the United States of the Fourth Amendment, that there's a the certain amount of any data that we agree to give over to a third party, and that's tied to our identity, that we have no reasonable expectation of privacy for that data. That's already been happening in the mobile world. It's already sort of gone way past where it needs to be. But as soon as we start to let access to our body of seeing what we're seeing, what we're looking at, what we're feeling, that's a level of intimacy that you can start to reverse engineer and model people so that you can start to not only predict their behavior, but control their behavior in ways that draw a very fuzzy ethical line between prediction and control. If it's only being optimized for profit, then I feel like there needs to be larger discussions about business models that go beyond just this surveillance capitalism that has got Facebook to the point that they're at, which has been able to bootstrap an entire immersive industry. But I wouldn't want to have that same surveillance capitalism that has gotten to the point where they're at now projected into VR forever so that everything that you say or do or think is going to be recorded on a public record and be able to be exploited and be used against you, especially when it gets out into countries that may not have good actors, that if it's illegal to be homosexual, you can determine your sexual preference by looking at your eye-tracking data. So if that is tied to your identity, then all of a sudden you have this opportunity for people to really exploit that information in ways. So I feel like there's a lot of role for kind of red teaming out these conversations and to at least have those conversations now and to think about how do we architect these systems in a way that we can create a world that we want to live into and not a world where we're having aspects of our data get into hands that are being used to control and manipulate us in ways that aren't necessarily ethical.
[00:32:28.260] Andre Elijah: And I'm really glad they're now doing this in person as opposed to taking shots on Twitter because the conversation now is completely different and way cooler. But yeah, no, I agree. I think, yeah, I think it's important to have the discussions. You know, I just don't want, I don't want the discussions to slow things down to the point where an entire industry dies off again. That's always been my point and that's always been my concern. But yeah, for sure. I think first and foremost, companies should be asking us for every bit of information that we give them. I keep on going back to Apple, but I love the fact that whenever I load up my phone and there's an app that asks for GPS data on the new operating system, it says, do you want to just do it while the app is open, or is this a thing that happens in perpetuity? I think that's a really important thing to have and to be able to answer. So yeah, for sure. I think as VRXR devices start reading our thoughts, tracking our muscles and everything that we're doing, I think that we should be giving them consent for every little bit of data that they take. So yes, I agree.
[00:33:26.832] Kent Bye: All right. I'm glad we got to that. That's why I wanted to have the conversation face-to-face, because I think that there's a certain element of trying to find the benefits from this, because you can look at the medical applications and all the amazing things that are happening. And I agree that it's important to let the innovation happen and then start to lock things down later. But sometimes, if you let things go too far, then you can, for example, the United States change the definition of what is a reasonable expectation for the right to privacy, which then is hard to reverse that. So there's things that, unless they're talked about now, then we have this slow frog boiling in hot water that, unless we sort of pay attention to it, then we're going to create a set of cultural norms that then not only has companies being able to have access to that, but have governments that have a totalitarian bent be able to abuse that information as well. Because if there's a being able to have this interplay of having a government's access to the information, then it's one thing for us to think about, oh, well, I don't mind if Facebook has this in their private vault. You know, as soon as it gets into the hands of government or if it leaks out into the dark web, now all of a sudden you have anybody in the world has access to this information. So it's not just like thinking about Facebook as an entity and we just trust them. It's like this is information that's very sensitive. We don't actually want this leaked out into the dark web. Imagine 10 years of your biometric data out on the dark web. What could you do with that? I think it starts to get into some really scary Black Mirror episodes. So I feel like that we have to have these conversations and figure out, both as from what the responsibility is from Facebook, but also as the independent developer community, who has what role to play in this.
[00:35:04.998] Andre Elijah: So I agree. I totally agree that it's important to have conversations. I think we have to recognize now that, you know, especially with the disconnect that you and I have had for ages now, you know, that that conversation isn't something that can take place on Twitter, which is the land of hot takes, right? It takes looking each other in the eye and having a discussion and actually, you know, seeing and emoting, you know, you know, what we're feeling and what we're thinking. So I think it just comes down to, you know, maybe Twitter isn't the best place for, for discourse like this. And I think that's maybe where a lot of the vitriol and a lot of the animosity is playing into, especially with VR in particular, where there's this whole thing of indie devs, are they getting screwed? Are they going to survive the next wave, et cetera, et cetera, et cetera? And then when you have people that are posting the hot takes, not to say that they're wrong, not to say anything like that, but I think that's where all this is bubbling from. And so yeah, I think maybe there should be a town hall or something, or a privacy-centric, maybe it's part of F8. Right? Where it's part more Facebook platform versus, you know, the VR development side of things. I mean, that's where a conversation like this takes place in the future.
[00:36:10.146] Kent Bye: I would love to see them on the keynote stage bring this up as a topic that is unsettled, that it's up for the community to collaborate and figure out together. Because without that, it becomes like this religious fervor of, like, we're all true believers, and this is all just going to be amazing, and we just have to have this techno-utopianism that this is just going to work out without any of the deeper sociological or cultural concerns about what we still need to figure out as a community through discussion and conversation.
[00:36:34.297] Andre Elijah: I'm totally go with that being in the keynote, so long as it doesn't cut down on the time for Carmack or Abrash, because I would cry about that.
[00:36:40.240] Kent Bye: And finally, what do you think the ultimate potential of immersive technologies might be and what they might be able to enable?
[00:36:50.464] Andre Elijah: I think the video that they showed yesterday of Boz and his dad was what we want. The fact that here's a device. small, you put it on your head, it understands your surroundings, you can communicate with people and you can share that with someone else, I think that's the goal. As much as people love being on their screen on their phone or playing video games on their TV, there's still a sense of community that drives everything. No matter how antisocial we are, we still love that kind of outreach. And I think that what we saw in that trailer is kind of the ultimate goal, sharing really awesome experiences. Some of them are analog and just watching a TV, and some will be playing video games. But being able to do that with someone else and share that moment with them, and share any moment, whether it's something completely fantastical that's a game, it's a fantasy land, whatever, or something as simple as watching a basketball game, sharing that with someone that you wouldn't otherwise get to share it with. And I think that's great.
[00:37:44.851] Kent Bye: Is there anything else that's left unsaid that you want to say to the immersive community?
[00:37:48.817] Andre Elijah: I don't want to say anything to the immersive community, but Kent, I really like you. You're an awesome guy. And I'm glad we got to do this.
[00:37:55.481] Kent Bye: Awesome. Great. Well, thank you so much. Cheers. So that was Andre Elijah. He's a independent VR developer and the co-founder of Viewpoint Interactive. So I have a number of different takeaways about this interview is that first of all, well, I'm really glad we had this opportunity to kind of bury the hatchet in some ways. And just for me to learn a little bit more about where Andre is coming from and, you know, his journey into virtual reality, which he self-identifies as this lifelong nerd who has done all sorts of different things in technology, high-frequency trading, and then He went into the entertainment industry, working with Red Camera and collaborating with these different companies, and then got into video games and architectural visualizations, working and collaborating with these different companies to be able to do their outsourced R&D for immersive technologies. It sounds like now he's working with a number of different automobile companies to do different visualizations, doing these different branded experiences that would show at these auto shows. and also did this whole like architectural visualization of Drake's home which I got a chance to watch the video after talking to him and it's actually pretty impressive in terms of for his first project ever using Unreal Engine and also just kind of funny he's got these little posters of Gabe Newell and Tim Sweeney, John Carmack, a lot of these kind of luminaries from the VR world imagining that Drake would have posters like life-size posters of these people in his home so kind of funny to see stuff like that and So some of the things that he got to see at Oculus Connect, actually seeing this elixir demo where the hand tracking, where you actually have these different avatar embodiments, where you have your hands get transformed into like an octopus. I haven't seen anything like that. I didn't get a chance to see that. The experiences that were made available to the press were just the insurance demo. You'd have to wait in line for hours and hours in order to see the other demo that Andre saw. But it was interesting to see his reaction to that. My sense is after doing the Leap Motion hand-tracked demo back at GDC 2017, that is probably one of the most incredible hand-tracking demos that I've seen, just because you're able to actually grab things and grab a virtual object and to really have the collision really finely tuned. And I haven't seen anything like that done in any of the demos. And that was after many, many years for the Leap Motion, which Ended up getting sold to UltraHaptics and then they rebranded into UltraLeap. So it's a blending of UltraHaptics using sound waves, high frequency sound waves, in order to create these haptic effects. They were the biggest customer for Leap Motion and then they kind of merged all the different assets and now are working together. So yeah, the leap motion implementation of that and being able to actually like grab objects, that's the best that I've seen. And I haven't seen anything like that for some of the tech demos that I was seeing here. So I think it's going to get to that point eventually, especially if they start to add other things like, you know, the the control labs, if there's something that you put on your hand to help solidify it. The one thing that I did see actually of doing the hand tracking demo, you know, you kind of have to have your hands up above in a way that it's in the field of view. If you kind of drop your hands, then it drops the tracking altogether, which I think is pretty typical for most of these. But as I was walking around this virtual environment, more naturally trying to just kind of walk around, I did find that in order to keep my hand presence there, I had to kind of keep my hands up above in the field of view otherwise it would kind of take a second to find my hands and orient and so it would kind of break the presence in some ways and so I do think that once it's all ironed out that it's going to increase this ability for you to not have to know all the different controllers and all the buttons. I think it is unfortunately a bit of a barrier for people who aren't familiar with trying to figure out you know what the control schema is and oftentimes I find that once you start to use all those buttons and all the different ways of Locomoting around and you give it to someone who's never tried VR then you have to kind of go through this whole Education process and so it's very powerful those controllers but at the same time sometimes you just want to be able to have your hands in the experience and I feel like where this is gonna go in the future is that it's gonna make it a lot easier to jump into VR and to put anybody in a VR for them to be able to start doing things without having to teach them a lot of these abstractions with a controller. But moving on to like the big thrust of the debate and discussion, I can really hear a lot of the points of what Andre is saying, which is that, yeah, you know, Facebook's a data company. They're going to be collecting all this data, but let's not slow down the process of VR adoption. Let's push forward and let's not let all these, what he sees as kind of like inconveniences of the ethical open questions start to slow down the adoption of VR that these are just things that we can gloss over at this point and I think to a certain extent that's correct that we do need to have the VR industry keep going and that the quest does need to be a success that Facebook has a company needs to be successful to be able to continue to bootstrap and push forward virtual reality as a medium they're very gracious and be able to hold things like Oculus Connect and to invest in literally millions of dollars in order to hold these different types of conferences. So it's actually doing a huge service to continue to keep the momentum going. However, I would say that is also important to have these ethical discussions and to just keep the conversation going. A part of my frustration is that it's been difficult for me to find people to even go on the record because things are so unsettled and to have a conversation about these deeper ethical issues. And I think Andre's also right to talk about how a lot of these people that I would end up talking to, that it is such a massively huge issue that it kind of transcends any one of the individuals. You really need to have like a group discussion with an ethicist, a lawyer, people who are talking about the business aspects and to really think holistically about all these hard problems that haven't been solved yet. and whether or not we're going to continue to do this type of surveillance capitalism business model if that's Facebook's plan for the future to continue to harvest all this information and you know what that exchange is and if that is already created this asymmetry of power how to start to deal with that. So we've covered this in previous conversations and you know I think the thing Andre is saying, you know, it's not you know, let's not just drag Facebook. There's other companies I think this is a point that he's made is it's like look at Sony look how close down they are look at HTC which you know have been a little bit I guess hit or miss in terms of some of the responsiveness especially in their engagement in the community They don't have an equivalent Gathering that's bringing the community together. So they've seemed a little bit more scattered over the years and And then there's Google, which has pretty much gone AWOL when it comes to specifically virtual reality. They've been integrating a lot of their immersive technologies and principles into augmented reality and really bootstrapping and putting a lot of effort there. YouTube is probably their biggest initiative, but they continue to have these different startups. They do Tilt Brush and Google Earth VR and Alchemy Labs. So they're certainly continuing to move forward, but I'd say that Google's mistake has been that they've tried to go to scale first without actually engaging a community and providing value to a small group. They tend to kind of go straight to scale. So a lot of their focus has been trying to do immersive 360 video at scale, which if you look at the ecosystem, YouTube, VR, and a lot of 360 videos, leveraging a lot of the business models of Existing YouTube that's probably like their biggest stronghold when it comes to what they're doing in virtual reality But in terms of their developer relations and working with the daydream It was kind of like a false start in terms of not even actually having a virtual reality headset at Google IO of 2019 and so just kind of to show how a lot of the daydream is as a platform seems kind of functionally dead that there's not a lot of excitement and with it out there. I mean there may be some enterprise applications and I know that I've talked to some developers who are very happy with in terms of just deploying stuff out using the Daydream that it's actually got a little bit more enterprise support. But in terms of a consumer device it's certainly not something that you see a lot of people talking about. But in terms of Oculus, I've heard mixed and different things. Andre's had a good experience of being able to get very responsive to get answered. He's a part of the Launchpad program, which you get into the inner circle, you get access. But I've also heard from probably even more developers of people who aren't in that inner circle. It's like very difficult to get in with that inner circle. Like it is very much like you got to know a person within Oculus in order to like really get things done. And so I've had people that have been I'm like, hey, I'm in India, and I need headsets, and I can't get any response from anybody from Enterprise VR. And then I was like, all right, well, here, I'll forward you to somebody that I know, and then see if it happens. And then they do end up getting connections, but there has been frustration that it's not just open for everybody to come in and to be able to have that same type of response. And so it does feel like you have to know somebody in order to get some reactions with Facebook. So I guess that that would be a part of the additional context I would give from Andre's experience and I think in terms of the polarity point between myself and Andre is that I tend to be a little bit more of an idealist meaning think about these ideas and these concepts and you know, what would be a in a perfect ideal realm, but also just thinking about the future, being forward-looking, looking at what we know in terms of the patterns and what we can look at the past. And so it's been difficult to have these, in some ways, more philosophical discussions about ethics and privacy with Facebook, especially because, you know, just even like biometric data privacy, and the response I get from Facebook is, you know, it's very pragmatic. Like, we have not released a product that has included biometric data in that degree, so we cannot talk about the philosophical implications of that. until we actually have a product. And so there's this kind of pragmatism that I run into personally when trying to have these different discussions. But overall, Andre is also kind of caring for that much more like pragmatist, entrepreneurial, what is the bottom line. And let's not get too bogged down into thinking about some of these philosophical abstractions because we're just trying to like bootstrap an entire industry and make this viable. think that's an important dialectic to have because in the absence of having that red teaming idealistic perspective, then there's a kind of a blind moving forward. And next thing you know, we have 100 million users, and we've kind of architected the most dystopic surveillance technology imaginable, where it's in the hands of countries and corporations that are trying to request information that's very intimate, very personal, and could be compromising for the safety for a lot of people. And so just like having some of these discussions about the ethics and privacy, I think are are vitally important to have this dialectic between the the pragmatism of you know, what is going to actually help bootstrap this industry and let's not get too bogged down and keep the eye on the prize and doing what we need to Produce the quality platforms, but at the heart of it there's this other aspect of like cultivating trust with the community because you know Facebook as a company has Burnt a lot of trust in terms of the unintended consequences of a lot of their actions in the past especially because they have been very much preferencing the connectivity over Privacy and seeing that having that connectivity into you know getting that information into the wrong hands could actually be weaponized into like a different levels of information warfare or helping to potentially catalyze Genocide in Myanmar all these kind of like things that they were probably never thinking about when they started Facebook But yet unless we start to think about these Potentials then they're not going to design for that and I think that's part of what I'm just advocating for is just to start having conversations and to think about, yes, there are all these different trade-offs, but to try to figure out what the ethical and privacy frameworks that we need moving forward, because this is this whole new world, and there's a lot of implications that have yet to be resolved. And I think just keeping the conversation going is a lot of what I'm trying to do here. And that there is an element of my own lack of business acumen that can get projected onto a very successful business like Facebook. And in a lot of ways, Facebook shouldn't listen to me in terms of taking any of my business advice. But I do think that there's certain aspects of these deeper conversations of ethics that unless we have those now, then it could potentially completely crumble down the road when they're not doing the proactive things that need to be done in order to ensure that this is done in an ethical way and that they're not creating this asymmetry of power of having all this really intimate biometric data on us to be able to reverse engineer our psyches in a way that is going to have undue influence over us. And if that is in the hands of Facebook, that we can trust them. But what happens when that information gets out there? So For me, I think we got to the point of agreeing that it's important to have the conversations and it's important to have the dialogue continuing. And I have a whole series of different interviews and conversations talking to different people like Mozilla and 60AI and Magic Leap as well as Vin Agency at SIGGRAPH where we did this whole deep dive into different issues around ethics and privacy. Mozilla sent me out to the view source conference in Amsterdam, where I was able to talk to a variety of different people from the W3C looking at different issues around ethics and privacy. Privacy is like a huge topic within the web right now. And a lot of companies that are trying to architect the browser as being at the front lines of trying to create a browsing experience that is protecting people's privacy and to mitigate the different aspects of data collection. So this is like a huge battle that is happening within the technology sphere. On one side, the companies that want to hoard and collect as much data as possible, the data collection companies like Facebook and Google, and the other companies in the other standards organizations, whether that's the W3C or Mozilla and other people who are trying to build a future that has a lot of the privacy that's built into the technological architecture itself. So I'll be diving into a lot of that into my next series and to really see a lot of these different perspectives and see what we can learn from what's happening within the web browser world and how a lot of those deep insights and ethical principles can start to be applied to immersive technologies as well. So that's all that I have for today. And I wanted to just thank you for listening to this podcast. And halfway through, I've got three more podcasts to go into some of the highlights, I'll be diving into some of the technical aspects and some of the experiences from Nathie, a YouTuber, as well as some final thoughts from Sean Oaks, talking about the Facebook horizon, the future of the strategy that Facebook has with their social VR, And, you know, I ended up doing about 27 interviews, 17 and a half hours. I'll be able to dive into lots of different topics around education and other topics as well. But I wanted to just kind of dive into some of the discussions that I had that were more explicitly talking about Facebook or some of the announcements. And, you know, this is a huge effort that I'm doing here for the Voices of VR podcast, probably unreasonably ambitious for what I've set out to do, trying to capture a real-time oral history of all of this and to help facilitate some of these deeper ethical discussions. But I am reliant upon a lot of donations and support that I get from Patreon and listeners like yourself who not only helping spread the word about the podcast, but are becoming sustaining members of this podcast and wouldn't be able to do that without my Patreon supporters. So just want to send a shout out to my Patreon supporters, but also to invite other people who if you find these to be valuable conversations and to continue to maintain and Publish a lot of these conversations I've like I said probably record around 1,300 conversations over the last five and a half years and published over 800 of those but it's about two-thirds of the stuff that I've captured and one to continue to just keep on capturing and digesting and synthesizing and contextualizing all this information and so I If you want to see this as a project continue, then consider becoming a member of the Patreon. Just $5 a month is a great amount to give that allows me to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
