Immersive advertising campaigns on phone-based AR platforms like AR Kit and AR Core are currently subsidizing a lot of development within the larger XR ecosystem of developers. I heard of many AR developers at Magic Leap who were doing phone-based AR contract gigs, and then working on Magic Leap in their spare time. At Greenlight’s VR Strategy Conference I had a chance to talk to Unity’s Head of AR/VR Brand Solutions Tony Parisi, who has been on the frontlines of evangelizing immersive and interactive advertising solutions to design agencies and brands. From what I hear from Parisi as well as from other XR developers, there’s a critical mass of phones that have AR functionality enabled which is generating a lot of excitement and traction with brands who are funding experiments with immersive and interactive storytelling on these new platforms. Parisi updates us as to where Unity is currently at, and what they’re committed to in terms of where the future of spatial computing headed.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
Here’s Unity’s Brand and Advertising Sizzle Reel:
https://www.youtube.com/watch?v=PxY7x6uqtqU
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So last week was the Magic Leap leapcon, and there's two things I think that are going on. One is what's happening overall in the AR VR spatial computing ecosystem, which I think is pretty much there's a momentum in every single indicator that I can see from the major tech companies that are moving in this direction. And I can also see that from what's happening in indie games, what's happening in the culture, what's happening with the youth and immersive entertainment. I just see that we're headed straight towards this path of an immersive future. And anybody who's listening to this podcast is likely already seeing those signs on the wall that this is kind of where things are going. However, not all the journalists that are out there are all in or buy into that this is going to even be a thing. I mean, it's kind of shocking to me when I see some of these pieces like Brian Merchant's piece in Gizmodo called the Magic Leap Con, where he's basically just questioning the core utility of augmented reality, that this is just a toy. In this final paragraph, he says, the general technology is cool, but in search of a use. No one ultimately could articulate why Magic Leap needs to exist outside of entertainment. Now, I've got hundreds and hundreds of interviews that sort of go into why it's useful outside of entertainment. I mean, just the principle of embodied cognition within its own is going to be completely revolutionary. In terms of like, we think spatially. And so as you have information presented to you spatially, you just remember it more. I mean, I just came back from the virtual reality strategy conference just to show how the HoloLens is being used in so many different applications and use cases from enterprise to medical to architecture, engineering, and design. to anybody that's doing anything with creating 3D objects, like being able to actually see them and prototype them. I mean, the list goes on and on, and I'll be covering some of those more in future episodes. But I'd say one leading indicator of what's happening right now is the phone-based AR ecosystem. I mean, it's absolutely just exploding in terms of the excitement around what's possible for doing these types of immersive phone-based AR games. I just went to Indiecade, and a lot of the independent games that are going to be coming in the future are these multimodal spatial games where you're moving your bodies through space. So getting out of these abstractions of using your thumbs and actually using your body to move through space. I mean Pokemon Go is a leading indicator for where the entire industry is going to be going and it's a matter of how do you translate moving your bodies through space into a story? And I think this is a big open question for what that exactly is, but a lot of augmented reality games now are being pioneered on the phone-based AR. So everybody that I talked to at Magic Leap, I don't think I met anybody that was like, I'm all-in working full-time 100% on Magic Leap all the time. It was more like a bunch of people that are already in the immersive space, they've been doing some virtual reality projects, or they're doing phone-based AR contract gigs, and they're getting those contract gigs from advertising. And in this podcast episode today, I wanted to talk to Tony Parisi because he is on the front lines of interfacing with brands telling them what is possible with immersive advertising. And so we have this new form that a lot of the storytelling is going to be experimented in these immersive advertisements that are going to be going into these immersive games. And if you make an advertisement playable, then it is even more immersive. And so what does that look like to be able to have some sort of immersive engagement within the context of an augmented reality experience? So this interview with Tony Parisi, we're going to talk about what is happening in the brand and advertising space within augmented reality, because at the end of the day, in order for the ecosystem to really thrive, we need to have some sort of revenue streams. And right now, the biggest revenue streams are with phone-based AR and these immersive advertisements that are happening. And so that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Tony Parisi happened on Wednesday, October 17th, 2018 at Greenlight Insights Virtual Reality Strategy Conference in San Francisco, California. So with that, let's go ahead and dive right in.
[00:04:15.947] Tony Parisi: I'm Tony Parisi, head of VR and AR brand solutions at Unity Technologies. I've been at Unity for a couple of years, originally starting with an oversight of all the VR and AR the company was doing that was starting to power different industries out there like automotive and the film industry and architecture and these other places that aren't traditional. game industry customers, and we took a look at a lot of different areas there. And we're starting to actually spin up businesses in a couple of places there. But I got really excited about creative media and advertising and had been working with our ads team. We have a very big ad platform, we can get into that. And I'm now overseeing rolling out an AR advertising product on mobile phones.
[00:04:57.878] Kent Bye: Great. I know that last year I was much more skeptical about ads and the future of advertising. And I have since went to the Decentralized Web Summit. I see that there's blockchain technologies. They're really not at a level of maturity to have any sort of viable alternative. I talked to Vint Cerf from Google. I was like, hey, maybe you should stop doing surveillance capitalism. I think it's bad. And he's like, well, OK, well, do you have something better to be able to give free information and knowledge to all of the world, is there a better way that you have? And it's like, I don't. There's not a lot of alternatives to really come up with new business models. So it feels like advertising is something that, I think people have a bad experience of advertising online just because of the consent and just sort of like jumping at you. So I still have hesitations for like, what is the augmented reality version of something that's trying to hijack my attention? But do it in a way that's maybe with my consent or do it in a way that I'm choosing and opting to have an immersive experience. The ecosystem, I think, has changed a lot more with ARKit and ARCore in terms of the platforms that are out there and the experiences. And so it sounds like this could be a way to actually sustain this type of innovation that is going to overall bootstrap and sustain the VR industry until we get to mass ubiquity in 2025. Before we get there, it seems like Unity is creating some solutions to be able to actually provide some revenue streams for immersive creators.
[00:06:22.493] Tony Parisi: Yeah, indeed. Let me address that last bit that you mentioned first, though. I'm sure most of your listeners have seen Keiichi Mitsuda's video, Hyper Reality, which paints this hellscape of augmented reality and advertising and invasion and all that stuff of privacy. We're not talking about any of that anytime soon. We're not talking about minority report anytime soon and obviously these are cautionary tales looking forward and recent events in the world have alerted us to possible issues around and actual issues around privacy and all of that, but what we're doing is much more modest, right? The augmented future and being fully immersed and having always on AR and mixed reality, we'll get there and we will have to solve some of these issues as we go. But the fact of the matter is nobody likes advertising that's that invasive. On the plus side, what Unity is really good at is helping our developers succeed, right? We have this content creation platform that makes it easy to create this stuff, but the fact is most of our mobile developers don't monetize by selling apps in the app store. They monetize through advertising and in-app purchases. So a big part of the mobile app and mobile game economy is driven by advertising. The way it works in a Unity ad is that you reach the end of a game level and you want to proceed in the game or you crash out and die and want more lives and you have the opportunity to watch ads in order to proceed through the game. This has become a natural exchange of value between the player and the publisher of the game, and the advertiser who wants to get at those eyeballs to advertise something. And so this is basically standard and established practice in the world. And what we're doing with AR, we're talking about AR on phones. We're talking about ARKit and ARCore, simple immersive content, stuff you see through mostly the back-facing camera, a product displayed on a countertop, or a car or vehicle in your driveway. a dancing character, a portal where you can walk into another world by moving through and holding up your phone and walking into it. And these happen at those same moments that you would have seen a video ad or a simple little what we call a playable ad, which is an interactive experience where you can tap. You've already been playing a game already, so you're leaning in, right? So you can tap and interact with a brand. And so we're doing that with the camera turned on, and the back-facing camera, and using all the cool tracking that's there with the new technologies, so that it's a really good, persistent, you know, you're really tracking and moving on the objects. AR experience delivered as this little bite-sized 30-second nugget, and then you get back to the game. So that is far removed. You know, maybe it's a first step toward this ubiquitous AR advertising world that you're concerned about, but it's far removed from that at this point. It is a a simple taste of immersive content and may arguably be a lot of consumers' first exposure to augmented reality and immersive content because it's relatively frictionless. It shows up in an app they've already got and it's in the natural flow of the games that they're playing.
[00:09:21.745] Kent Bye: Tell me a bit about how, because Unity is a game engine enabling game developers to create games, it seems like a natural path to have advertising that has some sort of game elements where you can express your agency, make choices, take action. And so, are you starting to see that with the actual interstitial ads or whatever immersive ads are coming up, like you have almost like a mini game within a game?
[00:09:46.633] Tony Parisi: Yeah, we are. I mean, we call those playable ads. And so what Unity delivers in its ad product now are multiple formats. Linear video, at the end of which there's usually a call to action called an end card. There is this playable, which is a mini game. So it's a little HTML5 game and you interact with it. It could be an excerpted version of the game that's being advertised. It could be a little game level you're playing within the game it's hosted in that entices you to install the other game. That's the way A lot of our money is made, it's through, you know, cost per install model. We get a bounty when a user installs a new game and most of that gets paid out to the developer that hosted the game, the publisher that hosted the game. So there's a user acquisition fee basically, right? And that playable could be a mini version of that game to entice people to play it. But we're also seeing now in our brand business, we've opened up our business to brands as well, not just game publishers trying to get users. And that's monetized on a more classic impression model, cost per impression. And advertisers like this brand playable idea too. And we just did a campaign with Sonic, the fast food company Sonic. It's called Sonic Slicer. It's basically Fruit Ninja with shakes. So you're waving your phone or tapping and slicing up shakes. And as you do, the flavor of that shake is revealed. And it's a way for them to roll out a new product that's more engaging than just watch a video. So that one's not even AR, but effectively those same mechanics can work with augmented reality as well, where the camera could be turned on, the environment could be incorporated, objects could be put into the environment, but you could still have those kind of game-like mechanics to interact with the brand content.
[00:11:16.575] Kent Bye: Is there a CPM difference between having a 2D mobile game ad versus an immersive ad in augmented reality? Do you have a deeper level of immersion, and are those ads selling for more?
[00:11:29.584] Tony Parisi: The ads are selling for more. I can't get into the exact details, but they sell for more than our standard video. Brands are really excited about it. A, there's novelty right now, which is driving a lot of our engagement and partnering with the brands. But beyond that, there is engagement. You're doing more in there than you would be if you're just looking at a linear video. And time will tell. We've just launched this product. It was in beta in the summer. The early engagement numbers we have, plus after ad surveys in terms of, hey, would you buy this stuff? Those are looking really encouraging. But we need more data over time to really prove that out and be able to sell this more to more and more advertisers. Early returns are looking good, and we're not the only ones out there doing this. Facebook's out there piloting this stuff now. Snap is doing it as well. We're seeing Amazon basically doing AR shopping, but they've also launched an ad platform, so it won't be, I can't imagine it would be long before they close the loop between the advertising and the purchase through their platform. So I think as an industry, we'll learn together about this, but for sure the idea is that higher engagement would demand higher CPMs.
[00:12:34.671] Kent Bye: Well, I've been saying that we've been moving from the information age to the experiential age and that we're having more immersive interactive participatory experiences that unfold over time that have this almost like this tension between the dissonance and the consonance of in order to have something that passes over time it has an arc and so you have this natural narrative arc that you're engaged with in some way and participating and so Because of that it feels like some of the things I've been seeing just in general the trends that you look to like comic-con or packs or South by Southwest you have these activations of these experiential marketing, big immersive theater type of experiences, escape rooms, just brand engagement, but it's a physical installation. And I imagine that in order to scale that out, we're going to see a lot more of that type of immersive theater type of experiences through phone-based AR. And so I'm just curious from what you've been seeing in terms of this, do you see a larger shift into experiential marketing?
[00:13:31.830] Tony Parisi: Yeah, for sure. And the advertising units I'm talking about, we view these as potentially parts of more holistic campaigns. So these could be in-store activations. They could be these live group events and shared events that would be delivered through apps and through some very high-touch ways of delivering it on a location basis. But they can be complemented with pieces of that experience being delivered into mobile ad supplies like ours, which then means that folks who aren't there on site can have a piece of that experience. The whole idea of a mobile app ad is you're going to see that wherever you are, on the train, at home in your living room, at the office if you're taking time out to play a little game. So these things can be connected, but you can have effectively an overlap in the brand content. You can have a continuum of the brand content. And you can take some of those things that are discovered through ad units and potentially even then drive foot traffic into malls, into these other experiential activations if they're persistent enough.
[00:14:31.842] Kent Bye: Yeah, I just had this idea or concept that you would have a site-specific experience that would be on the site, but maybe there would be a larger narrative that would be unfolding that you would have to maybe locomote through space and through the city. And as you're on your way, maybe you have different components of the Story and then once you get there you have some sort of culmination or climax to the story But it's on site. And so have you started to see any type of experimentation for a site-specific AR?
[00:14:59.563] Tony Parisi: Um, can't really talk about that right now. There's some things in discussion that we're involved in. So, you know, I can't speak to it, but the idea generally, yes, is exciting. And there's some other folks out there who are pioneering a little bit outside Unity. They're pioneering platforms that can help with this. I don't know if you caught up with Experial while they've been here, but they're really exciting. You should look into them. There is Sunnyvale, California based company that is all about turning a sports arena into a giant video game console in the sense that you can see content on the Jumbotron, you can point your phone at it and use some AR to then trigger an activation that may go get you to go get a can of beer and scan it and then win more points for something that's, you know, gamified within the experience while you're at the stadium, right? So imagine tying that to these kind of more mobile advertising campaigns. Super exciting set of ideas. So definitely take a look at what Experial's doing.
[00:15:55.854] Kent Bye: Have you started to see anything with specialized audio in advertising?
[00:16:00.316] Tony Parisi: No, not me personally. I've been hearing a bit of chatter about that. I mean, the thing about audio that we always forget is that that's on even like when the video stream goes down, when the screen goes black. It's all around us, whereas visuals are just in front of us, right? Our ears hear in 360 all the time, right? Whereas our eyes can only see in 360 when we turn our head, right? So I believe that that's going to have a major and critical role in this along with voice input and some other related technologies to make a more full sensory experience around this. I think with 360 audio we're going to have to, I mean, I don't know what the state of the tech is in terms of having good binaural and just having it through earbuds and all that, but I'm sure some folks can really drive that forward, or whether you'd need something more like the Bose headsets or something to really make that effective. If you could do it with ubiquitous hardware, the way we're talking about doing AR on the phones, that would be best for reach.
[00:16:55.324] Kent Bye: Yeah, I guess I was thinking of the Bose hardware that has the spatialized audio with the glasses that I've been hearing some exciting stuff about and some experiments that I'm looking forward to checking out as well. But yeah, I just feel like that all the multimodal aspects of our sensory experience eventually getting into this platform, but that starting with phone-based AR, maybe eventually into something like the Magic Leap or HoloLens, are you also starting to see any traction in terms of the head-mounted displays and advertising, just because it's such a small market, I don't know if it'd be even worth the trouble to be able to port that over, or if you really would want that context switch once you're in it. It almost feels like when you're in an AR, head-mounted display, or VR experience, you kind of want to just be, like you've paid enough to be able to get to this point where you really want a premier experience, you don't want ads in there, but have you started to see either advertising initiatives that are either doing head-mounted display, VR, or AR?
[00:17:49.761] Tony Parisi: Yeah, so it's kind of a two-part answer I would say to this. Right now the experiential marketing type of activations are happening with headsets as well. So there have been great, great marketing pieces with VR done, such as the Sperry store, the flagship store for the boating shoe. A New York-based agency did something at that store in New York, which was all about recreating the America's Cup race. I don't know if you got this, but they actually captured in 360 the Oracle team on the boat and then they replayed that and built a Unity app that had a Unity plug-in that drove a moving platform you stood on so you moved with the boat race wearing a HTC Vive and they blew wind and water at you. And so that's incredible. It's as immersive as you can kind of get with where we're at with this tech, right? So you basically got the void type of thing, but that sort of level of LBE. But it was done in this very flagship store, which is fantastic. But you know, the foot traffic coming through a store like that would be in the hundreds to small thousands. So again, it's about reach. And we are hearing there are some brand clients very much interested in HoloLens or Magic Leap style mixed reality activations in the same vein where they would be It's not about getting to all the consumers that have them, because we know there aren't that many consumers at advertising scale that have these headsets yet. But it's a deep experience where the advertiser controls the stack because it would be in store, for example, in a retail setting, in a store, in a mall, or on site, like I was mentioning with Sperry. So I think we'll see more of those things. But again, if those can be combined with a holistic campaign that also reaches people through other formats, it's going to be great. Now, if we want to look forward three to five years, and when those kind of headsets are potentially ubiquitous, and I think a lot of us hope and dream they are, that a mixed reality headset like a Magic Leap, price comes down, that we have the audio, like a Bose audio built into that headset, now we're talking about a different game. And now you might ask, yeah, well, I think it's going to depend. Is that a premium? headset that still cost thousands of dollars? In which case, yeah, most folks are probably then paying for premium content and don't want to be advertised to. Or is it a lower price than ubiquitous consumer phenomenon outside the living room or could be anywhere more like an Oculus Go? And then I think you're going to find that the dynamics that we have that drive advertising revenue today are going to be in play, which is you're going to find, if the latter is true, if it's ubiquitous, that a lot of people would rather pay with their intention than pay with dollars. And it'll be the advertiser who pays with dollars. If it ends up being a premium experience, there may be a bit of product placement. There may be a little bit of interstitial stuff happening in the menus, like you see today on a home Netflix or Amazon. You see these today in your home systems. We'll see. Time will tell.
[00:20:33.927] Kent Bye: Is Unity looking at the AR cloud at all in terms of being able to do geolocated type of content? Or is that something that would be integrated with something like 6D AI or some of the other AR clouds that are out there? Or something like Mapbox to be able to do the geolocated information to be able to actually put things onto the sides of buildings or whatnot?
[00:20:54.430] Tony Parisi: So the AR cloud is an interesting set of concepts that are getting lumped together. There's a handful of things, right? You've already mentioned geolocation. There is persistence, like within one space, you'd like to have the AR stuff you just set up be there when you come back tomorrow. We've seen Google make some strides there and Apple both with the AR cloud anchors and the world map tech respectively where you know you don't have to recreate your AR and then there's the shared multiplayer stuff and so when people talk about AR cloud they tend to talk about all three of those things and all three of those problems are getting solved at different velocities right now so 6D is about building that map of the world You've got Mapbox and other geolocation services, and then you've got the platform providers like Google and Apple working on the basic persistent stuff right now, and then you've got various entries in the multiplayer. So it's interesting, you know, when it comes to gaming, for example, and mobile AR, if you just look at mobile gaming in general, there aren't that many multiplayer mobile game titles out there that are wildly successful. There are a few. And the multiplayer AR ones, I mean, you can count them on the finger of one hand at this point, right? We've got Pokemon Go and whatever Niantic's coming out with next. That brings all these things together, basically, geolocation and everything else. So I think that's all going to take time to kind of mature. I think the ecosystem is going to have a lot of players in it, like 6D and folks like that. And Unity will have its role in that, which, you know, first and foremost, can't get into too many details about product plans there yet. At a minimum, you know that as those services emerge, Unity will do what it always does best, which is give developers real easy access to that in a way that's cross-platform, in a way that they can publish anywhere. So expect to see that kind of stuff from us, though no official announcements yet. So we're excited about it in sort of general terms, but there's a lot to figure out still.
[00:22:49.559] Kent Bye: Well, I feel like that there's been a lot of design innovations and best practices and what does spatial storytelling look like that's been really being prototyped in VR. But now that we're bringing the physical co-located space and potentially even scanning objects and putting things behind objects and interacting with your local context, I'm just curious if you've seen any AR experiences that are out there that are really defining some of those new user interaction patterns that some of these experiential advertisers should look to for some design inspiration if they want to create some of these AR ads?
[00:23:23.801] Tony Parisi: On the storytelling side, it's so early yet, and from a pure storytelling perspective, if you think outside of, don't think about ads for a second, I've seen some cool early experimentation. Nexus Studios from LA did this project around the Gruffalo, which is a British children's book, and they recreated some of the Gruffalo experience and turned it into some kind of scavenger hunt in the forest, effectively. And that was beautifully done, you know, it was location-based in England, and that was beautifully done based on an existing IP. We saw some demos from within at Apple's Worldwide Developer Conference, and I think Chris and team at Within are, you know, thinking hard about how to do storytelling when you don't control the environment and you need to make the story work anywhere. Could this story be told on a kid's bedspread and as well as the kitchen table or outside and what does that all mean? I think Magic Leap's wonderful content experiments and the fact that now objects and characters are falling off tables and doing the right things with objects in the environment and all that from the tech and demo side are paving the way to what's possible. But I think we need a couple years of iterating for someone to break through on what real storytelling is. And the good news about AR ad content is the story lasts 30 seconds. It's a much easier thing to solve right now. That's get a product on a table. That's how a character dance that you're seeing through the back-facing camera that maybe you're playing a game with or doing some martial arts moves with, but it's way less complicated than a full narrative.
[00:24:53.503] Kent Bye: Well, I've seen there's a couple of different orientations, I guess, using the camera that's facing outward to be able to put an object on your table and maybe something dancing in the background. But I've also seen from a lot of facial filters, either from Snap or Facebook and different brands and campaigns to be able to give you some sort of virtual embodiment that allows you to tap into some sort of, you know, wacky part of your personality that you want to both record yourself and then share it. So you have this viral sharing aspect. And so I don't know if you're finding more of those inward-facing facial filter expression of your identity types of apps. or just naturally, intuitively, more viral sharing in terms of people taking either a selfie or a snapshot and sharing it out versus if they're just taking a video of somebody that's dancing around an object on their table.
[00:25:39.828] Tony Parisi: Well, I mean, I kind of think you answered your own question. I believe the social apps that have front facing camera, really fancy tracker, even back facing camera, you can do body tracking of your friend and put them somewhere else, you know, segment them out and throw them in front of the Taj Mahal or something. Those are naturally more viral. They're naturally more shareable. Those are also the result of massive investments by those companies in computer vision and machine learning tech that they can embed into their app. They can actually, you know, it's It's not like ad content, per se, right? I mean, then you slap a filter on that's somewhat personalizable, but there's this intense technology behind it that is great, but mostly will run in these first-party apps that are inherently social. They're very user-centric. The virality is great. From a brand safety point of view, it gets very interesting, though. So some brands are hesitant to want to have too much user-generated content customizing or changing the way their brand is represented. So it very much depends on what the advertiser wants to do and how they want to reach them as well, if you think about that. But in some cases it's incredible, especially, I mean, oh, I don't know, heck, if you wanted to put a football helmet on someone in your NFL and you wanted to brand something, for example. Anyone's going to want to put that football helmet on their head if they're a fan, right? And it's not going to hurt the NFL's brand. It just helps with sharing and everything else. I just made that one up, but that's a free example, right? But in other cases, if it's a well-known product and the advertiser wants complete control over their brand, then you might not want to see that selfie. It just really depends. And it even depends on the nature of the campaign, not just the brand.
[00:27:16.284] Kent Bye: One of the things that I'm seeing in the VR ecosystem is that there's a little bit of like, okay, what's happening in this overall ecosystem, both VR and AR? I get the sense that there's been a lot of, perhaps as the AR ecosystem starts to have a little bit more phones that have the actual capabilities on it. There's like a viable critical mass so that now the advertisers and the brand companies and they're actually looking at the augmented reality first rather than virtual reality just because there's not enough VR headsets that are out there to be able to make it worth the effort to really do a mass global campaign. there's enough people with AR phones that they're, I guess, more willing to do some of these. And so I'm just curious from your perspective, like, what you see is happening in the overall VR and AR ecosystem.
[00:28:01.885] Tony Parisi: Yeah, I mean, I'm primarily focused on the advertising business because that is my day job, but I'm staying really on top of what's going on in the ecosystem to the extent I can. And what I've been seeing over the last year or so is quite a bit of a shift toward phone-based AR. This is not just for advertising and brand experience. This is game developers who want to reach people. This is enterprises who want just more scale or less friction around installs. And concomitantly, it's the investors looking at all this and going like, where's the action, where's the scale, where's the heat? And this has turned into this situation where a lot of folks are feeling like VR is really stalled or dead, and all the actions in phone-based AR. And I don't think that's true. I think a lot of folks tend to see all this stuff, they filter it, and then they end up having a binary response to it. as opposed to this long dance that's happening, this interplay. If you look back a couple years, all the heat was on VR. No one cared about AR through a device at all, right? We didn't even have ARKit and ARCore yet. But that VR heat drew a lot of attention to augmented reality platforms and technologies like Tango, and that had no small part, in my opinion, and renewing interest around AR that had been around for a decade before that at least, or depending on how you measure it longer, but on these mobile devices have been around for around a decade anyway. And so that has circled back to now all the attention being on AR, but I think that's just a moment in time. I think it's a moment in time where investors want to put their money in something that's going to scale, and creators want to reach as many people as possible, whether they're making a game or anything else. And developers who are building apps and doing services for their customers, they have similar dynamics, yet at the same time. All that said, I was at Oculus Connect 5, and the few thousand people that were at that conference don't seem to have gotten the memo that VR is stalled or dead. The exuberance there was incredible. The keynotes and everything else were largely about gaming and social, and Zuck was talking about Facebook. But in general, I mean, we have creators working across every vertical you can think of that were there and super excited. Enterprise uptake in VR is still happening, because you don't have these form factor issues, you don't have these adoption issues, you're not worried about the wires. The stand-alone, untethered platforms are coming out now with the Santa Cruz Oculus Quest looking really promising, which could be that thing that gets to Zuckerberg's 10 million units that he talked about at the OC5 keynote. That could be a threshold for a self-sustaining VR ecosystem. So, I think we need to wait and see how those things perform in the next year, in 2019, and I think we may see a resurgence of overall industry energy around VR again. It did get to this point where I think there was a natural correction because of over-exuberance post-Facebook buying Oculus, which was both a great thing and a bad thing because then You got a lot of exuberance and a lot of heightened expectations. And I think, you know, that was a natural correction. I think a lot of us were ready for it. Our CEO, John Riccitello, talked about it. He said, we're going to hit this trough. So don't get over your skis, don't get too excited, be ready to ride this out for a few years. And we're in that trough, but we're climbing up that slope, to use all those Gartner metaphors, to climb up that slope of enlightenment to whatever that thing is, I forgot what they call it. The plateau of productivity. The plateau of productivity. We're not there yet. But, you know, all signs are still pointing upward and it's looking good. And so, yeah, we're in a place right now where there's more energy in AR and the binary folks out there will just be like, oh, it's all about AR right now. But that is not a complete picture.
[00:31:34.200] Kent Bye: Well, as I talk to developers in the space, I don't meet very many developers that have created a breakout hit in VR that they're able to sustain themselves completely. Most of the people I talk to are doing other contract gigs, whether it's for phone-based AR advertising campaigns or if they're doing enterprise, or medical, or training, some education, so you have this diversity of portfolio that each developer is working on, but that it's a little hard for me to get a sense of what is happening in the overall ecosystem based upon just talking to these individuals, and so being there at Unity, I'm just curious, I know the last time we talked, you said that you still kind of see this doubling year over year in terms of the metrics of different things that you can see, that you still see this exponential growth that quite hasn't hit that hockey stick of the turning the point of maybe crossing the chasm into the mainstream. And so do you still see this type of exponential growth or doubling or do you see a slowdown in VR and acceleration in AR? And I'm just trying to get a sense of like if you still see that we're on this exponential growth trajectory.
[00:32:37.237] Tony Parisi: Oh, we're absolutely seeing a slowdown on the VR side of this right now. I don't have the latest numbers handy, and we're definitely seeing an uptick on AR. So I could get back to you at some point about numbers. We will look at this again. But again, we were sort of expecting it, and it hasn't really affected plans too much for us overall in terms of the investments we're making on the R&D side into supporting all the platforms across the whole ecosystem. from the deepest headset-based stuff to the mobile AR. So we're continuing to push forward on all that. We're doing more and more to try and make that easier for developers to move from one to the other. Obviously you're not going to design a super immersive VR thing that has a whole artificial environment and just repurpose that into phone-based AR because you didn't need the environment. The environment was your environment. But you might want to take some of those characters, you might want to take some of the mechanics, and so on. So we want to make those tools as easy as possible. And we have not pulled off the gas pedal on any of these platforms, because we still believe that in the long haul, that trajectory and that exponential growth will be there.
[00:33:42.098] Kent Bye: Well, I've been covering this space pretty intensely in terms of trying to look at the overall ecosystem, what's happening. And I have this own intuitive sense that, okay, this is almost inevitable to me, that this is where things are going. But yet, I was just reading a Gizmodo article covering the Magic Leap Leapcon, and the journalist was kind of like, oh yeah, this was kind of a stupid tech fantasy, like this is a distraction and just a reflection of what is the worst aspects of the tech industry. And I was just kind of like, that he wasn't able to see the larger ecosystem aspect of how each of these major companies are at least investing. I said that they were going all in, and he said, I don't think they're going all in with all this, but it just seems like there's a part of the strategic investment of all of the major tech companies right now that are really batting on this future of spatial computing. But yet I found myself like, how do I make this argument to somebody who is just telling me that I have this experience of the ecosystem, but how would you describe why you are convinced that spatial computing is going to be a thing and what the specific companies are doing that give a reflection to you that this is something that is going to either happen at some point, it's maybe an open question as to when, but it just seems like we're moving in that direction to reach there eventually.
[00:34:53.075] Tony Parisi: Well, first off, I would probably say, in terms of how you were reacting to that response, might as well save your breath, right? It's the Louis Armstrong thing. If you have to ask me about jazz, you're probably never going to know the answer, right? So I think let's leave that as it is. I mean, we could look at the same events and the same outcomes from LeapCon and come to a, you and I, and come to a wildly different conclusion than that person, right? And I think that's just the way it is. We would see the wonder when we would see the possibility. right? As opposed to the however it was characterized as bluster or fantasy or whatever. I mean, these changes don't happen without the folks dreaming, right? And so this is about the dreamers and we're still in a bit of that stage, right? In terms of industry being all in, industry is all in. I mean, Facebook clearly has not let up on their investments on this. OC5 is the latest indication of that, the Oculus Quest and all that they're doing. If you look at most of the other big tech companies that are in an immersive tech somehow and the bigger startups like Magic Leap, There's no turning back, there's no letting up on that. They have different angles on spatial computing, their investments in VR relative to AR may change and shift, and they may pivot once in a while, but I think there's a shared belief that spatial computing is the future among all the big industry players. and the more that we have brands coming online wanting to support that, as well as starting to see some, hopefully, breakout things done in AR, where maybe it's based on IP, like a Niantic type of thing with Harry Potter, or maybe it's some original gameplay. As we see some compelling gameplay, as we see more storytelling, as we see those few and amazing success stories so far that are deeply VR, I have to mention Beat Saber, And I could also mention all the great work Eliza and Jess have done with Spheres. They got picked up for distribution at Sundance. They're continuing, they're pushing. I mean, we're seeing some of the best content we've ever seen. It just gets better and better. If you think about that and you think about Just putting VR or AR in front of any millennial. I mean, it's just different you put in front of them They're gonna be like cool. Can I try that? They're not crusty old geezers, you know, and and I'm one of those crusty old geezers I can barely keep up at this point, right? You've got to let the young folks, you know Sort of point the way on this stuff at some point you just as someone who's too experienced too wizened or just been around the block too many times you start you get studio ears about it, you get kind of tone deaf to it, right? But I think, you know, younger generations have grown up and completely interactive and completely on video games and CG. This is where it is. And there's nothing radical about it. And so, you know, it's easy to reconcile that if you just think that there are always going to be folks who don't understand the next wave that's coming or don't quite see it. It hasn't quite been spelled out for them quite in the right way yet. There'll be some breakout aha moment later for other people as well. So I'm certainly not worried about it and you know, I'm definitely the glass-half-full person about this stuff. I think you are too.
[00:37:50.856] Kent Bye: Yeah, and also just looking at other things like WebVR, WebAR, WebXR, OpenXR with the initiative to have standardizations across the different platforms. I mean, just the collaboration amongst all the different companies to come up with those standards is a huge thing that just to me is an indicator. You have ARKit, ARCore for both iOS and Google, like the investments they're making in terms of the depth sensor cameras that are happening on the phone that are just going to continue to expand and grow, the Facebook adding 3D photos onto their website and 360 videos. There's a certain amount of wow factor with some of this stuff, the people engaging with it and being able to, for Facebook, having this roadmap where they seem to be with the center of gravity of virtual reality, but adding a mixed reality layer on top of that. I had a chance to do the dead and buried experience. at Oculus Connect 5, it was really impressive. I mean, I felt like I was just running around. They told us not to run, but I ran anyway, and it worked amazing. I didn't lose tracking or anything. With Windows, there's Windows Holographic, Windows Mixed Reality, headsets for VR, for the HoloLens, and all the stuff that's happening with all the Kinect sensors. I just feel like there's a spatialization that is happening, and that there's so much compelling use cases with embodied cognition, with enterprise applications, with medical applications, the principles of neuroplasticity. I mean, just like there's so much that is there that is going to make this interactive participatory future that is going to be more immersive, more interactive, and more participatory. And that to me, it's just, I see it, but I guess not everybody does.
[00:39:27.168] Tony Parisi: Yeah, and again, that will come in waves, right? Future is not evenly distributed. And all those technologies and standardization efforts you mentioned, and we haven't even touched on, volumetric capture, volumetric video, volumetric stills, all the amazing work being done there by companies like Realities, by artists like Oz, Balabanian, the new Metastage tech, which is the Microsoft partnership with their holographic capture tech, Metastage out of LA, Christina Heller used to be a VR playhouse. I mean, all these things will come together,
[00:39:56.558] Kent Bye: Just when I mentioned also depth kit which is from James George and that to me that's gonna be a low-cost volumetric capture that's gonna democratize the access of taking these these depth sensor cameras and just there's at least two or three experiences that were at Tribeca that was part of the Tribeca's initiative to be able to democratize access to the volumetric capture so that you don't have to have millions of dollars or even tens of thousands that you just get the camera equipment and start to capture stuff and it gets translated into an MP4 file that has this depth map that gets translated into a Unity experience of having a volumetric video. To me, that's just mind-blowing for what that is going to enable in terms of creativity and allowing people to have access to these volumetric tools and to create these super stylized experiences that are going to be, I think, revolutionary in terms of the creativity that's going to come from that.
[00:40:46.655] Tony Parisi: Oh, and I'm so glad you mentioned James George and Simile. They are definitely doing it in a more democratized way, using all kinds of devices that are easily accessible.
[00:40:55.681] Kent Bye: That's Scatter, yeah.
[00:40:57.062] Tony Parisi: Yeah, Scatter. I think they're called Simile now, right? Yeah, Scatter is the production company that made Zero Days VR and using that in-depth kit technology. Amazing. And I'm actually going to be hanging with James next week and Bill Platt, who does the Volumetric Meetup Group in New York. There's a lot of interest in that there. So yeah, democratize capture as well so it's not always original content creation. Bringing it into Unity where all the pieces can be put together with Tilt Brush Art and everything else and any other plugin you want to make amazing experiences. Frictionless access via web when the browsers finally do web AR. I mean, there's been talk about it. I haven't seen the products released yet. The WebXR standard is now becoming a W3C standard. It's moved from an interest group, what they call a community group, into an official working group in the World Wide Web Consortium. So we're going to see lots of browsers supporting that now. And we've seen that Oculus and Samsung and Microsoft and a lot of these vendors are already supporting that. Formats like GLTF. It's just going to reduce friction throughout to have all these. I mean, anyway, you know me. I'm preaching to the choir. I think about the standard stuff, too. And I always talk about it. We wouldn't be having this conversation that everyone is going to be able to experience if we didn't have standards for audio equipment, for how the content is delivered. And that is inevitable for 3D too. And all of the sort of proprietary walls around that that have happened either for good reason just because you want to control the experience or for no good reason because Someone want a vendor control of this. Those are all breaking down. And we're seeing industries and customers ask for those to be broken down too. Advertisers don't want to write things in a proprietary format. They want to make one ad creative and run it everywhere. And that's just one example. For e-commerce, we're going to see the same thing. You need to be able to reach everybody with this, whether it's products placed into a Unity app, whether it's showing up in a Unity ad unit or whether it's just running in a mobile browser and just take a quick look at something like an Apple showing this now in Safari as well, right? Quick look at a product. So we're going to see all those barriers of silos kind of fall away over the next couple years and that to me is one of the more exciting things about this course.
[00:42:59.883] Kent Bye: Yeah, me too, for sure. So for you, what type of experiences do you want to have?
[00:43:07.060] Tony Parisi: I think I talk about this with you every time, and so while I'm super excited about creative media, and that's what I do for a job right now, and it's, to me, creative media is either monetized through premium content or advertising, but it is all about the creators and telling a story. I love that, and that is going to be the biggest economic driver that we have, maybe outside of games, or maybe it's both in tandem. To me, it's about education and specifically about recreating the entire history of the world and the universe. Taking the collective knowledge that's Wikipedia today and accessible like that and turning that into a living book of everything we've ever known, everything we're trying to discover. And that could be historical recreation or it could be molecular visualization for science. It could be any kind of education you can think of. To me, that's what continues to drive me here. It started with science fiction. It started with seeing Princess Leia on the tabletop in the original Star Wars. But to me, that was always about the precursor to a space-faring civilization that had all this knowledge at its fingertips, holographically, and we are totally going to get there.
[00:44:15.173] Kent Bye: Great. And so for you, what do you see as the ultimate potential of virtual and augmented reality, and what it might be able to enable?
[00:44:23.729] Tony Parisi: Understanding, in a word, and that's what education's about at the end of the day. The more we know, the less we fight each other. And so, you know, hopefully ultimately making us a yet more peaceful civilization. We're on an onward and upward march toward that, though, you know, we're kind of stumbling right at the second. And I believe that the more technologies we have for communication, sharing globally, communicating globally, acting globally, the better off we're all going to be. And so to me, that education's the biggest part of that. And with that kind of content and ability to share it and do it together and work on it together, we will develop a shared understanding. It'll be just that much harder for us to, I'll just say it, G-rated, mess around with each other as a species.
[00:45:10.757] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the XR community?
[00:45:14.961] Tony Parisi: No, other than keep on trucking, just keep on plugging away. We got this.
[00:45:20.405] Kent Bye: Awesome. Great. Well, thank you so much.
[00:45:22.387] Tony Parisi: Thank you, Kent.
[00:45:23.068] Kent Bye: Always a pleasure. So that was Tony Parisi. He's the head of AR VR brand solutions for Unity. So I have a number of different takeaways about this interview is that first of all, I think the big takeaway here for me is that phone-based AR is where a lot of excitement is. And talking to a lot of developers at the Magic Leap, LeapCon, there's a lot of those immersive creators who were doing these contract gigs for different brands to be able to experiment with what's possible for advertising and different experiences within phone-based AR. So the ARKit and ARCore developer ecosystem, because there's so many phones that are out there, it's at a scale that you can start to really make a viable living off of. And not only that, but that the same types of projects that you're able to do with an augmented reality, you're going to be able to upscale into some sort of head-mounted display Magic Leap experience. and then at some point maybe even create a location-based entertainment dimension of that and so you can start to see that there's going to be this progression of like maybe you have a brand activation that's using a magic leap at a store you are using your phone-based AR to draw people to the store, to the location, to do a very specific activation and experiential marketing is going to be a huge thing. I mean, we're already starting to see what is possible with what happens at South by Southwest with all these immersive activations, with these experiences that are completely immersive, you're embodied, you're moving through space. It's happening over time and you're immersed within the story. And I think that right now it's going to be happening at these different activations and there's different haptic integrations, like Tony was saying that. Sperry was doing activation that was doing essentially what sounds like a location based entertainment virtual reality experience and that if they're able to create an experience that is reflective of the brand and they're able to get foot traffic and people coming in, then that's going to give people a visceral experience of what their brand is. And then they're going to have those positive associations. And then I think it's just going to like revolutionize what is happening in advertising. If you can give people a very good experience, then you're just going to have a positive association with your brand. Now, right now, I'd say it's like we're at ground zero for what the best practices for augmented reality spatial storytelling are going to be. And I think probably immersive theater, from what I've seen, is some of the best practices of what it means to tell a story as you're moving your body through space. And for when I'm talking to a number of different augmented reality developers, that is the big thing that's different from people who see VR versus how they see AR. When you watch VR, you're kind of been trained to stand still and to look around and that maybe there's a room scale experience, but that we've had this teleportation mechanic because we can teleport and move around. And so still we're not using our physical bodies to actually move through the space. And so the best use cases of augmented reality are actually like locomoting and moving around. And so what does it mean to create a phone-based AR to start to physically move around a space? There was a really fun game that was one of the finalists at Indicade called Laser Mazer which was a really innovative way of taking a tablet and being able to do like an augmented reality game where you're actually looking through the tablet in a portal but you're actually moving physically through space. Because you're trying to walk through a maze and you have a limited space, they have a mechanic where as you push the button you are able to pause the locomotion in the virtual world. You can move physically so you can reposition yourself And so you actually kind of have to do this maze, both in the virtual world, but also a maze of using whatever constraints you have in your physical space and how you're going to move around space. So it's almost like a game within a game. And I feel like the Laser Mazer game that was at Indicate is a perfect indication for a mechanic that's going to open up all sorts of possibilities for augmented reality games. I have an interview with Sam Roberts where we'll be diving in and digging into some of the deeper trends that he's seeing within the curation of independent games. And it's a lot of multimodal experiences, a lot of immersive experiences, augmented reality, spatial experiences, interactive social experiences where you're trying to embody yourself into these different systems. So there's a lot of like tie-ins to both virtual and augmented reality that I'll be diving into that here soon. But I wanted to dive into this podcast just because I feel like it's at the baseline, the economic foundation for what the future of immersive computing is going to become. Because a lot of the excitement originally around virtual reality has died down. It hasn't had the penetration for the virtual reality headsets to be in enough hands. Augmented reality is going to be on everybody's phone and there's just going to be a way to be able to leverage it and use it. and like as Tony and I covered there's so many different aspects of what's happening in the overall ecosystem of all these companies that there really is the thing that they're investing in and you can say like if somebody is interested in the future of something you can see what they're going to be investing in their R&D as well as their new emerging technology products and you have the HoloLens from Microsoft you have the Oculus Quest which is coming out which is going to turn a corner and Take virtual reality in a way that just like the Go, talking to different people at the Virtual Reality Strategy Conference, the difference between all the different friction points that you had to use the Gear VR, basically like three or four different steps, and also your main primary communication device is going to drain your battery. It's just too much friction for people to have just easily just popped in and out of VR. But now that the Oculus Go is a dedicated device that you just put on your Go and you're immersed within a virtual reality, The amount of engagement that's been happening with some of the experiences, like from what Dave Cole says of NextVR, it's just like turn the corner and it's like people are using it a lot more. Just anecdotally, from what I've heard from other people within the VR industry, it's a device that's also like catalyzing a lot of new engagement. And so there's also like ARCore and ARKit. Just overall, the ambient computing that is happening, when I talked to Clay Bevor, the vice president of Google, of the AR and VR, like ambient computing in a lot of ways is the intermediary that is going to lead us into more augmentation of virtual and augmented reality. conversational interfaces, the Amazon Echo, the Google Home, the Microsoft Cortana. Being able to just speak and have these interactions with conversational interfaces is a part of this future of immersive computing. It's much more natural and intuitive and that's just got to also be tied into all the AR and VR headsets. So there's the cautionary tale of Keiichi Matsuda's hyperreality and I think that's an important piece to look at because one of the things that Tony is proposing is that perhaps some of these devices are eventually going to be subsidized as some of the smartphones were subsidized by these telecommunications companies that were like, hey we want you to sign a two-year contract and if you sign this to your contract then you'll get the phone for free and I think that as time goes on we'll probably start to see some of that with the augmented reality headsets. It's going to be a while, I think at least two to three years, maybe five years. And the 2025 year is a prediction that I'm making. That's just based upon what I'm seeing. I mean, things could obviously change. We could have like a huge economic downturn and, you know, we could go into a very long winter, but. I think the future of immersive computing is going to reach certainly by 2045. I think that is a safe bet that this technology is such a paradigm shift in so many different ways of spatial computing and immersive computing. But I see so many indicators that immersive entertainment is something that people are hungry for. They're hungry to be deeply present, to be engaged, and to be immersed, and to have some sort of experience where they can interact with it, and participate, and share with their friends, and connect to other people. But overall, all this technology is helping us connect to ourselves, connect to each other, and connect to the world around us. And any platform that has mass audiences are just going to want to have advertising that are trying to promote other experiences or to grow things that you've created. And I think word of mouth can take you so far, but if you really want to launch something that is time-based and time sensitive, then advertising is just going to need to be a thing. just to spread the word about something that is new and it hasn't existed before and so any creative venture whether it's a game or an experience or a movie there's always going to be time-based limitations for some of these things that are limited and that you need to have advertising just to let people know that it's even happening and that if it's an event that only happens once then word of mouth doesn't do you much good because you needed to be there in order to see it and so anyway I think advertising is just going to be a reality that I think last year that I had this conversation with Tony Parisi, I was really resistant and wanting to find a completely different way. But just learning more about the nature of advertising and the nature of getting marketing out and getting people engaged, this seems to be where things are going. And that after being to Facebook F8 and seeing some of the augmented reality ads and to see what's starting to happen with Snapchat as well, There's a viable ecosystem where people are making money and that it's going to lead into more and more immersion and that this is just a single step, but that a lot of the apps like Snap and Facebook facial filters and Instagram, as well as these immersive augmented reality games that are happening on ARKit and ARCore, they're going to be bootstrapping the immersive industry at least for the next year or so. So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listeners-supported podcast, and so I do rely upon your donations in order to continue to bring you this coverage. So, you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
