Google’s mission is “to organize the world’s information and make it universally accessible and useful,” but what happens if there’s a shift from the Information Age to the Experience Age? Google’s Daydream mobile VR headset is part of that answer. It’s Google’s next phase moving beyond the minimum viable VR Cardboard headset where it’s starting to really leverage the Android hardware and software ecosystem to help bring virtual reality to the world at scale.
I had a chance to talk with Andrew Nartker, Product Manager of hardware and platform for Daydream & Google VR, as well as Andrey Doronichev, Group Product Manager, VR Products at Google working on the software, apps, and experiences. We talked about designing mobile VR for extended use, the differences between Daydream and Project Tango, the state of positional tracking on a mobile phone, some of their Google apps that are being developed, using voice as a primary input, how Tilt Brush and Vive development fits into Google VR, how they’re using artificial intelligence to do video stitching with the Jump camera, and adding experiences to Google’s collection of the world’s information. We also talk about some of the favorite experiences in VR, and look to the future of what’s next when it comes to mobile VR and bringing VR to the masses.
LISTEN TO THE VOICES OF VR PODCAST
Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to The Voices of VR Podcast. Today I talk to Andrew Narkur and Andrey Doronovich about the Daydream mobile VR headset that was just announced last week at Google I.O. So Andrew's on the product team for the hardware and platform side of things, and Andre's on the product team for the software apps and experiences. And so I was able to really ask them a lot of different questions about the different hardware, and what's the difference between Daydream and Project Tango, and where they're at in terms of positional tracking, as well as the whole Android ecosystem that they're developing across all the different manufacturers. And I was able to talk to Andre about all the different apps that they're creating, voices and input, their use cases, and the future of other Vive development. So we'll be doing a kind of an overview of Daydream and some of the big announcements that were made last week on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. Today's episode is brought to you by the Virtual World Society. The Virtual World Society wants to use VR to change the world. So they are interested in bridging the gap between communities in need with researchers, with creative communities, as well with community of providers who could help deliver these VR experiences to the communities. If you're interested in getting more involved in virtual reality and want to help make a difference in the world, then sign up at virtualworldsociety.org and start to get more involved. Check out the Virtual World Society booth at the Augmented World Expo, June 1st and 2nd. So this interview happened at Google I.O. last week, and it was actually on the Googleplex campus and the Partnerplex. It happened within the boardroom within Google, so it was kind of this huge table sitting at the end, and I just had a chance to sit down with Andre and Andrew and have them give me an overview of Google's Daydream. So with that, let's go ahead and dive right in.
[00:02:12.863] Andrew Nartker: I'm Andrew Narker. I'm a product management lead for our hardware and platform efforts on the Google VR team.
[00:02:18.968] Andrey Doronichev: I'm Andrey Dorodichev, and I lead the product team for software, apps, and experiences.
[00:02:25.151] Kent Bye: Great, so maybe you guys could describe to me, like, what is Daydream?
[00:02:29.593] Andrew Nartker: Yeah, so Daydream is something we're very excited about, and it really came from our insights that we developed from Cardboard over the last few years. Something that we thought was the right vehicle to get VR to everyone. And so what Daydream does is really build upon a lot of the things that we've learned about from Cardboard to make a high-quality mobile VR platform that we think can scale to provide VR for everyone. It's made up of a couple big things that we'll talk about, but really some of the big pillars of it are providing the right smartphone, specs, and software, powered by our Daydream Ready certification program and our work in Android VR mode, so that the smartphone does all the things you need for high-quality VR, a viewer and controller design that we're actually working on ourselves and also opening to the Android VR ecosystem, and a set of experiences that both Google will provide as well as give developers the right showcase to distribute and monetize their compelling VR applications. All that wraps up into something that we call Daydream that we expect to be the way that a lot of people experience high-quality VR on their mobile device.
[00:03:30.437] Kent Bye: And from the software perspective?
[00:03:32.457] Andrey Doronichev: Well, we have a set of first party applications that we announced yesterday. We're super excited about those. Some of them designed from scratch just for VR. So like YouTube VR was rethought, all of the UI is just right for this kind of input and output. And we think of those apps as a way to showcase great use cases for what VR is good for. Like you think, play movies, it's your ultimate media room, you have all the content that you can consume there on a huge screen, because we have DRM supported in the platform level. Now with YouTube and Street View, you can essentially go visit any place or event. And it's like your teleportation machine. And then we also introduced Google Photos VR, which is combined with Cardboard Camera that was launched earlier. It's a magical way to capture and relive your memories and go back in time, essentially. And then, of course, there's Play Store that is a way for all the ecosystem, which will really provide the bulk of the content for this platform, to distribute the content at scale and monetize it.
[00:04:33.942] Kent Bye: Now, I see that the mobile VR is going to help grow the overall VR ecosystem, and there's going to be both mobile, desktop, and room-scale VR. But where do you see the real sweet spot of mobile VR is?
[00:04:47.269] Andrew Nartker: So if you look at what's happening today, I think we're all super excited about the high-end VR systems that we've seen from folks like Oculus, HTC, and Valve. They're really compelling. They provide that experience like you feel like you're there, and it just is such a magical thing. And from working on cardboard we found this also kind of amazing quality where everybody already has a high-powered mobile computer with them all the time. Billions of people have it. It's something that's already there. And so we expect mobile VR and specifically the daydream kind of smartphone VR system to actually be something that the way that the majority of the world experiences high-quality VR for the foreseeable future. It's going to be a system that's widely accessible, simple, and kind of easy to adopt into your life, and compelling enough to be able to let you do things that you care about. So we expect this smartphone-based mobile VR system to really be this touchpoint that a lot of people now around the world will have access to those glimpses of really high-quality mobile VR experiences today.
[00:05:43.807] Kent Bye: And so maybe you could talk a bit about like this casual gaming perspective because it seems like there's gonna be a little bit of like hopping in and out instead of like having your full attention for long extended periods of time.
[00:05:56.444] Andrey Doronichev: Well, actually, this small bite-size approach is something that we tried and successfully deployed with Cardboard. With Daydream, we're actually focusing on longer sessions, so more immersion and more expressive input. And if you look at that reference design that we showed on slides yesterday, that's what it's all about. We added the head strap, the controller is where you would expect it to be, it's in your hand. And the specs for the phone are designed in a way that the high-performance VR mode is sustained over a longer period of time. Also, if you look at the system software that provides the Daydream Home, the Store, those things are fully designed to be used in VR, including parts of the Android system itself, like notifications. If you get a phone call or incoming notification, you will see it in VR and will be able to act on it. So all those things make very clear indication that we are actually designing this for longer sessions.
[00:06:51.169] Kent Bye: I see. And maybe you could talk a bit about some of the hardware partners that you have. It sounds like you have a lot of different phone manufacturers lined up to be able to produce the hardware in order to be Daydream compatible.
[00:07:00.640] Andrew Nartker: Yeah, so one of the things that we're excited about is just the momentum that Android's been able to provide over the last many years for the varieties of ecosystem partners and actually developers that want to reach many people. You've seen Android VR devices scale around the world to billions of people in every country in every pocket. And so really what you can think of is this Daydream platform that we announced really taking advantage of the way that's already happening with smartphones today. They're really compelling, powerful devices that are becoming more and more accessible, more and more affordable each year as the years go on. And so we're excited that the Android ecosystem is already there. They're already building compelling, high-powered, beautiful devices. And what we're doing is we're working with them closely now to figure out how together they can adopt and build fantastic daydream-ready smartphones. And not just smartphones, but they want to build the companion products that go along these beautiful smartphones. So viewers and controllers, think of those as a common bundle that we're actually starting, but then expecting Android OEMs to take and continue to innovate and improve upon. So all we're doing is giving them the basics so that they know exactly how to start building great experiences. And we expect them to continue to innovate and give consumers the great choice that Android's always provided.
[00:08:13.563] Kent Bye: Maybe you could talk about the process of porting some of these applications that are already out there in a 2D plane and then that translation and try to go beyond just a port and then if you feel like you've been able to transcend and create something that is way better in VR than the 2D.
[00:08:29.322] Andrey Doronichev: Yeah. So this is one of the amazing advantages that Daydream and Android VR mode gives developers is that you might already have a successful Android app. It might be already there. Let's say it's a real estate application or a news app or a media app that is starting to add VR content to it. And that is where Daydream can act as a mode in your app. So essentially, as a user, you drill into your favorite, say, news media application, and then as you discover 360 video embedded in it, you put your phone on the viewer and you continue as a VR session. Now, we do require developers to make sure that their app is fully usable once it's in VR mode. So you cannot just, like, require the user to put it in and out anytime you want to switch a piece of content, for example. That wouldn't be a good experience, and not for Daydream, which is designed for longer VR. But what it also allows developers to do is to make the same APK available both on Daydream Home in VR mode and in 2D mode, which enables the smooth transition and ease of deployment, low-friction deployment of VR features in your existing products. Now, of course, we expect a lot of developers design apps for VR from scratch. In fact, if you look at us, YouTube VR is its own separate app designed from scratch to be used, to be launched and used in VR. And that will happen a lot. We think it's going to be great. And there, portability is also important. And so since a lot of VR experiences existing today are built with Unity and Unreal, we worked very hard to make sure both of those engines are supported from get-go. And we're super excited to have them as our launch partners announced yesterday.
[00:10:07.337] Kent Bye: And so during the VR at Google talk on Thursday morning by Clay, he mentioned that both the Daydream as well as Project Tango are on the same Google VR team as well as in the same building. And he said something like, I'll let you extrapolate whatever that means.
[00:10:24.091] Andrew Nartker: What does that mean? So I actually look at it from the perspective of, you know, what's virtual reality even trying to do? And it's really trying to make you feel present at this place or this moment or this event or with this person. And to do that, it's going to require a lot of technology. It's going to require more than we have today, I think, is kind of the end state. You're going to see people wear devices that are beautiful, that can kind of make you and teleport you to places that are far away, but make you feel there. And so we think of things as, you know, Tango and just other technologies as being these really compelling, interesting application spaces to go explore. They could be explored for a variety of reasons, whether it's VR, but also other applications. You know, Tango has this magical augmented reality window that you can hold up and start to see things that are not really there, but feel there on any smartphone coming forward in the coming years. And so you can start to see these technologies over the coming many years kind of coming together in fun ways, being added to by other technologies, whether it's like advanced input or haptics or just kind of things that we know will be necessary to make you start to feel there and interact with a space. Those will all need to be brought together to provide these compelling experiences that a lot of us get excited about right now though We're extremely focused and excited about just giving this technology to everyone with the daydream platform that we announced this week using the smartphone that you will already have a daydream ready device with Viewer and controller that we imagine a lot of people have access to and can widely adopt. It's simple. It's intuitive. It's And with something that they're already doing with their daily lives, which is using their phone to download apps, transact, and do stuff that they care about, we see that as a really compelling kind of first step to getting a lot of people excited and actually aware of what VR can do for them.
[00:12:05.808] Kent Bye: There's been a lot of discussion within the larger tech industry that we're in the midst of this huge shift from the information age into the experiential age. And we can just look at the Z generation and their use of Snapchat and people who are a little bit older being utterly confused and baffled by this new experiential age. And I feel like a lot of the virtual reality and augmented reality is sort of leading us into this transition. So from your perspective here at Google, working on the software, Google's traditionally, their mission has been to give access to the world's information and knowledge. But as we're going through this paradigm shift into our primary attention being focusing on experiences, where do you see Daydream fitting into the whole future of Google?
[00:12:50.523] Andrey Doronichev: Yeah, well, you sort of answered your question. That's exactly my answer. Information is a broad word. It starts with text, probably talking about something and probably showing a picture and then probably capturing a video and replaying the video. And now you can re-experience that or you can experience something that is happening far, far away. That is the ultimate form of information. And Google has been always about organizing world's information, making it accessible and useful. So organizing world's experiences sounds like the most obvious next step and that's what we're doing. That's why we're excited about VR and AR for that matter. We think the information will be more rich and more experiential as we progress on our path to capture and replay reality and also construct new virtual realities with all the tools that we're creating. And Daydream is just incredible next step on this path. As Andrew just mentioned, we think there will be many more things, like technology will evolve and many more things that will be added to make it even more rich and even more experiential. But certainly we think it fits into the core Google strategy, core Google mission to organize world's information.
[00:14:00.107] Kent Bye: Do you have any insights on that in terms of the hardware perspective, that there are specific things that are kind of driving that?
[00:14:06.275] Andrew Nartker: Yeah, I think, you know, one of the, I'll kind of answer it as one of the things that I've thought a lot about and I've kind of realized about myself over the last few years is that I do like collect experiences. So I've got really young kids, you know, every single thing that they do, you know, their first steps and words and all these things, like I'm capturing them on all of my devices. I'm showing them to my parents, their grandparents. And we're just really, you know, getting collectively interested in just their growth and their kind of aging. But we only have this like approximation of it. Like we don't really have the experience to go back to. And so I see this, you know, this AR and VR kind of tech ecosystem as something that you can already imagine going forward. And my son just actually learned how to ride a bike last month. That first moment of actually taking off your training wheels and seeing him so proud and so happy and just having it just as a flat photo, that's going to be kind of a thing of the past. Going forward, you're actually going to be able to go back, feel like you're there with him, actually re-experience that really important family moment again and again and again. And the technology is already coming together to be able to already imagine that future. It's going to happen sooner than we all think. It's going to require things that we're not all used to and things that are going to be hard to do. But you can already imagine how a lot of these pieces are going to fit together to let you go back and re-experience things and start to collect important places and events around human history and start to go back and re-imagine those and re-experience those. So I'm super excited about this. I think it's all in front of us. I think all of us have a lot of work to do. But it's something that Google as a company and the industry is all collectively excited about.
[00:15:37.676] Kent Bye: At the keynote, the CEO of Google said that the AI revolution is just starting. And there was an announcement of Google Assistant. And I imagine, in talking to some people on the Google VR team, that there are different ways that you're using machine learning within VR. So maybe you could talk a bit about how do you see the future of VR and AI coming together?
[00:15:59.348] Andrey Doronichev: Well, AI is a vast technology that will improve many things that we do across the board, including VR. And, you know, like we already applying some of those techniques. So if you think about, for example, jump camera that we have. the computational photography and like very smart deep learning algorithms that are used to stitch those feeds from cameras it's a groundbreaking technology like you really get from 16 cameras filming together you get a single perfectly stitched stereoscopic 360 degrees video that doesn't have any stitches that's magnificent you know you look around most of the 360 videos that you see you'll see a person walking around but then at some point the person just distorts and turns into like half person and then reappears again. Now that doesn't happen there. That is because of the power of cloud computing and processing with computational photography and vision algorithms that we're using. So this is just one example, but I think many more things will come up. You know, I saw a very interesting experiment of a deep dream distortion on a video, on a VR video. which looks astonishing. It looks so cool and trippy video around you. And this is another example. This is probably like a cellular technology demo, but that's pretty cool. And so anywhere between those two, from deep, low-level technology of stitching camera feeds all the way to artistic representation, I think AI can improve things in many ways.
[00:17:23.222] Kent Bye: I think another one just to follow on that is that you know with the announcement of Google Home and being able to talk to Google Assistant but primarily through voice and so I think most of our communication actually comes through body language so I can imagine a time when there's AI assistants that are actually embodied and you can start to see a little bit more body language but yet I think they're gonna have to get a lot more motion tracked input control in order to kind of run the algorithms on that in order to kind of model some subtle body language that goes beyond just voice.
[00:17:55.632] Andrey Doronichev: That is a great idea, by the way. And speaking of voice, specifically, you know, like YouTube app that we launched relies on voice a lot, and it's magical. You just put on HMD, and you talk to the app, and it searches for whatever thing you need, and it's right there. You can experience it immediately. It's pretty incredible. But I love your idea of using AI to simulate body language. That would be pretty cool.
[00:18:19.070] Kent Bye: And I think there's been a little bit of confusion of the differences between Google VR's Daydream and Project Tango. Are Project Tango phones going to be Daydream ready?
[00:18:30.378] Andrew Nartker: So right now, we've kept them very distinct. And so they're unique technologies. And a lot of the reasons why is because of coming back to the thing we talked about. We want this technology to be widely adopted by everyone right now. And that's basing it on the same technology and the same smartphone that you kind of have already. It's this flagship, you know, high-end smartphone that's powerful and does what you need. And then making sure that we can just make the certain things that we need for Daydream available on today's phones or coming forward on these Daydream Ready phones. And what that means is that making sure that the displays are a little bit better, so this low persistence display spec, making sure that the processors are a little bit more powerful and able to sustain VR performance over time, and then making sure you have the right sensor suite right now to do this seated, kind of standing VR experience that we think will widely scale to a lot of people. Tango has a different kind of method of making sure that they're putting forth their use cases and coming forth in current technology today, and we're excited about where they're going to. The one most exciting thing I've seen from them lately is just the variety of interactions that you can have when you have this like magical augmented reality window in your hand. And so over time those VR and AR kind of application spaces will seem more and more alike, but right now they're so distinct and so unique that it's fun to see how both of them will, I think, grow.
[00:19:44.582] Kent Bye: Well, one of the things that I saw here at Google I-O, which was really compelling, which was a demo of the area learning capabilities of Project Tango, which is essentially like positional tracking within a mobile headset. However, I don't know if the latency would be good enough in order to drive a VR experience. But what are some of the big challenges moving forward with positional tracking? Is this something that you feel like could be achieved with inside-out depth sensor cameras? Or do you really need an external stabilized sensor?
[00:20:10.343] Andrew Nartker: Yeah, I think all of us are still excited and exploring this space and we found that just positional head tracking is not sufficient to have that experience where you can actually move and interact through a large virtual space. You actually also need, as you kind of alluded to, you need controllers that actually follow around with you that are also tracked. You need a really robust interaction mechanism and input scheme and actually apps that are actually purposely developed for this type of like fully interactive space. it's not really sufficient just to have one piece of that puzzle. You really need all of it to have a really compelling experience that makes you believe like you're there. That's a big challenge to like take on that full space of making sure that you believe that you're there. That's something that you see from a lot of these great VR systems today. It's exciting, it works, and it's technology that I dream of over the next coming years to be more widely available to everybody.
[00:20:59.324] Kent Bye: And so just being kind of familiar with different photogrammetry techniques, I could imagine a time using this vast repository of Google Street View images that you'd be able to essentially reconstruct a 3D mesh image of the entire Earth's streets. So is this something that you, with the Google Street View, is this something that is actually going beyond photospheres and doing 3D models?
[00:21:23.193] Andrey Doronichev: Well, I'm extremely excited about showing the world the real world and enabling the world to see the world. And Street View is the first step in that direction. Of course, the 360 mono photosphere is not really the most perfect VR you can imagine. And as you said, you can imagine stereoscopic, you can imagine full 3D reconstruction of places and we are working on many ways we can improve our data set and or generate more detailed scenes out of data sets that we have. I'm certainly confident over time we will be able to put users in places far far away through their VR devices starting with Daydream.
[00:22:04.638] Kent Bye: Well, I know that Google has one experience on the Vive, which is Tilt Brush. Can you talk a bit about how does Tilt Brush fit into the overall Google VR ecosystem?
[00:22:14.182] Andrey Doronichev: Oh, absolutely. Yeah. So Tilt Brush is one of the VR applications on my team that we launched publicly. We are in love with this one. Obviously, extremely compelling experience, not only because it's fun, but also because it is a creation experience, not just consumption. And, you know, I'm very proud and happy to discover a case where VR can be used for creation. I think creativity is going to be incredibly important field. So Tilt Brush is our way to explore full interaction with six DoF controllers in VR and also just understand the space. We realized that the vastness of possibilities that are unexplored right now out there in VR and AR cannot be covered with just one specific focus on one platform. So, you know, as an application team, we are experimenting with all of them. Daydream Labs announced yesterday tries their experiences on pretty much any hardware available. So those experiences that are compelling, we try to release. So Tilt Brush is a great example of that. It's only released on PC, but that's where it belongs. And we continue investing in it and exploring it.
[00:23:30.107] Kent Bye: Well, I hope to see a Vive room scale version of Google Earth. That would be amazing to be able to walk around. That's a great idea. So maybe we could shift gears and talk a little bit about your own personal experiences in VR. I'm really curious about some of your favorite memories of being in VR.
[00:23:47.419] Andrew Nartker: So yeah, so I've looked at VR for a couple years now and actually got involved in the original cardboard efforts. And the thing that really made it compelling to me was actually seeing high end, high quality VR. Actually, we had researched, you know, telepresence or synchronous communication of people in different spaces in VR very early on. And it was this really magical thing. But you kind of, you know, look at it and you see it, it's going to take a lot of technology, a lot of cost, a lot of friction to really become this thing that widely gets to everyone. And so in coming forth with Cardboard as our first effort, that's what really excited me because it made it more obvious that you didn't need all of that technology. And I think in a lot of people's minds they got tripped up on just how much VR required from people and from the technology around you. I'm excited that you can keep it simple, you can keep it short, My mom's even tried cardboard and she actually even now knows what VR is used for. It's used for, you know, just going and peeking into other places and kind of feeling like you're someplace else. And I think that was a big hurdle for the industry, you know, three or four years ago when a lot of us didn't know how to tell the world how to think of VR, how they could actually adopt it in their life. So for me, it's just, you know, making it simple, making it something that a lot of people can now understand how to see like their use case in is the thing that just gets me so excited about like the way things are going today.
[00:25:00.827] Andrey Doronichev: What about you? My personal experiences? Well, I have four or five key points across different platforms that led me to where I am right now. Number one moment, which I remember, which was groundbreaking, was just the very first time I tried an old prototype by Valve, the Tag Room, that they were showing around way before the whole ecosystem started. And that was the moment when I made a first step in VR, and I was like, oh my God, I'm surrounded by these blocks. They don't look like anything real, but they're totally real, and I'm there. And I'm sure a lot of people heard or tried this experience, like a very old first demo. My second amazing experience was with Cardboard when we went to a school for the first time to pilot the very first prototype of Expeditions. And the moment 20 kids got the cardboards and looked into them for the first time and the teacher started the expedition and this, whoa, in the room, that was something that, oh my God, I cannot forget this moment. That was incredible. And that really showed that, hey, it's not really about like, you know, high power GPU. There are many things to amazing VR experience that create this compelling, amazing memories and moments. My third moment was probably when I first tried Oculus Toy Box demo, and the moment when the guy who was demoing it handled me the Zippo lighter. I'm sure a lot of people tried that too, and oh my God, it was incredible. Amazing job. Just this fact that you can interact with a person, just to have this moment of passing something, I think it's going to be defining a lot of experiences going forward, that people interact and do things together in VR. Next one I would say was Tilt Brush as we launched it and just spending an hour and a half without realizing that modeling out the whole family room in my house with flat paint. and deciding whether or not I should put a chair and a table where I wanted to put them and then just taking off the headset and just like, oh my god, I spent there like an hour and a half and I didn't notice that and I had so much fun. That was the first time I spent a lot of time in VR having fun and completely forgetting about, like, not as a tester, not as a developer, but really doing things because VR is so amazing. And then I think one of the last moments, the most recent moments that I had was with the latest YouTube VR app on Daydream. And I was testing the app, just like previewing, pretty rough state still on the Daydream dev kit. And again, I lost track of time. And an hour later, and God knows how many cat videos later, I was like, wow, this thing works. Controller is a great thing to do. Oh my god, having head and hand acting separately is so useful. So yeah, I guess those were like the key moments that I remember. Obviously many more things, amazing things I've experienced, but those are the things I wanted to highlight.
[00:27:57.500] Kent Bye: Yeah, time dilation is a thing. I think that the most extreme example I heard was from Carl Krantz from SVVR where he spent 12 full hours in VR and you only thought that it was about three but no breaks no restroom breaks or anything So I think time dilation actually is going to be a big deal, especially if you're going for longer sessions within mobile VR So something to be aware of for everybody but just to kind of wrap things up here What do you see is kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:28:26.063] Andrew Nartker: Yeah, I actually am kind of excited to think of VR as something that maybe is on the same trajectory that headphones and sound was on many, many, many years ago, where it might not have been obvious or clear to people that you could record a moment in human history audially and actually have that be available for everyone to re-experience forever and carry around with them. And that was kind of an odd thing. And now you look forward and you see every single person with this like computer in their pocket, headphones always with them. They're able to go places and kind of go back to these moments and just listen to them and re-experience them again. It's a common thing. Like everybody's got that idea. And I think you'll see over the coming, you know, decades that AR and VR will be that to us. It'll be something that we carry these memories around with us. And anytime we want, we can go back and make use of them or just reimagine them. So I'm excited about that. I think it's going to happen and it's going to be a lot of fun.
[00:29:18.126] Andrey Doronichev: Yeah, for me, VR is a way to democratize access to experiences and places and knowledge and things that otherwise it's really hard to access at scale. Right. So I would love to go inside pyramids. I would love to go to space. In fact, the biggest reason why I was focusing on my career early on was because I really, really wanted to end up in a place which lets me go to space at some point. And I feel like I'm finally there. I actually have a direct impact. So even if I don't go there physically, I will most likely be able to go there virtually in the very foreseeable future, even to Mars, in fact. So to me, just letting anyone in the world to experience the world in its beauty and its fullness and really be there, really be present, such an incredible promise. And we're so close to making it possible. It's astonishing.
[00:30:14.488] Kent Bye: Anything else left unsaid that you'd like to say? I think that was a great session.
[00:30:18.011] Andrew Nartker: Thank you so much. Yeah, thanks so much, Kent. Really appreciate it.
[00:30:21.014] Kent Bye: Awesome. Yeah, thank you. And so that was Andrew Narker, product management lead on the hardware and platform efforts, as well as Andrei Deratochev, who is on the product team for the software apps and experiences. So a lot of different takeaways from this interview. First of all, Google is definitely taking the approach that they have been taking with Android. And so the good side about that is that there's just going to be a multitude of different companies producing different iterations of Daydream VR headsets. The downside is that with the Android market, we see a little bit of fragmentation. So it'll be interesting to see if that even within Android and on the Daydream platform, if there's some sort of slight variations that start to happen between different operating systems and each of the different manufacturers producing different types of either headsets or controllers that are specific to the different phones. This is something that I think that will develop over time. But overall, this is a very positive move towards creating this open and vibrant ecosystem that definitely within the next five, 10 years will be one of the biggest and perhaps most dominant platforms for virtual reality and bringing it to the masses. Also, it was pretty clear from this interview that at this point, Daydream and Project Tango are pretty separate. I don't think we're going to see Project Tango phones that are fully Daydream enabled with the different low persistent screens and everything else that is required. And as well as the Daydream phones are not necessarily going to have the Project Tango 3D depth sensor cameras are going to be required to do all the different computer vision and area learning and all sorts of other augmented reality features that are going to be on Project Tango. But that said, they're in the same team in the same building. So in the long run, I think we'll start to see more of a fusion to have a mobile headset that will do both AR and VR. But at this time, in the short run, at least, I think the technology stacks are different enough that they're keeping them separate so as to not limit how far VR could go, because they really do need some extra hardware in order to really enable Project Tango. It was also interesting to hear that voice, I think, is going to be one of the primary user interfaces for Daydream VR. Andre said he's navigating around in YouTube and able to spend lots of time just speaking your text and having it translate and work really well enough for it to just be faster and more easier than trying to enter text through VR, which is a very suboptimal experience. I'll be talking a little bit more about the 3DOF nature of the controllers when I talk to the lead of the software SDK of Daydream later in this week. Also, it was kind of interesting to hear that they are going for something that's beyond just a little bite-sized experiences like the Google Cardboard, but it really intended to create and cultivate a deeper sense of presence and more extended use of mobile VR. And I think having a controller there is going to make a big difference in that way. So I did get confirmation from Andre that they are using some sort of AI machine learning to be able to stitch video from the jump camera. I know that when I interviewed Jim Goldoldek from GoPro in episode 281, he did imply that the jump camera did have some secret sauce that was going on in the back end and the cloud processing to be able to process the video, but Now we got a little bit more confirmation that they're taking these cameras that are different perspectives and being able to you know kind of reconstruct either a point cloud or do some digital light field calculations to be able to reconstruct a 3d mesh and then from that kind of output a rendered video. So we still don't have any very specific details as to what's exactly happening there, but what that implies is that you just take Street View footage from photospheres that they've taken from multiple perspectives all around the world from the Google Street View initiative, and you combine that with other potential AI and photogrammetry techniques and 3D reconstruction techniques, and it's not hard to imagine that Google probably already has the capability to be able to create a 3d mesh of the entire earth and globe and perhaps maybe even take satellite photos and look at shadows and be able to determine the height the different objects and multiple perspectives of satellite photos even and to start to reconstruct the entire earth So also the other thing to kind of throw into the mix is that they do have Tilt Brush and it's a Vive experience. And so Tilt Brush is part of Andre's team of software apps and experiences. And so Google has been shown to release things for the Vive. In fact, I'm pretty sure that when talks about the original group of people that went to go and develop for the HTC Vive when it was announced at GDC, Google did have a team that was sent there very early to start to be working on different projects on the Vive. So they've been working and collaborating with HTC and Valve for a long time. In fact, if you look at one of the lists of the partners of the phones for Daydream-ready phones, HTC is on there. So they already have an existing relationship through Android with HTC, and they're going to have Daydream-ready phones, which From HTC's perspective, it's kind of interesting to think about having both Vive and the kind of mobile VR available. So all these things taken together, I would love to see a room scale experience of Google Street View or Google Earth to be able to walk around anywhere on the planet. And I think that's probably one of the biggest things that I'm looking forward to in terms of what Google is capable of doing with all their AI machine learning, this huge data set that they've got and just their Capability to be able to create things like Google Maps and Google Earth already where you know that's a huge data set that they have but they figured out ways to kind of Give you just exactly what you need and kind of stream it down and so for them to be able to do that and break down these 3d meshes in a way that kind of are sent down to you and you maybe can see an entire scene and figure out all the occlusion issues and everything else. But it's pretty mind blowing to think about, you know, in the future, it's going to happen here within the next year or two or three, that this is going to be possible to go anywhere on the planet and to walk around and just explore the earth from the comforts of your home in virtual reality. So, that's the big takeaways that I got from this interview. Super honored and humbled to be able to go to Google I.O. and to be able to get some access to some of these different people within the Google VR team. And, you know, just a note about the overall openness that I saw at Google I.O. You know, they are bringing together over 7,000 developers and sharing a lot of information, giving the developers access to these leaders in AI, machine learning, and VR, and just creating this open and fun atmosphere. So it'll be interesting to see over the long run how Google continues to grow and evolve. At the beginning here, I think Google is very interested in scale, and so they're not interested in doing something that's maybe the best quality of VR, but only reaches a fraction of the number of people. They really want to reach the most people possible. So continued efforts in a lot of their mobile VR and really helping take VR to the mainstream in a big, big way where a lot of people's first time experiences will likely be on Daydream. So to wrap things up here, if you do enjoy the Voices of VR podcast, please do spread the word, tell your friends about it. You can follow me at Kent Byeon Twitter or on Snapchat, where I've been doing a little bit more kind of daily video blogs and snaps about my process of going around the country and doing this coverage. In fact, tomorrow I'm going to be going to the Artificial Intelligence Summit that's being held by the White House, and that's going to be in Seattle. So I'll be up there and maybe doing some interviews or at least covering what's happening with public policy and artificial intelligence as all these technologies are coming into our lives and, you know, perhaps a little bit faster than our economy or laws are able to deal with. So I'll be kind of looking at that cross section as well. So check out Snapchat for some of the latest impressions from that tomorrow. And finally, you can also become a donor to the Voices of VR podcast at Patreon.com slash Voices of VR.
