People were waiting in lines for over an hour to see Pearl both at SIGGRAPH and VRLA, which is one of the best narrative VR experiences that I’ve seen so far. The central character in the story is a car named “Pearl” who gives selfless service to her father and daughter musician owners. The experience traces Pearl’s selfless service spanning a wide range of emotional memories doing mundane chores, road trip adventures, and key turning points in their lives. It takes inspiration from Shel Silverstein’s The Giving Tree, and it proved to be a powerful, moving, and emotional experience for many people watching it was SIGGRAPH and VRLA.
Pearl was directed by Patrick Osborne, who won an Academy Award for Disney’s animated short film Feast. It was produced as a part of Google’s Spotlight Stories using their custom VR animation software codenamed Moxie that’s being developed within Google’s Advanced Technologies and Projects (aka ATAP).
I had a chance to talk with Patrick Osborne at SIGGRAPH, where they premiered the interactive version of Pearl showing on the Vive. It originally premiered as a 360-video at Tribeca Film Festival, and was eventually released on YouTube during Google I/O. I have a chance to talk to Patrick about pacing and editing in VR, making a folk music video in VR, and the inspiration for the story.
LISTEN TO THE VOICES OF VR PODCAST
You can watch Pearl on YouTube now, and I’d recommend waiting for whenever the more interactive version is released on the Vive.
The YouTube channel for Google’s Spotlight Stories has their previous 360 video experiments, and they’re releasing some introductory videos including how to watch a 360 video on a phone without even using a VR headset.
In fact, I’d say that their latest Buggy Night release serves as a training video designed to help people learn how to watch 360 videos primarily on a mobile phone.
Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye and welcome to The Voices of VR Podcast. So when I was at SIGGRAPH a couple of weeks ago, there was a special section within the VR village for storytelling. And you would go into this area where there's a number of different booths and people showing experiences. And when I went in there, there was a really long line for people waiting to see this experience called Pearl, which was produced by Google Spotlight Stories, which is a part of Google's advanced technology and projects. So Pearl was definitely one of the most emotional and well-crafted VR narratives that I've seen. I had a chance to talk to the director of Pearl, Patrick Osborne, and talk a bit about his process and inspiration and some of his insights on editing and a little bit more information about the software that Google is producing in order to help with these types of stories called Google Moxie. So that's what we'll be covering on today's episode of the Voices of VR podcast. But first, A quick word from our sponsor. Today's episode is sponsored by the Intel Core i7 processor. VR really forced me to buy my first high-end gaming PC, and so Intel asked me to come talk about my process. So my philosophy was to get the absolute best parts on everything. Because I really don't want to have to worry about replacing components once the 2nd gen headsets come out, and the VR min specs will inevitably go up at some point. So I did rigorous research online, looked at all the benchmarks, online reviews, and what I found was that the best CPU was the Intel Core i7 processor. But don't take my word for it. Go do your own research and I think what you'll find is that the i7 really is the best option that's out there. So this interview with Patrick Osborne happened at the SIGGRAPH conference happening from July 24th to 28th in Anaheim, California. So with that, let's go ahead and dive right in.
[00:02:03.113] Patrick Osborne: I'm Patrick Osborne. I directed Pearl for Google Spotlight Stories. It's my first dive into anything VR.
[00:02:10.262] Kent Bye: Great. So tell me a bit about how this project came about.
[00:02:13.283] Patrick Osborne: I met Jan Pinkava and Karen Duflo, who are the creative directors of the ATAP group at Google. I met them at Annecy Animation Festival. I was there with Feast, which was my short film for Disney. And they said, think about doing a story for 360 then. This was two and a half years ago. So we weren't really sure that the headsets were going to be around in a way that we could do it. yet by the time we were done, but it turns out we could. So at first it was just for mobile. It was going to be for Google Cardboard and that sort of thing. But then luckily we were able to port it over to the Vive as well, which I think is a pretty cool way to watch it.
[00:02:50.003] Kent Bye: Yeah, I just had a chance to see it here at SIGGRAPH and I have to say it's probably one of the better narrative experiences that I've seen in VR. And the reason why I say that is because you're kind of using a car as the central mechanism throughout the story in terms of all the memories that are happening inside the car. And so it just felt like if I was a car, this is the perspective that I would be able to see of somebody's life.
[00:03:12.930] Patrick Osborne: Yeah, I mean, the origin of some of that is that I kind of thought it would be like an automotive version of The Giving Tree, the Shel Silverstein book. I mean, you always try to combine inspirations to make a film. So it's something between that, passing on car, passing on talent, that combination of like an object and also like a passion for music. thought that would be kind of interesting to play with. So you start thinking about those things and then being in 360, we're cheating the ability to move by making it in a car because I can actually change lighting and environment and weather and mood without disorienting the audience, which is the cheat that the car makes possible. Everybody knows where the steering wheel is once they sit down. If you don't change that, the cuts don't feel as jarring. You don't have to re-figure out where you are on any cut. We do take our time with the first couple so the first scene you're allowed to sit in and just kind of look around for a while before the film even starts. Once you look in the right direction it'll start but it'll let you like just hang out for about 30 seconds and then once it cuts we made sure that that second shot was long enough that you had time to kind of look around again and then the second cut you kind of are on board now and you can cut faster. So in VR now, I think each experience is teaching the audience how to watch it. And if people aren't careful about those first few moments when someone's in it, it can be a little bit distracting and weird. Because people just don't know how to watch this stuff yet. There's no equivalent of sitting in a theater, the lights go out, the curtain opens, the trailer plays. We don't have that stuff. So you need to teach people how to watch your film for every film.
[00:04:46.832] Kent Bye: Yeah, I think that the thing that I think is probably one of the more interesting things about Pearl is that the pacing feels a little bit more akin to a traditional film, whereas in most 360 films that I've seen, I think most people have to take a little bit slower pacing. But I think there's something with the car, like you said, that allows you to keep that orientation and allow you to kind of cut a lot faster than other VR narrative experiences that I've seen.
[00:05:11.107] Patrick Osborne: It is still slow. We did a version where I went and recorded my own camera, like I was documenting the film and cut that to the song. It's like a version for theaters, for film festivals and things like that. And that felt really slow if we left it at the pace here, if you're just watching in a theater. So this has 38 cuts and the theatrical version has 75 because it just felt sluggish. And if you put that amount of cuts into the VR version, it feels crazy fast and people just I feel like they're missing everything. So part of that is just the comfortability people have with VR or not yet. And I'm sure our language about how to tell stories will evolve with people's understanding of the medium. And right now, it's also interesting when you see an enthusiast watch it versus my parents. The amount of looking around an enthusiast does is way more because they're used to these experiences now and kind of get it. And my mom and dad will sit down and just stare straight ahead for a while. not expect to have to look around, and you have to teach that, too. It's so new that we'll land in a place where people like to be watching things, and I'm just happy to make something that's a part of figuring that out.
[00:06:22.668] Kent Bye: Yeah, and there's even a sunroof where I, at one point, kind of stood up and stuck my head through just to see the perspective change, which I imagine when the mobile versions, you weren't able to really do that, but in the Vive, you're able to really have that.
[00:06:35.451] Patrick Osborne: No translational tracking on the phones. So you can't do that. But it's cool to be able to sit in the backseat on the Vive, and you can have the kids version. We did have to make sure all the geometry was there, because in the beginning, there was no outside of the car. And then once we put it on Vive and realized, oh, people are going to stand up, we've got to make the whole car now, make the wheels. We just didn't have wheels. Stuff like that did need to be adjusted once you could move around the space.
[00:06:59.590] Kent Bye: So what were some of the big takeaways from doing one of your first narrative experiences in VR?
[00:07:06.273] Patrick Osborne: I mean it's really fun to be able to make something and then sit in it. To actually create a film and then be able to walk around in the environment afterwards is amazing. I wish I could do that for everything I've ever done. I do think we need authoring tools in VR, the actual process of storyboarding and coming up with story ideas and pitching and planning and like the fast iteration that you can do with Photoshop and animation tools doesn't exist. So it becomes a very cumbersome process of pasting drawings around a sphere, you know, to do that and the point of story is to come up with things quickly and be wrong quickly and a slow process doesn't help with that. Authoring tools making modeling tools for VR all that stuff is gonna. I'm sure it's coming I've played with a few already, but it'll definitely Make everything better, and that's what I want make creating easier
[00:07:58.951] Kent Bye: Speaking of tools, can you talk a bit about using Google's Moxie in order to create this experience?
[00:08:04.194] Patrick Osborne: Yeah, it's nice to... I like to break tools with Feast and Paper Man at Disney. I was into making things that were not typical in the CG look, which meant breaking the tools because the tools are made for the typical look. And having an engine that is your own, like Moxie, is nice because we could mess with it in any way. So I had a request about how this edge of this object looks. For me, visuals are all about value structure and edges. It's the same way I think about painting. And edge control doesn't really exist in regular CG pipelines. So we had to create a version of that. And because we're working with our own engine, we could hack the render buffer layer stream to make that work for us. And it's not really hacking when it's your own thing, I guess. It's just breaking it and fixing it. But if you don't have your own engine, sometimes you can't do that stuff.
[00:08:52.929] Kent Bye: And so, talk a bit about the art style that you used in terms of, you know, instead of going towards something that's more photorealistic, you're kind of safely on the other side of the uncanny valley and have a very distinct art style.
[00:09:04.625] Patrick Osborne: Yeah, I think the more abstract, visually abstract you make animation, the easier it is for people to put themselves into it. It travels better, it's easier for all audiences to kind of imagine their own life while looking at it. It's the same thing when you look at kind of an abstracted painting or a work of art. You can marvel in the details of something and that's a thing that you're looking at, but it's not yourself. And when it's simple, I feel like you put yourself into it more. Also, needed to run on a mobile phone at 60 frames a second, required some kind of look that was simplified, and I wanted to be direct about that look and not let the technology drive it. So we just decided on a look and made that work.
[00:09:46.634] Kent Bye: And can you talk about if there's any gaze-directed triggers or if it's just a purely narrative experience that's the same every time?
[00:09:53.337] Patrick Osborne: There are triggers and gaze-based things. The film is between like five and a half minutes and seven and a half minutes. It's about two minutes of fluctuation depending on where you're looking. Most of that happens in between verses. So the verses kind of carry because the music is there and we couldn't really mess with the timing of the music or the beat that much. But in between verses you'll see kind of pauses in the action and it waits for you to see certain moments for a certain amount of time. It doesn't wait forever. It'll wait for 10, 15 seconds sometimes. The very beginning waits about 30 seconds if you just want to look around in the other places for a while. Stuff like that. Some of the other Spotlight stories are very trigger-heavy, and this one has like maybe 10, so not a huge amount.
[00:10:35.818] Kent Bye: And so is that something that you're more interested in the narrative, linear narrative, or are you thinking about doing some sort of like branching type of story in the future in VR?
[00:10:44.083] Patrick Osborne: Because this was a musical, it required that. I think the branching thing is interesting, but it becomes exponentially more expensive for things that an audience might not see. And you really want to put all of your effort and time into making things that people see. So branching stories have always been a little bit cumbersome because it cuts down on the amount you're able to do or the quality you're able to get and the amount of time. You know, animation is time consuming and it all just comes down to people's hours of the day and how many hours you get to spend on this film and how you get to distribute that is what your budget is. So trying to get it all on screen is the goal. That's why choose-your-own-adventure type movies never really take off because it's so expensive to make each part and then the audience doesn't see it and whatever. I think on the directing side having interaction is great. That's another tool you can use to tell the story and be interesting and that's totally fine but on a budget side it becomes difficult.
[00:11:40.984] Kent Bye: So what do you think are some of the biggest open questions that are kind of driving your exploration into VR forward?
[00:11:48.313] Patrick Osborne: Well, there's the question of accessibility. It's nice that Pearl is available for cardboard and on every mobile phone. That means people can watch a version of it. Because for a while, VR on these headsets that are high-end is going to be a little bit of a thing that you watch when you're at your friend's barbecue and he happens to have one. You know, it's definitely like the vibe in my house becomes like this weird side entertainment thing that people do when we're having people over for dinner. People always want to try it, but no one else has one, so only one person at a time can do it. So it's kind of like conferences have lines for that reason. So I think until it's out there, experiences that work well on mobile devices is probably where most people will see it. But I can't wait until it's like so known that people have rooms dedicated in their house to this sort of thing. I think it will happen, it just needs to come down in price and be a little less cumbersome to set up. I haven't built a computer like I had to do for this since high school.
[00:12:43.528] Kent Bye: Great, so what's next for you in VR then?
[00:12:46.396] Patrick Osborne: You know, experimenting on the idea of doing theater with live actors, an audience that is in a headset, but you know, the actors are actually real people. I think that would be kind of fun. So trying to write stuff that works for that. So yeah, we'll see.
[00:13:01.389] Kent Bye: I see sort of like an audience full of like 50 people in VR, but maybe like a handful of actors in VR that are like different monsters or characters.
[00:13:10.148] Patrick Osborne: Yeah, I mean, you can costume them in all kinds of cool ways. It's really interesting. And mocap technology is cheap enough. And you can use the Vive stuff to actually do it. So playing with that, I think, is really fun. And I'm going to be doing some of that.
[00:13:24.034] Kent Bye: For you, what's one of the most memorable narrative experiences in VR that you've seen?
[00:13:28.236] Patrick Osborne: Let's see. A lot of them are semi-narrative, semi-games, right? So the Star Wars thing the other day was pretty amazing that ILM did, just because it's like your childhood. It's exactly what you want. Wanted that to be you know to see r2d2 next to you plays on a bunch of nostalgia. That's pretty amazing I liked trying to think of all I've downloaded everything so I Loved the Apollo thing because what I thought about that is like my dad would sit in here and do this You know it crosses generational lines that I think VR needs to do that now you need to have some experiences that you can show if you have like five or six things on your headset and a person comes over you're like That one would be for them. That one would be for them. That you know that the type of person that they are, that they would think that this experience is cool. To have a variety is great. And the Apollo thing was cool because I sat through the whole thing. I learned something. You also kind of get to be there. Graphics, you know, will get better. But I really like that one.
[00:14:26.103] Kent Bye: And finally, what do you see as kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:14:33.247] Patrick Osborne: It's hard to say. I mean, the whole Ready Player One thing would be awesome. The whole convincing, comfortable, haptic side of stuff is going to be really interesting soon. We're seeing some experiments with that here at this conference. I think that'll be cool. I do think the ability for AR and glasses and headsets that are very much fashionable and don't look any different than regular things like shrinking down the tech so you can't really notice it is super important for the future of it.
[00:15:03.524] Kent Bye: Awesome.
[00:15:03.824] Patrick Osborne: Well, thank you so much. Thank you. Yeah, no problem.
[00:15:06.468] Kent Bye: So that was Patrick Osborne. He is the director of Pearl, which is a Google Spotlight story that premiered at Tribeca Film Festival. And I had a chance to take a look at it and watch the new, more interactive Vive version that was premiering at the SIGGRAPH conference. So I have a number of different takeaways from this interview is that first of all This is highly recommended if you are doing anything within storytelling within VR I think this is one of the most well executed experiences that I've been able to see has a beautiful Story arc very emotional and I don't know a lot of people that I talked to at different conferences there's at least three or four people that told me that it brought them to tears and You know, you can watch it within YouTube and kind of pan around. Uh, but I think there's just something a lot different when you're actually in a vibe experience. It just gives your brain a little bit more sense of you're actually being there. And it was kind of like shuffling through somebody's memories of their life and. It made me just think about my own car and the different experiences and life that my car has had. And Patrick referenced The Giving Tree, which is this Shel Silverstein book from 1964. Essentially, it's a children's book and it's about this boy in a tree and he has this relationship with the tree where he's playing with the tree and then he starts asking the tree for different things. Whether it's to play with the tree or to climb in the branches or as the boy grows older he's essentially getting apples from the tree and then eventually he asks for the branches to be able to build a house and then the whole tree to build a boat and you know at each of these different chapters the tree is kind of selflessly giving itself to the boy and after the tree gives something to the boy and the tree is essentially happy and And so you kind of feel this selfless service type of energy of both the car within the story but a lot of the actual dynamic between the father and his daughter and music plays a huge part of this experience as well. It's definitely like a music video and It really carries that extra level of emotion that I think that a number of different early cinematic VR experiences that I've seen, especially Kite and Lightning's Sense of Peso, which uses this really beautiful opera song that really kind of helps carry that experience along with the amazing visuals that it included, but can't really underestimate the impact of music and being able to help tell the story. So another just takeaways that you know just I think over and over again I've just seen that in terms of editing you really have to slow it down a lot I mean in this case a factor of two it was really interesting to see an actual quantitative measurement of how much to slow it down because Patrick had cut a 2d version of this and said that they use 75 cuts but within the immersive version that they only did 38 cuts within pretty much the same length and so you get a sense that you really need to give people an opportunity to kind of look around and when you actually put into the experience I think that you need a lot slower type of editing. So I think the thing that Patrick is saying, and I've heard other people repeat as well, is I think that the audience is still really learning how to watch these types of films. And I think this is a really great experience to help train them how to really look around within the experience. And I think it was very insightful for Patrick to say that the VR enthusiasts are the people who kind of tend to look around and push the limits and edge of what type of things you can experience. within a narrative 360 video, especially if it's interactive. But yet people who are just watching a lot of these experiences for the first time, they've been so trained to just look straightforward at the frame that they've been given that it's really difficult for them to learn how to look around. And so I think that even if the filmmakers were able to come up with the most cutting edge interactive type of experiences, I don't think audiences would really be able to know how to watch them yet. And so I think we're taking slow steps towards that. And as people watch more and more 360 videos within a VR headset, then I think it's going to be easier to start to push the boundaries and the edges for what type of interactivity you can add in there. Also, it's really interesting that Patrick is going to be moving more towards using live theater actors within these 360 videos. And I think that's really smart. And then I think if you look back at some of the other interviews I did with Chari Melcher and, and this upcoming interview that I have with Cosmos Sharf, I think that more kind of theater trained actors to do either live improv or be able to put these actors within these experiences, but put them into all sorts of amazing different costumes or environments that you're able to kind of have a theater performance but have it like in a spaceship or the sci-fi dystopian future or something where you're immersed within an environment but able to see this live theater being acted out by humans that you know are humans and I think that is actually going to be really compelling within VR. And finally, this was done by Google's Advanced Technology and Projects section within their Google Spotlight Stories. They've done a number of different 360 videos that are out there. You can watch them on their YouTube channel. And you can either watch them on a Google Cardboard. And I think that they're planning on releasing a Vive version at some point of Pearl. And I highly recommend you waiting to see it on the Vive. But in talking to different people there at SIGGRAPH, Google is actually developing their own kind of like timeline sequencer to be able to do these types of stories and it's called Moxie. I haven't seen a lot of other information about Google Moxie anywhere online. Not sure if that's going to be the file name or the code name for their story sequencer. So it sounds like that Google is going to be releasing Moxie at some point, but no further information about when that might come out. But just by judging the quality of the types of experiences that they're doing with Pearl, I'm looking forward to seeing when that comes out and what that's going to be able to do for creatives to be able to more easily have a production pipeline to be able to produce these types of animated 360 degree videos. So that's all that I have for today. If you enjoy the Voices of VR podcast, then please do spread the word, tell your friends, and go sign up for my email list on voicesofvr.com to be able to sign up for future events and also to just keep posted on some of the big news that will be coming out for the Voices of VR here in the future. And if you'd like to support the Voices of VR podcast, then please do consider becoming a donor to my Patreon at patreon.com slash voicesofvr.
