Disney Animation was showing Cycles, which is their first narrative VR short film that they produced during private screenings in Park City during Sundance. It’s a beautifully-produced piece that explores the connection to place, memories and nostalgia in a way that only VR can. It was a part of an internal innovation initiative within Disney where lighting artist Jeff Gipson’s pitch was greenlite allowing him to direct the piece.
I had a chance to talk with Gipson as well as with senior software engineer and technical director of Jose Gomez. Gomez mentioned that he’s also working on other virtual production initiatives within Disney meaning that there are many ways in which VR technologies are changing the production pipeline within Disney. They wanted to use production quality formats within real-time game engine of Unity, and start to do the types of hand-crafted flares that are hallmarks of Disney animation within this project.
Gipson and Gomez shared their creative journey and challenges of the project, and this will no doubt be one of many more immersive projects and experiments from Disney. It’s also going to get Disney to think about how to distribute these types of immersive experiences.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. Well, an experience that a lot of people have at Sundance is these moments of serendipity and synchronicity that happen, especially if you have a little bit of an openness within your schedule. And just as an example, on Saturday night, I went to a party, and then I actually had a chance to meet a lot of people from Disney Animation, They knew who I was, they were fans of the show, and they said, hey, you should definitely come out and check our first VR experience that we're showing. It's at this private location on Main Street, and they gave me this ticket to go check it out. So, on Sunday, after I had a chance to watch the Under with Oculus, I went to go check out the Cycles experience, and then did an interview with the creator and one of the technical directors of that experience. And so, in this interview, I'm gonna be talking to Jeff Gibson, he's the director of Cycles, as well as Jose Gomez. So when I went down into the basement of this location on Main Street, before I saw the VR experience, it actually showed me a short little video giving a little bit more of a context as to what this story was, which I don't typically see on a lot of other VR experiences, but I think what things that I do see in terms of a trend at Sundance was that you're going from the mundane world And then you're entering into this interim liminal space that you're learning about this world that you're about to go into. And a number of different installations that were happening at Sundance either had like immersive theater actors that were interacting with you in some way, or there's a whole physical art installation that you're physically walking through to be able to give you some sort of symbolic representation. That's a spatial piece of art that you're walking through and experiencing before you go into the VR experience. And in this case, rather than going through that type of art installation, it was a video. And I actually think it did a great job of setting the context of what this experience was because it's actually the story of the director's grandmother who has to go into assisted living. And so it's this process of his mom talking to his grandmother to be able to transition from living in this home where it had all these memories and then transitioning into this next phase of her life. But you kind of take this montage journey through an entire lifetime of somebody's home and all those memories. And Jeff also collaborated with his mom in order to compose this music. And so I actually like, it was a very emotional video and I really felt the context and meaning of this piece. And I think if I would have just saw the piece, I would have maybe got it, but I got so much more out of it. I think knowing a little bit more of the backstory and the fact that this was actually like inspired by a real authentic scenario and that for me, it actually made it so much more compelling and interesting. So anyway, I'm gonna air this interview, but I wanted to give you a little bit of that context because that's the context that I had going into this experience. And then we'll be unpacking a little bit more about what Disney is doing with virtual reality in the end. So that's what we'll be covering on today's episode of the Voices of VR podcast. So this interview with Jeff and Jose happened on Sunday, January 27th, 2019 at the Sundance Film Festival in Park City, Utah. So with that, let's go ahead and dive right in.
[00:03:12.878] Jeff Gipson: My name is Jeff Gibson, I'm the director of Cycles, Disney Animation's first VR short film.
[00:03:17.443] Jose Gomez: And I'm Jose Gomez, I'm a software engineer at Disney Animation and I was the technical lead for Cycles.
[00:03:23.130] Kent Bye: So maybe you could tell me a bit about how did this project come about?
[00:03:26.895] Jeff Gipson: So a couple kind of inspirations, one was my relationship with my grandma and kind of if I was in my house, I was always at her house growing up. And eventually the time came where my family had to have that hard conversation, much like many families do, about moving her into assisted living. And seeing my mom and my uncle at that dining room table telling her, you know, having that conversation and just, After we moved her to assisted living, seeing her house empty and seeing the invitations on the carpet or my handprint in the driveway, my name etched in the back cabinet, knowing that this home held the story of my family, it was just a really powerful memory that drove the story forward.
[00:03:57.678] Kent Bye: Yeah, maybe you could talk a bit about the process by which Disney Animation is starting to get into virtual reality. Maybe what were some of the process by which you have to, in some ways, figure out the pipeline by which you're going to actually storyboard and sketch out and figure out how to even make a piece of VR?
[00:04:14.708] Jose Gomez: Cool, so part of my job is as software engineer over there is virtual production and for the last three years we've been researching on the use of the new technology like immersive technology and when Jeff pitched this idea and they paired him with us because they knew we have done some experiments in VR but we have to figure it out and also the other part is This came about in a talent development program where everybody in the company can pitch, but you only have four months to do it. So we assembled that team and we had to figure it out. So we tried to reuse as much as we could from our film pipeline, but some things like figuring out the spatial locations and all of that, it was totally new for us. So we used, I mean, many different things. I mean, you can talk about your background in architecture and how we how we use a scout and all that?
[00:05:03.884] Jeff Gipson: Yeah, so I was an architect before coming to animation. And that's one of the things I love about VR is it's so spatial. It's a spatial language. And so I designed the house in SketchUp and then Jose had written a tool called VR Scout Tool. And so we brought that into Unity within the first week, just a quick SketchUp model and was able to start thinking about where do we want to be in this space and how do we want to place each moment. And then we started doing some Quill storyboards. One of our animators, Daniel Pesce, is an amazing Quill artist. And so he came to us like, hey, how do you feel about doing some Quill storyboards? And he did some amazing paintings that we brought in. It was cool because we started to get a sense of volume and scale and kind of that appeal. We had done some flat cards, some drawings on flat cards, but it wasn't giving us that scale and that sense of proximity. And so it was really cool.
[00:05:46.024] Jose Gomez: We created, I mean, in Unity, we had that model in SketchUp and did flat cards, the drawings in Quill. So we did our animatic where we were placing things in different places and The time lapses we were simulating just rotating the Sun so you get that feeling it was pretty you know rough to watch but but it was the way of Really pitching the idea because when he just explained it to us It was like and then there is a moment and other moment and people had a hard time understanding how it was gonna work but seeing all in unity, you know, like I in VR with Quill gives you this special, it's not just a drawing, it's kind of a sculpture, right? So you can pick from the sides and decide. So that was really helpful. From there, we knew we didn't have enough time. We wanted to handcraft all the animation, but we didn't have time to experiment with it, so we used motion capture to do, like, all the choreography. So we had, actually, Jorge, our head of animation, and Lauren Brown, our producer. They were our motion capture actors, and they were acting each of the moments, and then we did retargeting of that motion capture into Big Hero 6 characters, because that's what we had available at that time. and we were placing that and when we were sure how the movement of everybody was going to be, then we handed it to the animators. And our animators used Maya to do the animation, but also we had an experimental tool, which is the pose tool, which is kind of a stop motion, but in VR. So this is the first show that we have used it. We developed it with an intern and my team, like a scout. I mean, the scout is not just me. It was an intern and some people from my team. And yeah, we used that to figure it out, and then animated Maya, and then we exported to Alembic.
[00:07:41.309] Jeff Gipson: Oh yeah, you can talk about cloth and... Yeah, I mean, one of the big things about our feature films is we love the simulated cloth, and we love that art-directed hair, and so we wanted to bring that feel to this, to where all the shapes are designed. It's not just there arbitrarily, so... That was another piece of why we used the limbic. One of our VFX leads, Ed Robbins, was a big proponent of like, hey, we should really try to get our feature quality animation into VR if possible. So we were talking about some FBX things at first, but then he was like, man, we should go for the limbic. Let's try to simulate the cloth. Let's try to get some simulation with the hair. And it was cool.
[00:08:14.311] Jose Gomez: Yeah, because one of the things, I mean, obviously we had to go down in resolution and quality compared to what we do for film. But cloth it was something that we didn't want it to look Kind of like in games, which is super cool for a game, but but we wanted to have those Handcrafted shapes that Disney's famous for right so get into our tradition not we do simulation but then we have tech anim artists that make sure the shapes are beautiful right complementary so this is one of the things also like why I We choose to do it from a single point of view. I mean, you are free to move, but you cannot be wherever you want in the room, because we come from film and we are used to save everything to camera and do the pose where it looks cool. So we made it so that you are looking kind of almost like you are a camera. You are not moving that much. And then we make use of this art directed cloth that makes everything to look a bit more beautiful.
[00:09:14.297] Kent Bye: Yeah, and I'm wondering if you could expand a little bit of what you see as this connection between space and architecture and memory and how you're able to use the medium of virtual reality to almost like create this nostalgia of someone's life as you're doing this time-lapse for all these different moments. I get the sense of like with these digital avatars, I get the sense of these very intimate moments that your grandparents had shared within the context of their home, but just trying to cast this larger arc over a long period of time of an evolution of a lifetime, but I guess the main character to some extent being the home of different things happening throughout the course of the home.
[00:09:51.108] Jeff Gipson: Yeah, I love that about VR is it can transport you to a place and you really are immersed in that. Even, it's so funny, I looked at my photo albums from back years past and it's funny to see, oh man, the carpet was that color then, or that was on the wall, I remember that. And it's so cool that VR can really show you a space. And that's what we try to do with Cycles, is show the space over time. And it's something that's so unique to VR is being able to experience a virtual environment. So I love that about it.
[00:10:18.537] Kent Bye: Yeah, and I'm curious what's happening with virtual reality. This is the first experience from Disney, and we're four or five years into the consumer renaissance of VR. And so it's taken a while to get the first one. But now that you have it, what's next when it comes to VR and Disney?
[00:10:34.564] Jeff Gipson: Yeah, one thing is, you know, it's always the question of, like, oh, distribution, distribution, what's up with it? So we've been showing at festivals and things, but I think it's cool because of cycles, it raises questions to our executives, like, what does it mean to distribute cycles in future projects, our VR projects? How do we do that? And so it's cool that now the studio is starting to think differently and think ahead, and it's kind of exciting, so.
[00:10:55.627] Jose Gomez: Yeah, I mean, I think it's great that this was experimental. I mean, we weren't even sure if this was going to go out of the studio. But I mean, I think there's something. And the more people that sees it, the more eager people are to let's do more. But I mean, the good thing is that we are kind of in the creative side. We don't have to figure out what's going to happen with this. And we want to continue providing material for them so that they are like, we have to do something with this.
[00:11:24.103] Kent Bye: Yeah, and I had a question about one of the, I guess, technical decisions that you made in terms of there's different action that's happening throughout the course of the house. And if I look too far away, it starts to dim down into gray and eventually black. And my sense of this is I can see how that could be used to maybe train people who are new to VR to make sure that they're focusing their attention. But I feel like in some ways I'm being penalized as a VR user to try to get a sense of the space because as I go into VR I like to kind of sometimes just look around just to get a sense of the space and I feel like I couldn't do that because I feel like every time I did that I was penalized for looking away from where you wanted me to look.
[00:12:00.988] Jeff Gipson: Yeah, I think so. Part of the reason for that decision was it's the whole film is memories. And if we think about our memory, you never remember what's behind you. So we use it as a storytelling device, but also a way to hopefully guide the eye. But I mean, I can definitely see, you know, the veteran VR enthusiast would get that sense. But I think it was also trying to direct, really trying to have the audience follow.
[00:12:23.011] Jose Gomez: I mean, maybe you're right. I don't know. I mean, it's taste. But I think we are in a moment where we should try to do things for people that are new to VR. I mean, when we are showing, we get so many people that they are kind of afraid. And this is the first thing that I've seen in VR. So I think it's good to be gentle. I think it's a good decision not to go for high adrenaline and you have to be a hardcore gamer to understand this. So yeah, maybe in the future we don't need to do things like that, but it was a creative choice. Many people like it and I don't know.
[00:12:58.903] Kent Bye: Well, I guess from my own episodic memory of going through the experience what it feels like is that I was in an immersive environment But I watched a 2d film Because I was robbed of the ability to be able to get that spatial ability to turn around and because of that my memory of it It was actually breaking presence of that place because that's nothing that I've ever experienced in a home. When I turn around, it turns black. And so it's like a virtualized layer of reality put on top of something that is beyond anything I've experienced. And as a viewer, I was learning that I'm doing something wrong by looking around. which I think is kind of the opposite of what VR should be, which is to try to allow the freedom for people to look wherever they want. If they want to miss the action, then maybe they can watch it again. But in some ways, it's a little bit of trying to control the frame, which VR is about trying to get away from that. So I feel like there's a bit of a philosophical shift of coming from a cinematic storytelling tradition from Disney that is very much into that control. But I'm like, no, no, no. Let's try to let go of that control and see what happens and give the viewer the freedom to make mistakes if they want.
[00:14:04.061] Jeff Gipson: Yeah, and I think it is. It comes from working in cinema and film and trying to really focus that attention. I was a lighting artist before working in VR and directing, so I'm really focused on like, how can we use color and light and composition to guide the audience's eyes? So I think that also played into that decision. But it's also interesting because I feel like we're in a time of virtuality where everyone's figuring it out. Like, how do we tell stories? How do we connect with audiences? And I think that's one thing I love about virtual reality is it's kind of like, there's no right or wrong way. It's kind of,
[00:14:33.265] Jose Gomez: So many creative ways to go about it and it's kind of exciting and so Yeah, you know, I would like to say that basically everybody's testing things and we tested this maybe other people like it then I mean, I would love to see other people using it right like It becoming something or maybe not Maybe you know the future is like oh remember people at the beginning trying to close everything. It doesn't matter I think it's important just to try things and see how it works. So
[00:15:00.339] Kent Bye: One of the other things that I really loved about Cycles was the use of light and lighting. And what I guess is referred to as a zeitgeibers, which is this indication of time. So you have the sun and the shadows. And I can imagine from your background in architecture of recognizing the importance of seeing how shadows move through a space. But it not only gave me a sense of the passage of time, but being able to do the time lapse. But it also felt like a piece that was maybe five or 10 minutes when it was really only like three minutes. It gave me this perception of sort of cramming in so much information so quickly that it perceptually felt like it was longer than it actually was. But I'm just wondering if you could talk a little bit about that process of the lighting that you were doing there, because I feel like that's a subtle piece that was an important part of creating an overall experience within Cycles.
[00:15:46.414] Jeff Gipson: Yeah, I mean, light is a big key of how we're showing the passage of time, and initially we had many rotations of the sun, but we ended up just going on a gradual, slow, kind of single rotation. You see those long shadows move through the space, and even directing the eye as well, using not only the Gomez effect where we're looking away and starting to saturate, but also lighting and motion to do that, but the light. I love it because in VR you almost feel that summer sun coming through that window, you know, and you see that glow. I love experiences where you almost just feel it, you know, your brain fills in. And even like the night shots, having that idealized Southern California night sky, and that's one thing I just love about VR. And coming from, you know, lighting artist background, it's just kind of in me to just be naturally gravitated towards that. And it was fun to try to figure it out. And it's also limited. We have a, when we make our feature films, we have such a tool set really dial in everything so there was a lot of the rain that we had to kind of let go running it in a real-time engine but it was also kind of freeing because then you're really figuring on big strokes what is most important in this shot or this scene and how do we complement that with light.
[00:16:50.634] Jose Gomez: Yeah, I mean, it's cool that he doesn't have to worry about the technology, right? But when he was pitching to us, he's like, yeah, I want to play with the light. And I want to maybe some different light setups. And we were like, OK. And he was like, but I want it to be seamless. And I want to go from day to night. And we were like, dude. What are you doing with us? But yes, so the fact that we had these time lapses helped us also doing the transition of the light so I mean going a bit more in the technical side we have a specific setup per shot and we use those time lapses to interpolate from one to the other so that It looks like you really are not getting cuts or change or anything, but we are transitioning and it's that flow. You are continuously into the same scene, right? So that's probably why many people comment on, it feels longer because it's just one long shot, right?
[00:17:40.393] Kent Bye: Yeah, because you're also, I guess, baking the light and then having to fuse them together. Is that what you mean?
[00:17:45.205] Jose Gomez: Well, yeah, I mean, yeah, we have to do many tricks. Yeah, we have to, we use a lot of baked lighting and also emissive lighting so that we can control how emissive they are. So if you turn it to zero, it looks like the lights are off. We use that in the TV. I love that shot with the TV at night where we are just flickering the light, but it gives so much life to that moment. So yeah, I mean, we could talk for hours about the light in the shot, but yeah, I don't know.
[00:18:12.088] Kent Bye: Well, I'm curious to hear your own experience of creating this space that you may have had a connection to already, but then recreating these different moments that the characters representing your grandparents, but also with music from your mother. There's a lot of deep connections for you personally. So I'm just curious as you're going through this process of those moments when you have this kind of surreal transport of quality that maybe go for you specifically over your own memories of being able to tap into that through the virtual reality technology.
[00:18:39.731] Jeff Gipson: Yeah, I mean it was it was an emotional experience for sure. I mean it's I always tell a story but it was one of the first times that we'd seen our characters in the set and had a little bit of animation on them and some lighting and we added in the score just just two and everybody had watched it but after everybody left I put on the headset and just kind of had a moment for myself to be in that space with these characters and hearing the music and it was just, it was emotional. I was in tears. I remember just being like, this is crazy. And it's, you know, it's a dream to direct a project like this at Disney animation. It was just, I get chills right now just thinking about it. It's, and I was in tears. I just kind of pinched me, you know, and, and it's still cool to, I think that's what's so, you know, these characters kind of embody a little bit of my grandma and grandpa. It's a lot of, it's all fictionalized, but just that feeling and having my mom be a part of it is, it's cool. I feel like, When I told her that I was selected and told her the idea for the film, just to have her have that emotion, her compose a piece, but have that emotion that she can project. And hopefully you feel that when you're in it. You feel, not only myself, but my mom as well, you know, feeling this. So it was an emotional project.
[00:19:41.698] Kent Bye: I think that one of the affordances of VR is that it's able to share these types of moments that you wouldn't normally think of would be a compelling story, like telling your mother or grandmother that she has to go into assisted living. But to be able to recreate the entire context, I feel like there's something about the storytelling of VR that's less about you as a hero individuating, but more of a either ego disillusionment or seeing how you as an individual are connected to a larger whole and being able to show the relationships between a place and different memories and how these moments in time are unfolding in these processes. But you have these bodies moving through this space with these memories that there's certain aspects of the story that I think that just wouldn't translate as well or be as emotionally impactful Into a 2d and I'm just wondering if you've thought about like why what is it about? The medium or this story in particular that you think works particularly well in VR, but maybe wouldn't work as well in just a 2d. I
[00:20:39.976] Jeff Gipson: Yeah, I mean, I think part of it is being grounded in that space. You're rooted in that house and you can visually see it and feel it and see the moments happening around you and you can feel the character's proximity to you. And there's the moment towards the end where you, that's the kind of more interaction piece where you would never get that if we were watching on a screen. You could suggest it, but that kind of moment, watching people, I can always see when people are viewing it, see that moment. They're like, that aha moment of like, oh crap, I understand who I am now. And they kind of moving around and it's, that's what's the power of VR. It's so cool to have that connection.
[00:21:14.739] Kent Bye: Well, from a VR design perspective, you're standing one spot and looking around and you were able to translate this experience into an augmented reality experience where you're able to then use a tablet and be able to almost like get a window into some similar scenes. And so I'm just curious to hear your design process for translating this VR experience into an augmented reality experience.
[00:21:32.930] Jose Gomez: Well, I mean, this is just experimentation, right? We are experimenting and see what it means. Once you have your assets ready for VR, I mean, for the game engine, then it's not that difficult, I mean, using one of the game engines to get into with their technology. So, for our experience, we use, like, machines pretty beefy, like gamer machines. So the only problems were like taking it to an iPad. We have to bake more the light. But basically it's just using what is available. We are doing image recognition and then we create an anchor out of that and we put the house and it's just a sneak peek. It's a way to show our environment. It's like a kind of a teaser. We're experimenting on what could it mean to, you know, these new technologies to promote your film or something like that. So it's not that much work and then you can do something that you can share in social media or things like that.
[00:22:27.800] Kent Bye: So I'm curious to hear some of the either biggest questions that you're trying to answer or open problems that you're still trying to solve when it comes to storytelling and virtual reality.
[00:22:38.689] Jeff Gipson: I think it's, like I mentioned before, it's kind of like the Wild West and I love that. I love that it's kind of just figuring it out. Everything, everything's kind of a question. And Cycle is really, for myself, it was pitched as VR because I felt like the space was such a big part of it. But now I'm really interested in, Continue on connection and other emotions. This one's a very dramatic emotional moment But there's other emotions that can connect with people in VR and so that's what I'm curious about exploring now is like What other stories can we tell in VR that not reuse all of what we did in cycles? but what other techniques are there that we can try to figure out that would help drive a narrative and so it's I It's not really an answer to feel really like mushy, wishy-washy answering it. But yeah, I think what I love about it is it is so unknown and you're just going for it. All of us are pioneers moving out and we're trying to figure out what's the language and what type of stories or what kind of experiences. And that's what's awesome too is there's a wide variety of experiences. There's a lot more that are interactive. There's some that are documentary. There's animation. And it's this big, broad thing that you can't just pinpoint one thing on.
[00:23:43.495] Jose Gomez: Yeah, I mean, to me, I'm an engineer. I like solving problems. So what I love is to partner with people like Jeff or our team. They come with problems and trying to solve it out. And the thing is, the things we have done here, we think solve the problems that we have for this particular piece. But we are sure they won't solve problems for the things that we want to do. So just continue exploring. And when there's a new technology, jump into it.
[00:24:13.138] Kent Bye: Great. And finally, what do you think is the ultimate potential of virtual reality? And what do you want people to enable?
[00:24:22.287] Jeff Gipson: I think it can get, I hate to use the word empathy, but I mean, that's truly what it does. You jump into, in our case, a character's memories and you see things from her point of view. And some of the documentaries, you see things from a point of view of other characters and it makes these connections that are so much more meaningful and much more impactful than sometimes a flat screen can do. Cause you're in there, you're with the characters and you feel them. And I think that's amazing for film, for storytelling.
[00:24:50.222] Jose Gomez: I mean, I love the story telling in VR, but what I'm really, really interested in is immersive computing, right? Like the use of VR as a tool. And I think the physicality and we have gotten used to work with computers with such a level of abstraction. We are looking at a flat panel, moving a mouse that is translating to a 3D movement on the screen. And we are removing those barriers with virtual reality. I mean, as I said, we were testing this post VR tool. to do animation. And what is great about it, when you get an animator there, it's just like playing with a toy. Because you are grabbing an arm, and you are moving it. You are reducing the levels of abstraction. So I think that physicality and that immersion is going to bring new tools that are going to let us work with computers in totally different and new ways.
[00:25:43.034] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the immersive community?
[00:25:47.715] Jeff Gipson: I hope they I hope you have a chance to come and view cycles We'll be doing some festivals and then hopefully we'll have a release at some point to where folks can download it and check it out And I'm just excited for people to see it.
[00:25:56.643] Jose Gomez: Yeah, no and just say thank you to you for coming and see and And I'm a fan of the voices of VR the work you're doing for VR is really great. So so yeah, keep on doing the good work Awesome.
[00:26:09.294] Jeff Gipson: Great. Well, thank you so much. Thank you, man.
[00:26:10.696] Kent Bye: Thank you so much. Thank you So that was Jeff Gibson, he's the director of Cycles, as well as Jose Gomez, he's a software engineer at Disney Animation, as well as the technical lead for Cycles. So, I have a number of different takeaways about this interview, is that first of all, Well, it's just a beautiful piece. It's about three minutes long, but actually felt a lot longer. It felt like maybe a six or seven minute piece. It's interesting because it's like this time-lapse montage and it feels like you get so much context about these characters and their lives and these different moments that it really feels like this intimate connection to this world that you're drawn through. And there's aspects technically how they did it that I may have never noticed. Like if I would have watched this experience, I would have never like said, Oh yeah, the way that the cloth was moving or the way they styled the hairs, you know, that's not something I'm necessarily paying attention to, but to know that this is actually a key component of a lot of the tradition of storytelling and animation that comes from Disney, it makes sense that they've been really trying to figure out this pipeline to really give it their own Disney-fied style. And I really think that they nailed that style just in the sense that it looked amazing and also the story was really compelling. Now, in my previous interview with Colm Slevin, he was talking about how we have this tendency of wanting to formulate these different rules. And in my mind, there were certain rules in VR that they were breaking. It's a little bit of an open question as to whether or not this is good innovation that they've created or something that makes it a worse experience for a certain class of users. So what they did is that as you're watching the experience, if you're looking forward to where they wanted you to look, it would be fine. But as soon as you started to like look away for where you wanted to look, they would start to fade it out. And if you're looking 180 degrees for where they wanted you to be looking, they would make it completely black. So it made it difficult for me to actually get a sense of the space because they kept disallowing me from looking at things when I wasn't paying attention to the main action. Sometimes I do that. I think within a VR experience is that I kind of like look around just to reorient myself, because I think it helps me to firm up those memories of that spatial environment. And then whenever you're taking that away through that mechanism of what feels to me like they're trying to control where I'm looking, which. If people have never had a VR experience, for example, and they don't know that they can look around, that is a very useful mechanism to help you know that, oh, you should be looking over here, because it subtly trains you that you should be looking in a certain direction. But because I don't know anything about the experience, I don't know if this is an interactive narrative. I don't know if me looking around is somehow controlling different things in different ways. It took me a while just even to figure out what was happening. And then I realized, oh, they're just trying to control where I'm looking. It just felt very heavy handed for me as an experienced VR user because it didn't allow me to look around and have the free will and the agency to be able to look wherever I wanted to. And if I don't want to look at the narrative that's unfolding, then that's on me to either like miss it or to watch it again. And I think that is the thing about virtual reality is that you're actually trying to provide a whole contextual experience. And at the same time, whenever you have a time-based narrative that you actually want people to really pay attention to, then there's all these tricks and things you can do sound, you can use motion, but also just trust the user to maybe want to watch it again if they miss something. And I think that's the thing in real life. There's always things that are happening and that you may miss, but in real life, you can't have the experience again. And so. I think VR inherently has these elements of FOMO or fear missing out. And so it's kind of up to you to pay attention to the narrative that's unfolding or for you to be looking at something else. So anyway, I'm still not quite sure as to whether or not I want this to really propagate because if anything, I would want it to be an option because I feel like it's a little bit of a crux for narrative designers to control where people are looking to make sure that people get the narrative as it's unfolding. But I just think that there's too many trade-offs for me that actually makes it a bit of a worse experience. So anyway, I think it's an interesting question that I don't know if I'm right or I'm wrong or where things end up going, or if this ends up being a little bit of like, this is actually precisely what we need in order to help train people to be able to understand how to watch a spatial story that's unfolding around you. But for me, I just, I would just want to give a little bit more trust to the users that they're going to figure it out and that there's something about your eye being attracted and drawn to things that are unfolding and also your ears are the most spatial medium and so using sound can also be a big impact there as well. So just technically that's like my biggest complaint against the piece. Other than that I thought it was like an absolutely amazing Story, very compelling. The animations looked amazing. And the lighting was really significant just because they're able to like show the shadows and do this time-lapse type of thing. I think time-lapse and spatial environments is just super trippy because, you know, usually when we watch the course of a day and the sun and the shadows moving around, it happens over a long period of time. And then when you're in a spatial medium and you just sort of like accelerate that, then you're able to, in some ways, jam in so much information that's happening over time. that your brain just kind of processes that's happening over time. And even though it was three minutes, it just felt like it was a lot longer. And so just some couple of other points that I think came out of this. Distribution for VR is obviously a big open question, especially when it comes to narrative. And just having Disney help think about that, I think, is very important. The trend seems to be that there's Netflix and then Disney, I think, is going to be starting up their own streaming service. And so the distribution platforms for some of this content is a bit of an open question. Are we going to have this like each of these content developers? Are they going to be having their own subscription models so that you could buy a subscription to Disney and see all the Disney films and then have access to all these story worlds? Definitely, that seems to be like a model where things are going to be going to maybe this subscription model, especially with these content developers. And so You know, coming up with a solution that actually works for the content production of these independent stories, as well as with these distributors, I think is one of the biggest open questions for how is this going to play out? But I think that as Disney starts to think about distributing this immersive content, it's just good that they're starting to think about how to actually get it out into the hands of the audience. Just a couple of other points here. One is the Olympic as an output format. It was interesting to hear that part of the directive at Disney was to try to match the same quality of the animations that they're doing. And it seems like the Olympic format is like this open source format that was announced back at SIGGRAPH 2011. and it's a number of different programs like Maya can output it, but that may be like the final production output that they're able to create this kind of baked lighting within these animations that looks just as good as it would if it were rendered out and shown on the final film. Of course, it's going to be a lower resolution than what you're getting on the final film, but they're starting to get to the point where using real-time game engines to match the same level of output of the animation quality that they're doing in these other Films I think is a huge change in the shift of where the future of this going is Essentially that allows for more and more opportunities for you to engage and participate within that experience if it's in these real-time engines And finally, Jose mentioned that he's working in virtual production, and I've got some unpublished interviews talking about some of the virtual production trends that are happening within Hollywood, especially on Ready Player One, an interview that I did at VRLA last year. But it does seem like these virtual reality tools are having a huge impact in terms of how films are being produced in Hollywood. And so it sounds like Jose is working on some of these virtual production tools. And so In essence, whenever you're designing these worlds that then are being translated into 2D film, it makes a huge difference if you're able to actually embody that virtual world and build those worlds within that world. And so just the process of 3D asset and art and world building, that's having these huge innovations and changes. It sounds like they're starting to use these different tools like Quill, which allows you to do a little bit more sophisticated timeline and folding of kind of like a rough prototype. So it sounds like there's a pipeline that they're creating using some of these existing 3D art asset creation tools that are making huge impact that are allowing them to do this pre-visualization stage with some of the either Tilt Brush or Quill or some of these other tools that they may be custom building. It sounds like they built a tool called Scout where they can bring in SketchUp architectural designs and then start to put them into Unity and also that virtual production also just so you actually have like a virtual camera and you can get some of the different shots and maybe do a handheld Previsualization or maybe eventually get to the point where you're you're actually maybe have a virtual steadicam operator That's actually working within the context of virtual reality and is doing this cinematography within these virtual worlds and so just the whole realm of virtual production I think is really starting to revolutionize the process of of special effects and 3D virtual worlds, especially these actors in these green screen films, they're able to get a sense in their mind as to what this world that they're in happening. And then they're able to use that in their imagination and get a lot closer than if you're just operating on a green screen with a bunch of motion capture actors and like these suits with like little ping pong balls, there's that. And also it sounds like they actually use some motion capture actors to be able to do some of the animation rather than just doing everything by hand. They're doing this kind of blend of motion capture blending with like these real-time game engines and in some ways this piece feels like they're Crystallizing their pipeline and their production workflow and I'm personally just excited to see what else they produce because I think they've got a real strong understanding of story and they're gonna be really trying to figure out what this spatial storytelling medium is able to afford and what they're able to do with it and It's just exciting to me that they're starting to experiment in that way. I So that's all that I have for today. And I just wanted to thank you for listening to the Vistas of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. I am relying upon people subscribing to me as an artist. And so as I was talking about the business models for the larger VR industry, I think this is one viable solution is that if you find different creators that you enjoy, then support them. It doesn't have to be me. If there's anybody that you're enjoying their work, you know, support the work and they can just make a living at it. And it just enables more other people to start to do that as well. So, yeah, if you're enjoying this podcast and find that you're getting a lot of value, then please do become a member of my Patreon at patreon.com slash voices of VR. Thanks for listening.
