One of the most inspiring keynotes at Unity’s AR/VR Vision Summit was by Alex McDowell on world building and storytelling. Alex was production designer for Minority Report, Fight Club, & Lawnmower Man, and so he’s been creating imaginal worlds for many years. Alex argues that the process of building worlds can inform and inspire stories, and that storytelling is coming from tribal oral traditions to singular viewpoints and now back to multiple perspectives with immersive experiences. The more multi-disciplinary VR and VR gets over time, then the more rich and profound the experiences are going to become. I had a chance to catch up with Alex after his keynote to unpack some of his innovative world building narrative work that he’s been doing at 5D GlobalStudio and with students at USC’s film school.
LISTEN TO THE VOICES OF VR PODCAST
Here’s a video of Alex’s keynote at the Unity AR/VR Vision Summit:
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:11.975] Alex McDowell: I'm Alex McDowell. I'm a production designer originally in film. I'm now creative director of a studio called 5D, and I teach at USC. I teach a medium called world building, so a sort of future of design for narrative media. And that's primarily where my interest in VR lies. I've been super interested in these kind of disruptive medium that completely change the way we think about the design space. So I've been working in a way in a fully spherical world my whole career because you're designing a world within which the stories then take place. But now we have to design a world that is the story. So I'm really, really interested in what VR is doing to stimulate that.
[00:00:50.221] Kent Bye: Yeah, it really feels like, you know, in your talk you're talking about like stepping into the shoes of a sci-fi urban planner where you're actually building the future of, you know, thinking about all these new technologies and the limits of that. And so, talk about what happens when you actually step into the world that you've created in the future.
[00:01:08.290] Alex McDowell: So I think there's two or three layers to that answer. First of all, how do we create differently? I'm really interested in these tools as creation tools first. How do I collaborate in a way that I've never been able to collaborate? In a raw, fluid design space, in a piece of architecture that is being built around me, my partners with my creative collaborators in the same space. So that's one whole area, which we're just scratching the surface. And that's not about, you know, B2C and taking it to market. That's about how we literally change the tools we use. And then I think there's kind of two or three parts that spring out from there. I mean, you can use a full VR or augmented reality, mixed reality, framework to develop narrative ideas and then you can push it out into transmedia. You could develop a comic book that way, you know, I'm working with a group called Ape Lab who are doing 3D comics that you can walk through. You can certainly make films that way and just end up still with a linear narrative but you've done your planning in a spherical space or you can push into VR and AR. So then augmented reality is a really interesting area that we're again just sort of scratching the surface. Being able to pull up a tablet and move it around in a space and augment that view is interesting. We're getting close with HoloLens and those kind of technologies to be able to put that on our head and be able to do this kind of mixed reality where you're enhancing the world and that's one kind of storytelling. And then we have sort of cinematic VR, video-based VR, where you're in a real space and the camera's sitting there and you've got the ability to go to places you've never been and experience them photographically. At the moment, you can't move the camera around so much. You know, it's harder to be really able to treat that as a fluid narrative space. It's more like observational or documentary. And then the space that I've spent most of my time in in the last four years, which is fully digital, virtual environments where we have complete control over the way the camera moves and therefore over the way that you move through that space and what your relationship is to the narrative environment and who you are and whether you can be embodied or have presence that's entirely different from the form that you're familiar with like the humans. So we've been working in this project Leviathan where every creature in there is a fabricated creature and you actually have the opportunity to exchange bodies with these creatures or with other characters to go close to people and hear what they're whispering, to look at people across the room and hear them more clearly, to shove your hand through their head and listen to their inner monologue. We're really trying to experiment with really pushing at the edges of what you could do now that you have this medium.
[00:03:43.635] Kent Bye: Yeah, and one of the graphics that you put up there today during your keynote here at the Unity Vision Summit that I found really fascinating was starting with this tribal mythological storytelling and then kind of fracturing out into this first-person perspective and that you really see that this immersive storytelling mediums of virtual reality and augmented reality it's kind of a return to that mythic storytelling. So maybe you could describe to me, like, what do you mean by that, you know, original tribal stories and then that journey into the first person and then back into the multiple perspectives?
[00:04:16.445] Alex McDowell: One of the things I've been interested in for a long time, and this predates sort of AR, is how collaboration across disciplines can kind of expand all of our capabilities. So instead of consigning ourselves to these silos and saying, I'm a game maker, I'm a filmmaker, you know, I do this, you know, I play chess or whatever it is, this notion of really a Victorian notion of specialization as being an end goal, it seems to me incredibly interesting and alchemic actually, that when you start mixing disciplines, you increase your capability kind of tenfold. You know, that's what steered me back to the idea of where does story come from, what is story, and the idea that story evolved organically and aggregated from many, many, many storytellers who all added nuance and experience and a piece of their own directive to a global story space that continued to get richer and richer and richer. So, as much as I love film, you know, and I love many of those single-authored, single-viewer gaze, you know, proscenium-framed, screen-based media, it's not really reflective of our human experience, you know, to be told constantly where to look or what to read. So, if we are to combine the capability of this 360-degree space with a new kind of opportunity to collaborate across disciplines. My sense is that we get back to the possibility of creating massive stories, massive story worlds, and that's where I'm primarily interested. If you could build a world that is so rich that there's no end to how many stories that it can output. And if you can bring more and more people into that collaborative space, you no longer need to say, well, where's the director? Where's the designer? You can say, what is our intent? What is our fundamental requirement of this world? What are we trying to discover? What disruption is in place? And we can allow these kind of narratives to evolve. And again, I would suggest that that's much closer to the origin of storytelling, where we just reacted to things around us that we didn't understand. And then we passed on our knowledge through generations or outward to multiple people. And I think that this is an amazing opportunity to do that. So I think it's a problem when the translation is, how do filmmakers use VR? I think VR is just an opportunity to think about a completely different medium. And as I said, I love film. And the filmmakers who make films are amazing, creative people. But it is a different mindset to create narratives or to create experiences inside a fully spherical universe.
[00:06:49.488] Kent Bye: So how do you think that translates beyond just kind of put people into a first-person perspective into an environment? I mean, are there other lessons that we can learn from the Greek and Roman myths?
[00:07:00.015] Alex McDowell: It's a great question. I mean, I think one of the things I would say is, and I quote my friend Sebastian Silvan, who was the CTO at Weta, is that if anyone could tell you where this is going, they're full of shit. You know, it's like we really don't know how the story space is going to evolve to actually give us different kinds of experience. But I think My approach is that we can move towards these kind of big themes, like what is the future of the oceans, let's say, just to use an example that I'm working with right now. The ocean is so enormous that none of us can get our heads around it. It's a massive, endless horizon of surface that very few of us can really picture what it looks like or feels like to be below and to be immersed in. Once you sort of take that step and you get technology that allows you to immerse yourself completely in an unfamiliar environment, my sense is that that environment provokes the stories that it requires. And I think that's really the parallel. I think if you are making your way out into a kind of primitive landscape from the cave, you are reacting all the time to things that you don't fully understand and have no capability to deal with, and you're tuning your capability every day, right? And I think stories was always a big part of that. how do you train the next generation to look out for the saber-toothed tiger is, you know, tell really kind of dramatic stories about what happened when you last met a saber-toothed tiger. So that's not somebody thinking, oh, I can come up with a saber-toothed tiger as a fiction and then turn it into narrative. It's that sort of interplay between the things that either scare us or disturb us or are just unfamiliar to us and how they provoke new kinds of stories. And my sense is, and I see this happening all the time, we do huge collaborative exercises. world, a fictional city called Rolau in my conference in 2014 with 300 people and we wrote a thousand stories in two and a half hours with those people. You can do really really complex problem-solving with large groups if you frame it correctly and I think that's, so I'm not answering your question at all other than to say I'd have no idea where it's going but it's a lot of fun finding out.
[00:09:04.390] Kent Bye: Yeah, with the Leviathan project it's a mixed reality experience where you have a table that you have both in a physical space but also a virtual reality space and you're kind of doing a lot of different game mechanics where you're moving objects around and you're kind of directed with fireflies that are guiding you and then you have this all kind of like choice you have to make and kind of immersive experience where you merge with a character and float around. What was really striking to me about that experience was that it felt like it had a whole mythological and story background. Like, I actually did feel like I was stepping into a world that was rich and had a big, huge backstory. So when you look at that and reflect on this project, what are some of the biggest lessons that you think you're taking away from the Leviathan Project and what you see as it's contributing to this language of immersive storytelling?
[00:09:52.945] Alex McDowell: Well, because it was a research project, and actually really remains a research project, it came out of my lab, it went to Sundance, it's here at the Vision Summit, and then it'll go back into my lab. The main takeaway is the cumulative effect that we have of multiple different kinds of interactions that are taking place in a world that we understand with our whole body. I mean, I think the high-level requirement of these kind of experiences, which is a complicated one because it requires us to be able to move around in the real world and have some volume to work with as well as the purely virtual. You can't just do this with a game controller. But the high-level thing is that I am moving one-to-one with my virtual environment. So I feel my presence is being expressed in a way that I can really deeply relate to in the virtual space. not only turning my head, but every forward motion, every arm reach. And to some extent, and this is where I'd like to take it, is to what degree can we measure things like galvanic response, or blood pressure, or heart rate, or breath, as well as gesture. How do we really start understanding gaze tracking and where we're looking? Can we kind of cumulate this direct emotional response for the user participant inside that space? And then start using that to tell better stories. So I think it's a tricky medium. It's incredibly compelling to be fully immersed in the way that you are. We don't yet know what kind of effect it has. You start putting young children into this space. What we've been hearing and talking about is that it's so compelling to be fully immersed that they can't separate the virtual memory from their real memory. They actually don't make a distinction. We're working with neuroscientists to say, If you are in a spatial relationship with the volume around you, and everything that you're getting back from that world is behaving as you would expect it to in the real world, what is the difference? What is reality? So we have some interesting, deep, philosophical questions to ask. And we have a potentially very dystopic future here, with everybody sitting with boxes on their heads, plugged into the Matrix, and forgetting that they need to talk to each other at all. So I think it's really our responsibility to keep a very close eye on the fine-tuning and the detail and the reasons behind this kind of absolutely compelling immersion. And that's why I think augmented reality is equally interesting, because whatever happens in augmented reality, you are being reminded of the real world all the time. But what we did in the Leviathan Project was to say, don't forget haptics, you know, don't forget the tangible relationship to space and touch and feel and your sort of physical presence in relation to your virtual presence and how can we play with those things together and I think that's going to evolve into making some interesting demands on the technologies and tools we build. You know there's a very interesting robotic platform called Birdly where you lay in it and then you have a direct mechanical relationship to a very magical experience of flying. I think those kind of things are going to be part of it. I think we just have to keep our eye open on the full breadth of this and not go directly into like how quickly can we lose people in this fully immersive space so that we may never get them back.
[00:13:06.013] Kent Bye: What do you want to experience in VR then?
[00:13:10.035] Alex McDowell: You know, the Ocean project is very compelling to me. I really, really want to be able to go down to the depths of a submersible and I want to be able to change scale and I want to be able to move through time. So, coming from cinema, the things that I cannot do in cinema is I cannot change my relationship to time and I cannot change my relationship to space. other than through the camera and I can't change my relationship to scale. So I really am interested in kind of really pushing at all of those capabilities. So how can I reduce myself to the scale of say plankton and understand an entire new type of creature. 90% of the animals in the world are plankton and we don't have any comprehension of what they're like. So I think I'm increasingly fascinated by the real world aspects. We're now just about to, I'm going to Berlin right now to do a deep dive into how we could look at the refugee camp and all of the issues around refugees and migration and massive urban growth and how we could use these capabilities to look into the future and kind of extrapolate forward and start telling narratives that might help shift those futures. And I think that's another huge part of this is that
[00:14:15.558] Kent Bye: With this compelling a medium the kind of stories you tell can literally change people's lives And finally what do you see is kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:14:29.442] Alex McDowell: Well, I think that, you know, what I was just saying touches on a lot of that. I think this idea that you can get more deeply immersed and get into a closer relationship to story space and I think return ourselves to, as I was saying at the beginning, to a direct relationship to our world. We've gone through elaborate processes and translations to cater for the lack of capability of the tools we have. You know, we've converted the imagination in our minds into lines on pieces of paper, plans and elevations and sections that we then have to unpack into three-dimensional objects to make architecture. It's a very complex and elaborate process that people spend their entire lives learning how to do, which is really unnecessary actually because what we really want to do is pluck our imaginative ideas out of our brains and express them in the real world. What I'm really interested in is the combination of practical things like rapid prototyping, like the digital and virtual technologies that can drive AI through algorithms, through procedural design processes, to an immediate impact in the real world. I'm really interested in real-time, I'm really interested in collaboration, as I say. And I think I'm interested in this merging of media. I strongly advise against creating another silo, you know, the VR industry. I think we have to think about this as a place to import best practices from all these different industries and actually look at a new kind of production methodology, kind of creative and scientific engineering methodology that can be pushed into whatever part of the media space it is most useful and where it has the most impact. Great. Well, thank you so much. Thank you. You're very welcome. Great talking to you.
[00:16:08.299] Kent Bye: And thank you for listening. If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash voices of VR.
