I interviewed David Attenborough’s Conquest of the Skies director Lewis Ball at Venice Immersive 2023. See more context in the rough transcript below.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. It's a podcast that looks at immersive storytelling, experiential design, and the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So continuing on my series of looking at the different immersive experiences from Venice Immersive 2023, this is episode number 11 out of 35, and the third out of five of looking at the context of place and home and environment. So this piece is called Davin Attenborough's Conquest of the Skies by Louis Ball. And this is a piece that's actually available. You can go check out on the MetaQuest TV. Just search for Davin Attenborough or Conquest of the Skies and should be able to come up. It originally started as a 2D documentary and it's kind of remixed into this new spatial context where It's a mix of different stuff that's shot natively and stereo 180 virtual reality. There's some stuff that was originally shot in 3D for 3D TVs and then work to take that stereoscopic effects and put it into a virtual reality context. And then you have some stuff that's just purely artificial intelligence driven to be able to convert it into more of a spatial context. And so it's a really well-told three-part story series that's looking at insects and then reptiles and then the birds and how each of them have defied the odds of gravity and evolved through the course of being able to fly through the skies. So really quite a compelling experience and looking at this kind of nature documentary and 180 video that I think works really quite well. So it's looking at the context of environment and evolution and also just broadly an educational type of experience. And so it's got a lot of pedagogical aspects to it. So, overall, I think the Center of Gravity is looking at this kind of mental presence because it does have that educational, pedagogical feel. And then there's also this sense of environmental presence as you're going into these different spatial contexts for these birds that have a lot of reconstruction. There's actually just a lot of different techniques that they're using. Compositing flying dinosaurs and taking natively shot stereoscopic imagery and then others, just pure CG renderings, and then taking some shots of what's happening in a studio and doing a lot of movie magic in order to create this really compelling immersive experience called David Atborough's Conquest of the Skies. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Louis happened on Friday, September 1st, 2023 at the Venice Immersive in Venice, Italy. So with that, let's go ahead and dive right in.
[00:02:36.388] Lewis Ball: My name is Lewis Ball, I'm an immersive director. I write and create interesting visual projects and I try to tell interesting stories in our new formats, headsets being the main ones.
[00:02:49.815] Kent Bye: Maybe you could give a bit more context as to your background and your journey into VR.
[00:02:54.699] Lewis Ball: So, I was always interested in film, and I started my career in film, and I would be working on film sets, and I was quite young, and I was hungry for more work, and I kind of found myself being introduced to the world of 360 shooting. You know, it was a wild time when the first headsets were popping up, where we had some things like the Gear VR. because anything could be a VR film in those days, like World's First Roller Coaster was a viable pitch for a project. So I found myself traveling the world, filming all sorts of stuff, under the guise of a camera operator. And as I became more familiar with the tech and the communication it offered, I started devising my own creative concepts around that.
[00:03:40.866] Kent Bye: Great, and so what was some of the first projects that you started to work on?
[00:03:44.449] Lewis Ball: I was working for a company at the time, so often it would be commercial projects. A lot of documentary as well. We worked really closely with some really talented directors like Mary Matheson. I was a director of photography for a lot of her projects with Surround Vision. Google Female Planet was really interesting where we got to use different techniques to try and make fresh a template that we constructed for the series. So every episode we shot for five episodes would be a little bit different than the previous one. So another stuff would be nature. I was really fond of nature documentaries. I always thought like virtual reality was a good medium to explore the animal world and nature because a big part of me is really attached to like artistic mediums and things that are deeply personal to me. But for something that has universal appeal I think at the animal world and nature really offers that to everybody because It's rare for us to get so up close and personal to a lot of what nature has to offer and as it disappears it will get even more difficult. So I was really interested in nature. I made a documentary about animal whisperers and one episode of which was about an elephant whisperer and that was quite an important project for me as it helped me grow as a filmmaker. and really devise new methods of visual communication, and especially in shooting one thing and learning how to shoot around animals and whatnot, but then taking that and trying to construct a story through what are sometimes lucky moments you get when you're filming nature.
[00:05:17.955] Kent Bye: So we're here at Venice Immersive 2023 and you're a part of the Best of selection with a documentary series that you've been directing that was featuring David Attenborough. And it's my understanding that there was a previous series that was also from last year. When did these kind of bigger projects start to come about when you were exploring this intersection between nature documentaries and VR? If this is the first one or if you worked on ones before this?
[00:05:42.705] Lewis Ball: So the VR world, as I'm sure a lot of listeners will know, it's a small community but we all kind of get to know each other and I've been around and I got to know Atlantic Productions and Alchemy and through a connection they already had this idea going where they'd like to create this story that they had. and Meta were super interested in this to distribute. So that's kind of how it was just the right time and right place and right kind of experience that I could offer. So the stars just aligned for everybody really. I know Alchemy have loads of experience with similar nature projects, but I think for this instance we were really looking to go like above and beyond. try and shake things up from the previous series that they'd offered, you know, offer some more different creatures. We've got lots of dinosaurs, lots of CG scenes, things that are kind of new for this series specifically. So the idea was to go the extra mile, go above and beyond, and let's offer viewers something new.
[00:06:39.191] Kent Bye: Was this your first series or did you work on the one that was also with David Attenborough as well?
[00:06:44.233] Lewis Ball: No, this is my first Attenborough series. This is my first project with Alchemy & Atlantic.
[00:06:48.976] Kent Bye: OK, so this is the first project that you've been working on. You talked a little bit about the relationship with the production company, but how did the specific topic come up to decide about flight, that this would be a focus?
[00:07:05.058] Lewis Ball: I think coming into it you're looking at the way something like this is made is that we're looking at there's a little bit of reverse engineering in that let's look at the material we have and let's look at what we can take from this archive of stuff we have and what we can add to it to create a cohesive story and I think What I love about this project is that it's about evolution. I think it came at a time in my life and at a time in Alchemy's life and the company's life where we were looking at it and thinking, how do we grow? How do we go further? And I think everyone involved was of that same mindset. So that's what happened. We're trying to tell a story about growth and how to expand beyond what is expected. And the Conquest of the Skies is all about nature unshackling itself from gravity and flying.
[00:07:51.751] Kent Bye: So yeah, I watched David Attenborough's Conquest of the Skies at home ahead of Venice Immersive, and I have to say that the navigation of the player for meta is a little confusing, and I don't even know if I watched it in the right order. I sort of watched it in some order, but I was like, wait, I may have watched it in a jumbled version of it, but I'm curious, what is the recommended order, first, second, or third, or can people really watch it in any order? So I'd love to hear you break down what was in episode one, two, and three, just so I can be a little bit more clear in my mind of what the proper order should have been.
[00:08:27.043] Lewis Ball: So it's funny you say that, because people have meetings about this stuff, and we've spent hours talking about how do we name these episodes and how do we present them, because we know that the distribution method, which is Quest TV, is particular, and it has its own limitations in how it presents to the viewer. So we knew that we didn't want episode numbers pretty early on. We just wanted to have episode titles. So the correct order in which to watch this, if there is a correct order, they've been designed so that you could pick any episode and enjoy the story and not feel like you've missed out on something. But the best experience would be to start at the very beginning of the evolution of flight, which would be with the insects. So you'd start 320 million years ago with the vanguards of flight. So you'd start with that episode of Vanguard, you'd watch the insects evolve the flight, and then you'd move into reptiles, which are the rivals to the insects. So the idea being, who will achieve this conquest? So we've got the Vanguard and we have the Rivals, which were the reptiles that took flight after the insects. And the final episode is about birds, who were the victors. So they are who dominate the skies today. So the correct order is Vanguard, about insects, Rivals, about reptiles, and Victors, about birds.
[00:09:46.746] Kent Bye: And is the reptile one, is that where they have like the dinosaur animations in it as well?
[00:09:51.767] Lewis Ball: Yes, yes.
[00:09:53.279] Kent Bye: Think I actually watched that one first and then insects and then the birds So I felt like I had ended where I needed to be but I felt like oh when I was watching the second What was my second episode? You know, I was like wait, I think I maybe should have watched this first so I think that speaks to in some ways the confusing nature of some of the oculus TV things because I would love to be able to just click a link and then watch them in order or if there is a preferred order and But even to be able to go from one episode to the next sometimes is like more difficult than I thought it should have been. Anyway, that was just my side comment on the distribution mechanism of the software limitation. But yeah, I don't know if you have any reflections on that.
[00:10:27.082] Lewis Ball: You're right. There are so many moving parts to a company like Meta, so I don't know the whole picture. And none of us do. So what is the correct way on a global scale with so many different types of projects? And there are a lot of gifted individuals there who are thinking about this all the time. So I don't know. All I do know, it just caused us to think about things a little bit differently. So in an ideal world, I guess you'd want to tap onto the series and see each episode in succession, almost like other streaming platforms. The feeling I get is that with immersive content on the Quest TV platform is they're trying to really mix things up. I think they want viewers to see something and then have a diverse range of other options to choose from in the things that are being suggested to them next. So there must be some thought put towards it.
[00:11:15.625] Kent Bye: Yeah, I think even when I watched that episode, I didn't even remember if it was recommending like the next episode or not. Certainly with the Tales from Soda Island, I had that problem. But anyway, that's more of a distribution logistics thing with meta, but I wanna maybe get back to your actual experience because the thing that I found really amazing about the piece was just the micro stereoscopic center photography that's in this piece. There's a lot of real amazing shots of nature and a lot of shots that are like, I was just thinking to myself, how do they capture this? Because it's such a unique moment of having to shoot a bunch of stuff, or being at the right place at the right time, or other moving magic to sort of construct different things to happen in a way that feels organic. It's tough for me to know exactly some of the different moments that are just like, wow. in awe just of the moment that you're capturing that fits in a larger story, but also just the technical aspects of being able to do this kind of small scale micro cinematography in a stereoscopic way that was taking something that's a lot smaller and really blowing it up in a way that takes me into another scale of perception that goes above beyond what I can see as a human scale. So let's maybe start with the technical aspects of the technology. Like what kind of tools did you use to be able to get some of these micro scale shots?
[00:12:30.728] Lewis Ball: So we always knew that we wanted to, the whole concept of this series is to share the perspective with these creatures and the diverse range of creatures means a diverse range of technology we're going to use. So there's never one standard camera setup that we could use throughout the series because we're using a bunch of different techniques. It's a mix of good editing, so a glance here might cue a shot there, whereas an owl just glancing anywhere is a cue for a good cut. We can intersperse that with some footage shot separately of a mouse in the field and make it look as if One is hiding from the other and one is hunting the other. These are really simple tricks that are very easy to fall into. It tells a story very clearly. And there's a lot of stuff captured in studio. Dragonflies will be captured being held in position where they're flapping their wings so we could get slow motion shots with a green screen behind them and then composite our background. behind it. So there are lots of neat tricks and it would be very time-consuming to go into each and every one of them. But you'd be surprised that a chunk of it is shot in the studio, like the super up-close macro stuff. Whatever we can get away with in the studio is preferably shot in the studio.
[00:13:47.527] Kent Bye: Okay, well, I guess knowing that it was composited explains a lot because it was like, you know, to actually shoot a nature documentary and get some of the shots, I was like, oh wow, this is really impressive. And I don't know if that's like nature documentary ethics thing where like if you're watching a 2D version of how much of the same type of composited trickery happens already in nature documentaries that I just don't know about, or if this is something that is unique to VR because it's a new medium, then there's like a way that you have to adapt to how you're shooting it in order to Capture some of these different moments because I was just assuming oh, this is a nature documentary This has to be being shot out in nature, but I don't know if this is something I've just been tricked and fooled by the 2d version all along
[00:14:28.662] Lewis Ball: I'm a humble guy, right? I don't know the ins and outs and what is correct and socially acceptable in this whole sphere, and nature documentaries have been around for decades. But I will say that most of the animals you see in nature documentaries, they're probably recognizable by other documentary shooters. I'm sure there are only a handful of owls and a handful of lions on camera that have been captured on camera across the world, because these animals are known. And Atlantic is obviously a really rooted company, so Alchemy and Atlantic were bonded, and Atlantic has lots of roots in nature documentary in the past, so it would be not only a good thing to utilize that experience and try and take what was learned and captured back in those days of 2D capture and enhance it for today. So a lot of cool things we're doing. We might be using techniques from before and tricks from before, but we are thinking about it in a brand new way and trying to distribute it in a brand new way. So everything's being thought about.
[00:15:29.080] Kent Bye: There's a shot that also comes to mind that's in the first episode, which is there's a fly that's coming off and it does a complete 360 turn. Is that something that flies normally do, or is that just one of those shots that's just like, I can't believe I just got a fly doing that?
[00:15:44.747] Lewis Ball: Apparently it is something that they do fairly often. The sophistication with which they fly is really impressive. And it's kind of one of the things I like about the series is that it shows you such a variety of different creatures from like the really peculiar ones to the super commonplace ones. And just to see something so interesting happen in such a commonplace creature that we're so familiar with and probably hate for the most part is one of the things I find cool about the series.
[00:16:13.604] Kent Bye: Well, it's also unfolding like in a slow motion fashion where I'm sure you're shooting at a higher frame rate. Is that correct that you're like how many frames of second were some of these different slow-mo shots that you were shooting in order to get this slow-mo effect of some of these things that normally when I'm seeing these insects fly, I'm seeing it just Whatever the normal frame rate that I'm perceiving, but if you increase that with the capture of these devices Then you're able to see it in this kind of really magical slow motion like it's not only bigger in the scale But you're also perceiving it in a way that maybe with some of these predators Maybe they have the frame rate of their perception is actually that's how they're perceiving some of these things like that's for them that to be normal, so kind of in the Marshmallow Laser Feast in the eyes of the animal where you're trying to adapt into another mode of perception with these different techniques. But yeah, I'd love to hear about any of the slow-motion technical details for the different ranges of frame rates that you were using to capture some of these moments.
[00:17:10.077] Lewis Ball: It's interesting, isn't it? It makes you wonder what time is. Because those flies must be perceiving time differently to us when they're doing such acrobatics. In terms of frame rates, so the film is delivered at a frame rate of 60 frames per second because we don't want to really dip underneath that. Some of the material ended up being 30 frames that we increased for the final deliverable. But for the most part, all the slow-mo would have been shot at 1,000 frames per second. and slowed down. In the original script and in our voiceover there were details of how much we slowed the camera by and the shot by for the viewer. But I think because in a headset, especially in a headset, overloading the viewer with information, especially like quantities and numbers, I always felt like it diluted the content a little bit. It's enough for me to know that it's super slow, it's enough for the viewer to know that it's incredibly slow because There's not much of a difference between a hundred times slower and a thousand times slower when I'm hearing it I can't perceive the difference anything beyond a certain point. I don't appreciate the difference rather
[00:18:14.176] Kent Bye: Yeah, I guess for anyone who's like fans of the slow-mo guys on YouTube, I know there's different cameras that shoot at different frame rates and there's a bit of a trade-off between as you increase the frame rate then you actually decrease the field of view or you have to have enough lighting to even be able to perceive things. So yeah, there's just like technical considerations for shooting at these different frame rates means that you have to have these other ways to compensate for the different trade-offs that you're making. I don't know if that's something you ran into as well.
[00:18:39.493] Lewis Ball: Absolutely, and this is why it's so important, having flexibility and having control over the shooting environment. Slow-mo guys are brilliant, but every aspect of what they're shooting is always controlled in terms of they know exactly what they want to capture in exactly the right time. And it's also costly as well, all this data you're capturing. Hard drives might be cheap, but the amount of data you're capturing is exponentially higher than traditionally we might be capturing for a normal project. So, it's always a challenge and I think this is why the studio setup is quite important for a production like this, just to get those details. So, philosophically speaking, I would love to just hike around Peru with my little run-and-gun system and capture everything, but it would be a completely different style of film, it would be a completely different style of project, and it would not be what we're seeing today.
[00:19:32.884] Kent Bye: Yeah, I was really quite compelled to have this perspective shift and the different perception of time of each of the different episodes has its own moments of awe and wonder of being able to see something that I've never seen before. And I think that's what the real strength of the series is, is to go through the evolution of flight. And there is this progression to the piece where you're going through this evolutionary trajectory. And in the second episode, you start to do a little bit more techniques with shifting over to the CGI, but also some more It's not necessarily mixed reality, but more of just like visual effects compositing where you're shooting a stereoscopic view of a forest and then you're also adding these CGI dinosaurs that are flying through in a way that also is kind of matching the stereoscopic effects. And so there are certainly moments where I was like, wait, is this a CG piece? Is this all CG or is that forest looks really good? It looks too good to be CG. And so as I was watching it, there was this moments of trying to like, both appreciate the storytelling of what I'm seeing, but also like trying to parse through like the production process of some of these things. And also in watching the video that you produced for the Venice Biennale, talking about the project that, you know, it was confirmed that there was these kind of elements of mixed reality, but there seems to be like a fusion of these different types of techniques of taking what's available with the ability to do like a stereoscopic 180 with these new lenses that have been on the market within the context of the last two or three years or so to be able to get these really high fidelity shots, but then to also blend it in with other aspects of the CGI visual effects from more of the film world. So yeah, I'd love to hear some of your thoughts on the expansion of these different types of mixing and matching of all these different modalities to be able to tell the story.
[00:21:13.280] Lewis Ball: It's a really good question, and I was just thinking as we were talking about this, it was really challenging. I remember having these ideas, and we were working with a ton of different material, and we wanted to tell a cohesive story. And there were beats we chose early on before production. We knew there were certain dinosaurs we wanted to show, and we had a much grander shot list for the beginning. But this compositing is quite challenging, and it goes through a lot of iterations. I knew we wanted to keep it as real as possible so when we could capture an environment and composite CG dinosaurs in. I guess from the offset it makes sense to say that we wanted to keep it as real as possible. So even though priority is telling a good story and that's why we're not against using something shot in a studio or using a CGI dinosaur because we need those images to tell the story the way we'd like to. And then it comes to, well, shall we create the environment from scratch or shall we composite? So the priority is always to use as much as we can. We want to capture as much authenticity and reality as possible and keep it real as much as we can. And our shot lists for the original scenes, you know, we had triple the amount of shots we do in the final film. There'll be a few more locations, a few more different angles. And it's just very challenging and time-consuming to get everything feeling like it belongs in the scene. So it's one thing to capture the scene and capture a good space to composite on, but when these creatures are moving around so much, which they do because we're trying to showcase how they flew and how they interacted with their environment, it became apparent that we needed to cut down and really nail the main beats that we were trying to hit with the story. So we did have to make some compromises, but good ones so that we could really give it all we've got for the shots that we kept in there. I do think there are scenes where we just had to completely CG environments. And again, it's usually these decisions are often made like there's some creative leeway. So as a creative, you'd let you have your own ideas of how things should go. But as a well-behaved creative, you'll also try and take into consideration the production and how much time and resources we have to dedicate to certain aspects. So there's only so much time as a whole that we can dedicate to the project as a whole team Because we have a delivery date and we have targets to hit and this is one of the interesting things for me like coming from a documentary background also commercials are much more smaller and short and fast-paced and whereas this was made over a lengthy period of time. It's one of the longest projects I've worked on on a larger team, so you had to be mindful of how resources were being spent. And the CGI was very time-consuming to get it to look right. Things might look good from certain angles when they're composited, but if they move further away, sometimes the size of these creatures would disproportionately Change so that something would feel off, but you couldn't quite put your finger on it until the penny dropped one evening So did you have to like in order to get that perspective right?
[00:24:13.836] Kent Bye: Are you modeling that in 3d and then having that fly in or you just eyeballing it to make sure that it looks correct?
[00:24:21.048] Lewis Ball: Very limited modeling, but yes, you need the horizons to level out and you need everything to match on a scale, but we try to limit the amount of areas that the creatures would pass behind. So, for example, if there's a tree or a branch, we would alter the motion so that if it helped us, if it was going to save us a few weeks, we would alter the animation somewhat just to help us out. because at the end of the day this is a project with a deadline and this is one of the things I came to appreciate. There are lots of different angles of filmmaking. It's an amazing thing to just be creative and just make what you want but this is also a separate kind of amazing where you have a lot of resources and if you spend them wisely you'll nail it. It'll look beautiful but you have to spend the resources wisely. So we could have had a longer Quetzalcoatl scene. We could have seen really up-close macro 16K renders of its eyeball, but we would have lost other parts of the story. So it's a balancing act, I find.
[00:25:16.793] Kent Bye: And when you're talking about these shot lists and having additional shots, are these shots that you actually ended up shooting and then not including during the editing process? Or is this something that you had just stopped it at the pre-production phase of not even shooting it and just kind of like working with what you got?
[00:25:34.403] Lewis Ball: No. Going into production we knew exactly what we wanted. Pre-production was our kind of experimental phase. I made some storyboards. I'm not a very good drawer but I was able to mock up some stuff for the art department to really understand what we were going for and there was a few weeks where we would go through iterations of what the entire segment would look like within the episode. and eventually it was just like we just all agreed on a happy medium and it was very mindful like that mindfulness is something that everybody in the team preserved throughout so there is room for experimentation but in the production phase we never misstepped really.
[00:26:12.295] Kent Bye: Yeah, and as I think about the third and final episode about the birds, I have some really spectacular shots of this snow owl. And I learned a lot about the feathers and how silent they were when they were flying around. And so how did you get these shots of this owl flying around?
[00:26:28.319] Lewis Ball: So the owl flying around was quite lucky because Atlantic, being the kings of nature shooting that they are, they had a lot of material already captured of the snowy owl. So what we had to do was go dive deep into the archives, which is no small feat really, like digging up archives of a shoot from years ago and understanding how we can make it work. how to make it stereo, how to make it work in a 180 space. This is something very new. So that's an example of something that we took that really, for me, a VR person, VR for me is something I capture with a 360 camera. Because I came into it with this mindset. I take my camera, I shoot, and I get a 360 sphere. And then I learned about interactivity and game engine stuff. But then to think about using stuff from years ago and trying to figure out how we could immerse people in it, that was a new type of thinking. And I'm glad that it seems like you didn't really catch the trick. So that's one of the cool things about it, is that we got to dive into this material and reassess it completely. Say how do we tell an immersive story? How do we make people feel like they're immersed in this world? I think that's one of the biggest takeaways I had was that it's not the capture method that dictates whether this is immersive or not it's really the delivery and in the same way in a project doesn't need to use every degree of the 360 degrees around you at any every given time and It really doesn't. Immersing is about taking a person, drawing them in, and presenting them something in a way that they feel like they belong there, that they're present. It's about offering presents. So, yeah. The snowy owl, and there are a couple of other examples where it's archived material, and it's like a little secret.
[00:28:18.013] Kent Bye: Okay, well that's why I was asking about being in awe and wonder that this was captured in stereoscopic and there's some compositing that's happening and there's also digging into 2D archives and making it stereoscopic and through a post-processing effect. Ruining the magic in some ways, but as I was watching I was like, how do they capture this owl in stereoscopic effects?
[00:28:37.638] Lewis Ball: I think that's just, it was like a wake-up call for me when I was able to look at material that had been captured in 16x9 and realize, hang on, the sky is dark and I could like fade out the darkness to the darkness of the edge of the sphere of the heads. You know what I mean? Like when you're thinking about it differently you realize that I don't need the rest of the sphere for this part and we had an amazingly talented compositor working on this with us and he's able to devise like completely groundbreaking techniques to make this really work in a headset and go really above and beyond and really trying to enhance the material that was shot in not that many K and really holds up in a headset in 2023.
[00:29:20.984] Kent Bye: I know there's Oz Balabandian. He took a drone shot that was moving laterally across the space looking at like these buildings, but then realized that as it's flying in a straight line, he could take a shot from this perspective and this perspective and then have this stereoscopic effect from a singular monoscopic camera. But because it's traveling in a straight line, you have this stereoscopic effect. So as long as the scene's not changing drastically from when it's moving from the 60 millimeters or whatever the IPD that they're using to create this then he was able to fake this stereoscopic effect just by having that camera movement so I'm imagining there's gonna be a lot of techniques like that where there's a moving camera or there's gonna be just using AI and other generative media to start to create the spatialized effects of that and But I'm curious to what percentage of this film, was it all done in that type of compositing technique of taking a monoscopic shot and making it stereoscopic? Or was there actually using some stereoscopic cameras to be able to shoot some of this?
[00:30:22.641] Lewis Ball: No, so the monoscopic captured material would be, in terms of percentage, probably 5 to 10% of it was probably captured monoscopically. Phantom, for example. So getting the two slow motion cameras synchronized back in the day when they captured wasn't really possible. So what they did was capture monoscopically and then in post-production made stereo, those images. and about 50% of it would be stuff that we've captured, and the remainder would be stereoscopically captured material. That wasn't necessarily, so the Barn Owl would have been captured in 3D, but 16 by 9 3D, so it would be stereoscopic for television. This was the selling point of this original series that Atlantic produced back in the day.
[00:31:09.600] Kent Bye: So they were actually producing for 3D TVs or something like that?
[00:31:13.074] Lewis Ball: Yeah, this was one of the main selling points. I think this was one of the first Attenborough series of its time in 3D and it was a Sky series. So Atlantic produced it for Sky. And then we have our intro sequences. So the intro sequence is a few drone shots and time lapses and our intro sequence especially where In each introduction starts with CG and it kind of lulled you into this universe and to give you a sense of what's to come a feel for what's to come it starts very low and I think the introduction for each episode follows the same template even though they're all unique so you'll be the slow crawling moving shot and It will transition into the real world So for the first episode you're quite low down by the riverbed, as you might be a dragonfly, and that transitions to the real world equivalent of that, where you're just flying over the river. Episode 2 is in that you're under the canopy of the trees, like a dimorphodon dinosaur, like the early flying reptiles. And then we transition from the 3D into the real world forest. And for the final episode, we're soaring above the trees. And they each have different methods of capturing that motion. Yeah, loads of different techniques. But to answer your question, the monoscopic archive material will be about 5-10% if I remember correctly, and then about 30-40% of stereoscopic 16x9 material that we enhanced to make it immersive, to fit our 180 sphere. And that involves growing the area that's captured, so it's really precise compositing techniques. and it's feathering the edges and making it feel like tricks where sometimes we had shots that were panning from left to right. And as you were saying earlier, that trick of like taking the subsequent second and adding it to the previous, you know, this trick of making more of the image than what you have based on the latest sections. So there was a little bit of that going on as well to try and capture a larger area.
[00:33:13.448] Kent Bye: And so in terms of the 50% of stuff that you had shot fresh, how many different types of lenses were you using to be able to capture? Because you have different scales, and if they were custom lenses, or if this is custom off-the-shelf things, or if this is something that's part of these production companies' thing is to produce the unique technology to be able to even tell these types of stories.
[00:33:33.644] Lewis Ball: The rigging would be custom, but the lenses will be tested, and I wouldn't say standard, but professionally standard lenses, and they'll be paired up with their stereoscopic partner.
[00:33:44.187] Kent Bye: I know there is an SLR camera that had one lens that had a stereoscopic, but you're just using two cameras, essentially?
[00:33:50.509] Lewis Ball: No, for the moving shots, so for our moving intros and for our transition shots, we used a R5C, so a Canon camera, with their RF mount. 5.2 millimeter fisheye lens, so it's two lenses that fit onto one lens that you hold, and it has two lenses and those two spheres are captured, or those two circles are captured onto one sensor. And from that you can attain, if you have the correct recording kit for your camera, you can get an 8K 60 frames per second little setup, which is really helpful for moving shots, because if you'd like the system to move, having it lightweight is really helpful, and that's what we were trying to do.
[00:34:32.865] Kent Bye: Okay, and the 60 frames per second is an interesting number because a lot of what the Quest tries to go for is either 75 or 90 frames per second. So you're at a little lower frame rate, but just like you can watch TV at 30 frames a second, or sometimes up to 60, but the film has like 24 frames a second that has this cinematic quality. Do you think that you're able to get away with a lower frame rate because we are trained to have this film look that comes from like a lower frame rate?
[00:35:01.552] Lewis Ball: I think the cinematic look is something specific for cinema. It's even in the word. I think there's a lot to take. They are just different mediums and there's a lot to enjoy and appreciate and there's a lot to learn. I don't think we should dismiss any of the things that cinematographers have learned and the things that we've learned over the years through filmmaking. But there are other things that we shouldn't be so dogmatic about so frame rate like I have an idea as well like we have to be 24 frames a second or like 180 degree shutter angle must be preserved throughout but in practice if increasing the shutter angle means I can get more light to hit the sensor and It's a good thing increasing that image quality is better than preserving this dogmatic idea of what cinematography and what is cinematic 60 frames we would have gone for higher but we weren't capturing anything in a higher frame rate and the higher you go with the quest you're Diminishing the bitrate and the resolution as well. So it's always a trade-off and 60 frames Resolution we delivered in a 6k is workable
[00:36:06.093] Kent Bye: Yeah, I guess I was less advocating that they would have like a 24 frames per second, but more of how in film it is 24 frames per second, which is lower than like other frame rates. But, you know, most of the VR experiences on the Quest are like a minimum of 75 or 90. So the fact that it's a little lower and that it was OK, I feel like is it's already lower with film is a little bit lower. So, yeah, that was just more of the thought.
[00:36:29.927] Lewis Ball: It's encouraging like I just think the story like people will forgive anything if it's a good story And if the sound is really absorbing you like we're talking a lot about the visuals But the sound really is what ties the whole thing together like we've got the music really lulls you in and it's comfortable and it Fits like a glove everything just works and the sound design is just incredible like you feel immersed because you're hearing every swoop of the wing and It's not noisy. It's just expertly crafted. So the visuals alone Maybe if you were watching it without the sound and it was silent. Maybe you would think that maybe you think maybe you would pick up on the slower frame rate or the slightly jagged edges of some of the wings and some of the old material that we've repurposed and But I think when it all comes together and when you're telling a nice story, that's what sells it.
[00:37:21.086] Kent Bye: Yeah, I agree a hundred percent, especially with this piece. I think it's, yeah, I just was really engaged and immersed in a piece and I really appreciated the story. So yeah, what's been the reaction in terms of meta? There's been a lot of change ups from their layoffs and everything else. And so like, how's the reception been to the piece and is it meeting their metrics or were they happy with it or any feedback that you get with how that was received by the wider community?
[00:37:45.988] Lewis Ball: Everyone I've worked with at Meta and everyone who talked about the film, the project with us, has been lovely. They've been thrilled. It was very collaborative. We would have weekly meetings where we'd discuss all the progress, even when there was very minimal progress. Even with me just sketching pictures onto a piece of tissue, they were always involved and they were really part of the whole process of bringing it to life. Those are really good suggestions and obviously they have their expertise on what they think can enhance a story in an immersive world. And I don't know, I just had a really good time speaking to all of them. I don't know about changes in meta and whatnot. I do remember it causing a bit of a kerfuffle somewhere in the middle of production. But in terms of our project, nothing was ever set back in any way. So we were quite lucky. Or we may have been quite lucky. I don't know about other projects. And the reception in general, like we're here at Venice, that's really good and everyone I've spoken to here seems to be enjoying it and that's really pleasing for me. Like it's weird for me because I spent nearly a year on this and it was my life for a long time. I would wake up and think about this and fall asleep to dreaming about it. And I haven't thought about it since we delivered in December last year. And then coming back here two seasons later, it's a little bit surreal. I did do this. It's a bit weird. But I think, you know, obviously, I can't take too much credit for it. Like, the team was just amazing. And you've already got a legend. Like, you've got a national treasure telling this amazing story. Like, my job was pretty easy, to be honest. I just had to... show up in meetings and draw a few ugly pictures and all the talented people just made it work.
[00:39:26.811] Kent Bye: Awesome, yeah. Yeah, I guess as we start to wrap up, I'm curious what you think the ultimate potential of virtual reality and immersive storytelling might be and what it might be able to enable.
[00:39:41.994] Lewis Ball: That's a good question. Well, obviously the potential is limitless and anything is possible. I think the strengths of virtual reality and immersive storytelling lie in the actual storytelling. I love feeling engrossed and pulled into a story and it might be this like traditional mindset that I've learned and I just love the idea and then maybe I'm just too attached to the ideas of filmmaking but I just love having a journey and I love getting up close and personal to a person or a subject or just something just offering me some poetic insight and using like tools that we've seen before like it's just about storytelling and I think there's so much we can benefit from like looking at previous films like having a visual motif and Introducing it and then reintroducing it and then ending with it and you know seeing it progress I just that's what really works for me. Not like I love things coming neatly Organizing themselves to tell a cohesive story so I think the potential is genuinely limitless and I think if we can somehow like I unshackle ourselves from all kinds of expectation and really understand the creative limits, but also appreciating the expertise and the wisdom of storytellers before us and trying to incorporate those ideas into our own mediums. I think there's something special on the horizon.
[00:41:10.054] Kent Bye: Great. Is there anything else that's left unsaid that you'd like to say to the broader Immersive community?
[00:41:15.648] Lewis Ball: Yeah, I'm really proud to be part of this community. We're at the frontiers, right? This is great. Everything is so new. Everything you see is so different. It's a small community, but everyone's passionate. Everyone's really creative and multi-talented as well. And I'm just super proud to be like... Just walking around Venice these past few days and seeing some great stuff, it feels like a band of misfits has come together, you know, found a new way of expression and it's up to us and I really like that. Especially you, Kent, like you're a very peculiar cat and I really like that about you.
[00:41:53.460] Kent Bye: Awesome, well I really enjoyed the piece and it's available on the quest platform so I highly encourage people to go check it out and yeah I think it's a just a really great story that's well told and like I said taking you into all these different perspectives and shifts of time and as you travel through time as well so yeah hope to see more of this type of work as we move forward and yeah thanks for taking the time to help unpack both your process and some of the Mystery of the techniques and all the technologies to make it all happen So thanks again for joining me here on the podcast. So thank you.
[00:42:24.894] Lewis Ball: Thank you. Thank you. Thank you
[00:42:26.405] Kent Bye: Thanks for listening to this interview from Fitness Immersive 2023. You can go check out the Critics' Roundtable in episode 1305 to get more breakdown in each of these different experiences. And I hope to be posting more information on my Patreon at some point. There's a lot to digest here. I'm going to be giving some presentations here over the next couple of months and tune into my Patreon at patreon.com slash Voices of VR, since there's certainly a lot of digest about the structures and patterns of immersive storytelling, some of the different emerging grammar that we're starting to develop, as well as the underlying patterns of experiential design. So, that's all I have for today, and thanks for listening to the Voices of VR podcast. And again, if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listen-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
