Play episode

#1556: “Currents” Boldly Defines a New Era of DIY 180-Degree Immersive Filmmaking with a Kit Under $5k

March 29, 2025by kentbye

CURRENTS is a stunning debut from Jake Oleson as it pushes the grammar of 180 filmmaking to new heights. Oleson comes from the world of advertising and music videos, and he went through a rigorous pre-production process where he had actually already edited the entire piece before shooting any production footage by using still stereoscopic test photos and videos. The insights from this pre-production animatic are evident, as there are some of the most stylized movements I’ve seen, and he managed to not trigger any of the common motion sickness triggers. There are some really powerful cuts that seamlessly juxtapose different environmental contexts for dramatic effect.

CURRENTS tells the story of a young woman who leaves her Vietnam rural countryside home to the city seeking more economic opportunity. Aside from a short exchange with her mother at the beginning, the rest of the entire story is told through spatial storytelling, driving electronic music that was also composed by Oleson, and a shader-filled point-cloud scene with some animated motion capture. It’s really quite a beautiful piece that gave me a glimpse of the next phase of cinematic immersive filmmaking.

It’s also worth mentioning CURRENTS in the context of other musical experiences since Oleson is both a musician who has a background in music video production. The music was specifically composed to serve the narrative in this instance, but it will be interesting to see more and more immersive music videos start to crop up now that the Apple Immersive video format is starting to get licensed out to folks like Vimeo, who commissioned this piece.

Apple’s push for 180-degree immersive video has brought up some broader discussions about the merits and downsides to 360-degree films. From a creator’s point of view, 180-degree filmmaking is a lot easier to do logistically as it is closer to existing production pipelines, which is brilliantly demonstrated in CURRENTS.

It’s also worth noting that CURRENTS has a point-cloud scene that shows up in the middle of their cinematic 180-degree immersive video that represents a key turning point in the story and journey of the main protagonist. There’s also some really poetic shaders that are translating the static scenes into even more of a dynamic visual representation of the business and chaos of the city. It has quite a narrative impact when juxtaposed against the photorealistic footage and custom-written and driving musical score.

Even though the Apple Vision Pro has been out for over a year now, SXSW is still an opportunity for folks to check out the Apple Vision Pro for the first time. It was probably easiest to check out on CURRENTS who had six different headsets and was using the Sandwich Vision Theater app in Kiosk mode to seamlessly jump into the experience bypassing the time-consuming eye tracking calibration step.

I was also particularly interested in Olson’s production pipeline, the camera gear and kit costing less than $5k (Canon EOS R5 Camera Body, RF5.2mm F2.8 L Dual Fisheye, & DJI Ronin-SC Gimbal Stabilizer), Topaz Video AI Enhancer upscaling tools to go from 8k to 16k, and all of the due diligence that he did to understand and mitigate motion sickness triggers. The end result feels like a music video that tells a timeless story of the tension between rural and urban homes and what is lost in the pursuit of career opportunities.

Below are links to of all of my SXSW 2025 coverage by the Voices of VR podcast. And be sure to check out this Linked article: Recap of All SXSW XR Experiences Including Voices of VR Podcast Interviews with Every Artist.

SXSW XR Experience Competition

SXSW XR Experience Spotlight

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

Podcast: Play in new window | Download

Share this
FacebookXRedditEmail

Rough Transcript

[00:00:05.458] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. It's a podcast that looks at the structures and forms of immersive storytelling, and the features of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So continuing my series of looking at different immersive stories from Southwest Southwest 2025, I'm going to be talking about the Voices of VR podcast. And also, this is my last episode in this series. And so this was like my third favorite experience from South by Southwest. But it was one of my favorite conversations just because Jake Olson is a really dynamic, new, immersive filmmaker that's coming on the scene. He's coming from like more of a filmmaking background. He's done a lot of music videos and advertising. So he's got like a production pipeline that is really dialed in. And so he actually did a lot of pre-production on this piece that feels like it's a music video. And it's one of the more compelling cinematic 180 videos that I've seen so far. Lots of different hyper-stylized movements, but more than anything, he's telling a really compelling story that is juxtaposing the differences between a life living in the country where there's less opportunities, but also more connection to the land around you, and then moving into the city where there's more opportunities, but it's more chaotic and a lot of visual noise around there. So it's a spatial story that poetically tells a story. And Jake's also... a musician, and so he specifically composed the musical track to fit the story. And so you really have this tying together of both the story, the spatial storytelling and the music, and also all this grammar of storytelling that he's pulling in. He did a lot of pre-production animatics to be able to really make sure that the camera moves wouldn't be too motion sickness inducing, and also just a lot of preparation. He actually had everything edited before he even started to shoot anything. And I think it really shows when you watch it. It's just a really amazingly tight experience that I found myself coming back to again and again. I had access to it before the festival started and I watched it like three times just because it was just a really compelling piece. I found myself coming back to it. And it's sponsored by Vimeo because Apple getting licensed to some other different proprietary formats for the full Apple immersive video. And there's lots of different potential distribution options that are still getting fleshed out. But I think this kind of represents like a new phase of immersive filmmaking because with Apple Vision Pro, it's a really great target device, really high resolution, like 16K is what he can render out to. And it just looks brilliant when you watch it. So I had a chance to talk to Jake and it was a real pleasure to hear a lot more around some of the gear that he's using, as well as his whole production pipeline and just his entry into the immersive filmmaking space. So we're covering all that and more on today's episode of the Voices of VR podcast. So this interview with Jake happened on Tuesday, March 11th, 2025 at South by Southwest in Austin, Texas. So with that, let's go ahead and dive right in.

[00:03:06.573] Jake Oleson: All right, so hi, my name is Jake Olson. I'm the director and composer for Currents, which is a VR180 film for the Apple Vision Pro. We premiered it two days ago here at South by Southwest 2025, and I've been wanting to make this project about Vietnam. A friend of mine created a production company in Vietnam called Buncha a few years ago, and Vimeo reached out to me a few months ago. They're partnering with Apple to create an app, a Vimeo app for the Apple Vision Pro. And they wanted me to make something that would get filmmakers excited about immersive as a medium. And I pitched them this concept exploring our relationship to cities and what we sacrifice in order to move forward. And I've been living in New York City for about 10 years and as much as I love it, I can't help but think that it's taken a piece of my soul. And millions of people around the world in rural places move to cities for more opportunity. And my friend Tiffany, who's the creative VP on the project, told me the story of this woman living in The countryside of Vietnam, in Ninh Binh, the northern region, had this dream of moving to Hanoi. When she finally made it there, she hated it. It was just sensory overload, super intense, and she left. And that, as a storyline, just got me interested in thinking more about just investigating what it would be like to go from such a pristine corner of one country to like one of the fastest growing economies in the world within Vietnam and so so I pitched it to Vimeo they loved it and I went to Vietnam a few weeks early and met with a lot of young people who had made the move themselves from the countryside to the city and updated the script that I'd wrote and we did casting and found an amazing actress named Mi who also had experienced the story herself and through many conversations with her and Buncha, the production company, kind of formed a more solidified story following this character living in Ninh Binh, the mountainous northern region of Vietnam, who leaves her mother to move to Saigon, which is this beautiful, intense city where there's millions of motorbikes and just bursting at the seams with energy. It's sensory overload, so there's always construction going on, and it has some of the worst air quality, unfortunately. And yet, millions of people move there, have moved there, for seeking more opportunity. And created a script, and also did a lot of experimenting with shooting on the Canon R5, which is a pretty nimble camera. with the Canon's immersive kit, their stereoscopic 5.8 millimeter lens and the Ronin for stabilization. And yeah, just this was my first immersive film. I come from like a traditional filmmaking background and documentary, and I was pretty intimidated to get into this field and try to figure out what makes the Vision Pro special and how we can try to push things, but also not make people sick. and had a lot of fun playing with what's possible in that space. A big part of the process for me in my practice is writing music, so I'm an electronic musician and I wrote the music as I was writing the script and then continued to work on it when I was in Vietnam and that informed the story quite a bit and just like the feeling and tonality of the piece. And there's a pretty dynamic VFX kind of animated sequence that happens three quarters through the film that to me is a representation of the character's inner journey, you know, pushing against this energy of millions of people flowing from the countryside to the city as she's trying to make it back home. She's moving through this point cloud, like a river that's just flowing against her. And she has this like, very minimal nervous system animation that we shot at onyx studios in new york city with mio this amazing movement artist and amy gardner choreographer and charlie kemp who goes by great eterna did the point cloud animation just absolutely killed it it was very difficult to do in unity it was spatial and we worked with this character animator named luigi honorat and um Tokyo. They created like a custom pipeline to bring the Houdini character animation into Unity. That was a huge problem-solving puzzle that they had to figure out and yeah, I had a lot of fun making it. It was a bit of a trial by fire because I was figuring it all out on my own and I did turn to people like Hugh Howe for advice and James from the Explore POV app helped give advice on working with the R5 and upscaling with Topaz. And I think it's funny, I've been, since experimenting in this medium coming from traditional filmmaking, I'm eager to continue to, but also in search of more streamlined pipelines with hoping that the new Blackmagic camera might provide because it was quite the process, all the post-production processing and the amount of data that was needed. I mean, we're still going to need an insane amount of hard drive space to shoot these projects, but yeah.

[00:08:47.068] Kent Bye: Nice. Well, let's go back to like a bit more into your journey into this immersive space because you say you're coming from the background of a filmmaker. So maybe you could just give a bit more context as your background and your journey into your first encounters with VR, you know, like watching other stuff and then kind of getting a sense of what is out there and if there's any other kind of inspiration points. So yeah, just a little bit more around your background and your journey into the space.

[00:09:12.003] Jake Oleson: Sure. So I directed commercials and documentaries and music videos for eight, nine years and became really interested in making electronic music around five years ago and not directing ads anymore and focused on music for three years just exclusively and was searching for a way that I can kind of express the intensity of the music I was making that wasn't live action. So animation and VFX and also was having deep anxiety around the generative video AI kind of movement that was happening and how it might be possible for filmmakers to earn a living in a world where you can generate images and spectacle pretty easily. And so started experimenting with this Luma AI app, which basically it's a 3D scanning app using neural radiance fields. And I was curious why no one was using it to tell stories. A lot of people were scanning their houses and trees and things, but that kind of became my gateway into, even though it's not a generative tool, it creates like these hyper-realistic 3D models of the world that I was having a lot of fun doing camera animations in. It was a very fast and iterative way of exploring a story and an idea that I could kind of do on my own and started kind of playing around with other AI tools and collaborating with other VFX artists and animators. Because I really hadn't done any work within that space before experimenting with nerfs. And up until that point, very limited experience with VR. I hadn't tried the Apple Vision Pro before until the project came up. And when I first tried it, I was just blown away by the fidelity of the device and the also intimidated. I'd never felt so transported before and I can't help but think that people are going to be really tempted to spend a lot of time in these things and that we might use this technology to escape. And this opportunity with Vimeo to create like a project that was a real story that felt close to my my experience, but also far enough away to learn more about another place and another another person's personal experience felt like a nice place to start experimenting versus doing kind of like a tech demo test of a bunch of stuff flying towards the camera or pretty picture spectacle imagery which I think is cool you know the technical kind of focus projects but I come from a documentary background and I'm interested in using my projects as a excuse for research and learning about the world and I've been using music as a way to explore the tonality and emotional nuance of a story and kind of create this fun feedback loop where I write the music and then that informs the script. And then I shot in animatic using stereoscopic stills on the R5, which was a really big part of me feeling comfortable shooting the project, which was going to location with my producer and shooting her standing in

[00:12:38.407] Kent Bye: So there's just like a stereoscopic photo that then would be animated, or what's a stereoscopic animatic?

[00:12:43.610] Jake Oleson: An animatic within commercials and film, we use basically like shooting the film before the film is shot. That might be with storyboards. It might be with location photos or stand-ins. And in Premiere, we'll just take all of that footage or imagery and cut together like an edit of the piece to get an idea of pacing.

[00:13:06.192] Kent Bye: And is each of those photos, and when I think of animatic, I think of it's animated in motion, but also the storyboard is very static, so I'm just trying to get a sense of if it's just like a still shot that gives you a feel of it that's edited, but not much motion.

[00:13:19.736] Jake Oleson: Yeah, so the animatic that I made for Currents was a combination of still stereoscopic photos and then test videos that we had taken in New York. I was most intimidated by the editing within the Vision Pro stereoscopic VR180. How do you cut between diverse spaces and not jolt people? How long does it take for your eyes to refocus? How close can you get the camera to a person? How fast can you move the camera before they get sick? Just all these questions that without any VR experience were really pressing things that kept me up at night, not knowing what was going to work and what wasn't. And so I just had to shoot tests, as much tests as possible. So the edit was basically done before the project and seeing the stereoscopic kind of, the majority, I said 70% of it was stereo images and then 30% was like test videos shot in New York. Seeing it within that context made me feel a little bit better going into the shoot. Like, okay, when we cut between, you know, the club and the rice fields with the mountains in the background, like that can be a really riveting emotional moment that creates this big release and expanse that after spending the entire You know, the majority of the film in this cramped city will have, like, really deep impact. Because there were early versions of the film that we were cutting in between her being in the city and being home throughout. And it wasn't working in the animatic. It felt like there was no tension. There was no payoff. I think that might be a product of the medium bringing the viewer even closer to the character's experience, where in traditional 2D, you can intersplice those match cuts between a live action character in a city and then, you know, them reminiscing of being at home in the countryside, and it will feel emotionally poignant. But for this project, I really wanted people to feel claustrophobic and kind of overstimulated and have that build up over time. So when we finally make it out of Saigon, it is this breath. And even one of the drone shots, we had a drone shot of her leaving very early in the film with the bus going by. It took a long time to do it right. And we cut it because we just didn't want to bring such an epic scene It felt like flying shot so early on in her journey. So we kept all of that for the end to give the viewer this kind of feeling of a payoff. And I kind of leave it open to interpretation in terms of It's been interesting hearing people's impressions of where Lin, the main character, stands on her relationship to Saigon and the city. Oftentimes it has to do more with the viewer's relationship to the city than it does with Lin's character, which I love.

[00:16:24.048] Kent Bye: There's a lot to project on in this piece, because it's like a poem in a lot of ways, that you're watching this spatial story that's unfolding. I really, really, really love this piece. In fact, you managed to send me a build ahead of the festival that was a little bit lower resolution, but I ended up watching it like three times before I came, because I found myself... Well, at first I watched it, and then I wanted to watch it again before I came, because I'd watched some other content and I just wanted to refresh, and then I just found myself, again, just out of nowhere, wanting to come back and just watch it again, because it just felt like there was... kind of a pacing and a rhythm. It felt like a real music video in a way that it makes sense that you had developed both of them together, like both the music and the story, because they really tie together. And also just to hear all the things you did as a pre-production process of planning it out. There's a lot of spatial grammar of the ways that you're telling the story that were unique in the way that I hadn't seen, very stylized. It also makes sense that you're coming from a music video background in advertising. And so you've got your own production process to kind of really have this tightness and efficiency in which the story is being told. And so, you know, there's this discussion that I was having with Adam Lisagor where we were talking about this difference between 360 video, like what has really started on Meta and the Quest platforms and Apple's distinct shift towards just really limiting the field of view to 180 and not having so much cognitive load for the viewer to be dependent on looking in the right place. They can look around in the 180, but you can for sure direct the attention a lot better than if they're facing the completely wrong direction. So it's making it simpler to watch, but also simpler to produce. But there are distinct differences, and I think I was exploring a lot of the themes of the 360 versus the 180, where Adam Lisagor was saying that he really thinks that 360 was telling the story of a place whereas my interpretation is that once you move to 180 you're able to tell stories of people and have more close-ups of their face and their emotions and it's just a different way of being able to focus in on telling character stories in a way that isn't always necessarily focused on the place you can still feature the place but sometimes telling the story of a place you really need to have the full context of being able to look all around like there's a couple pieces ways of knowing and shelter that we're showing here that I really think work better as 360 video because those videos are more around the locations and those places that they're telling the story. But you're also telling the story of the place between the countryside and the city, but there's a main character who is driving our attention of what we're focusing on. And so I felt like this is like a paradigmatic example of like, oh, wow, maybe Adam's right that this is, starting a new renaissance of this cinematic filmmaking that is leveraging and simplifying the production processes for all these other disciplines to start to come in and like yourself coming in and making your first immersive piece and really in my opinion knocking it out of the park in a way that you're really tying all these things together so anyway that was just some of my initial thoughts on your piece and love to hear any any other reflections

[00:19:21.141] Jake Oleson: Yeah, no, thanks for that. I really appreciate that. It's funny, there was a very different story that we were going to shoot for this project about six days before production that had a storyline with the motorcycle girl, and they become friends, and she wants to escape the city and escape on a motorcycle, and they have this mushroom trip. And I just hit a point where I was like, this is not going to work. Or at least that's not what my first exploration in this new medium should be about. I think it needed to be kind of streamlined so that we can let the location wash over people and really just explore a single character's relationship to place and self. How does one's identity change and shift as they move into a new city, into a new chapter of their life? And how could I lean into VR180 and what it offers as this larger spectrum where the viewer is the cinematographer, they get to choose where their attention goes, and not ask too much of the viewer in terms of following storylines and having a lot of dialogue. things that they could miss, and that influenced the design of the project, because there's only so much you can do, right? So we kept the main action in the center of the film, so then unconsciously people know that if they want to kind of look around, they can. There's all these periphery details, like the woman she lives with is off to the window in her apartment complex to the right, and then some little details in the marketplace and other things. you can go there and you can come back and not feel completely lost. It's a simple enough story where you kind of get the gist of it with her leaving her mom with the setup and then coming to the city. And that resulted in a very meditative piece that was more about environment and tonality. and the music and developing a relationship to place that I really enjoyed exploring I think for the next project whatever that might be I'd like to try to focus more on narrative and see what different avenues that might open up because like So much of our energy as filmmakers and storytellers is spent developing the bond between the viewer and the character. Whatever we can do to break down the screen and dissolve that is paramount. And when you're working in a medium that you feel like you're physically there with the fidelity as high as the Vision Pro headset, so much of that work is already done for you. I really do feel when looking at people shot in live action on the Vision Pro and it's really reaching that highest fidelity that I'm witnessing them, I'm sitting with them, and I'm not watching them through a screen. And I'm seeing them for like who they are is the same way that we're sitting here having this conversation. It's activating different parts of my brain and it's making me be more attentive and more present and more empathetic. And that's a completely new palette of emotions to paint with as a storyteller. that are powerful and that can be used to manipulate people and be incredibly destructive, but also healing and transformative. I think that we can use this technology to really dig deeply into the human experience and stories that we might not relate to, communities we might not relate to, and begin to change the hearts and minds of people that would otherwise not be changed. The film, as Roger Ebert said, is this empathy machine And film is an empathy machine than VR and the Vision Pro is like an empathy machine on steroids. It just feels like magic and I'm just so excited to see where it goes. But I'm also very eager to be a part of the movement of independent creators that's... thinking critically about how we're going to use this technology to connect people rather than just escape the reality that we're in. The biggest compliment that I get from people watching Currents is I need to go to Vietnam. I love that when people take off the headset they want to have a physical experience out in the real world. And I think that that's the key when using new technology. It's not to just spend more time locked in, escaping from whatever world we're in. I know it's tough. We're living in a terribly difficult world with a lot of stimuli and Kearns kind of dives into that. But at the same time, there's no denying that having real world experiences, that's what we're wired for. There's something very different that happens when you're sitting in a room with a person, having a conversation, making eye contact. And when you're painting with a medium that has the potential to touch on that, I think we have to do it with a lot of responsibility and critical thinking.

[00:24:20.720] Kent Bye: Beautiful. Yeah, totally agree. I wanted to dig back into your origin point into coming into this immersive space through Vimeo. And how is it that Vimeo heard of you and how did they know that you were interested in doing an immersive project since you hadn't done one before? And so maybe just kind of flesh out the beginnings of this commission.

[00:24:39.761] Jake Oleson: Yeah, so I had made the first short film using exclusively neural radiance field scans, and that won this award at the Runway AI Film Festival. And I spoke on some panels after that. Vimeo asked me to speak on this Fast Company panel. I kind of kept experimenting with machine learning tools, becoming more involved with VFX artists. People started to see me as this person that wasn't afraid to experiment with new tools and see the potential that they have. And when Vimeo, when Apple asked Vimeo to or when Apple granted Vimeo the ability to use their spatial algorithm to stream spatial videos on their Vimeo app, they wanted to find a creator that could like Vimeo. get people excited to make work in this new medium and so they reached out to me and see if that was something that I'd be interested in even though I hadn't done anything in spatial or immersive and you know, it's very intimidated but the resources were pretty small for the first initially and I had this idea of shooting this project in Vietnam for like a year or so and And I kind of pitched them, like, if we really want to do something special, we should put some real resources into this. And I think we can make something that can be evergreen and people can keep coming back to this. Because that's, I think, the problem with a lot of technical-focused work that we see, whether it's in AI or whatnot, is it's rendered irrelevant within months, sometimes weeks. But if the story is good... And if it's exploring something that is fresh, then it's evergreen. It could live on a streaming platform 10 years down the road and it could still evoke something in people. And so Vimeo being the awesome company that they are, their CEO, Philip, is really supporting creators in a way that is incredibly exciting. They just launched this filmmaker fund with Nikon where they're giving $30,000 out to five filmmakers and pairing them up with directors like David Lowery. And I think Philip and Vimeo, they have the right idea. The way that we're going to shift culture and really set the precedent for how these new tools are going to be used is by doubling down on creative people that are critically thinking about the implications of this technology. And I did not take that responsibility lightly with this project. I thought a lot about, am I the right person to tell this story? I also am not an immersive filmmaker. Can I even do this? Is this going to make people throw up? But after speaking to a bunch of immersive filmmaker friends and doing a lot of research, trying the Vision Pro and putting together the treatment, I felt confident enough that we could do something really special. And I'm always excited by the opportunity to jump into the deep end of a pool in a new technology, especially because there's this sense of ignorance. I don't really understand the rules or what other people have done. And that could be a really negative thing, but it could also be a really fun thing as well.

[00:27:57.441] Kent Bye: Yeah, it's kind of an interesting trend with the Apple Vision Pro with all of these traditional 2D filmmakers coming in and making their first Apple immersive video projects that they've brought in first-time filmmakers and sometimes it works out okay and other times it feels like there's decisions that are coming more from the 2D realm or not necessarily translating. There's a lot of experimental energy with trying out things and to me I think it's been a mixed result in terms of Sometimes you get like the closeups get really too close and the proxemics and sometimes experiences that do make me emotion sick. With your piece though, it was comfortable and I'm sure there are things that could be motion sickness triggers for people. And then people have different tolerances. So I don't know if you had any buddy that was actually experiencing any motion sickness because some people have different tolerances to camera movement and whatnot. But for me, it was fine watching currents. But you said that you were really concerned about it. And it seemed like you really did your homework in terms of really trying to investigate what those triggers would be and what were the limits of how far you could push it. So there was a lot of experimental movements of the camera that were novel and unique that I haven't seen a lot of. And so, yeah, maybe just talk about that process of as you're coming in, how are you doing your due diligence to make sure that you understand what those motion sickness triggers are? And then as a creator, this need to push the boundaries, but also like try to do this checks and balances to make sure that as you're watching it, you're not making yourself sick or others sick.

[00:29:28.979] Jake Oleson: No, absolutely. I mean, that was why the animatic and those tests I shot in New York were so pivotal and made me feel confident enough going into the project that we would make something that we wouldn't have to go back to Vietnam and do reshoots. You know, tilting the camera, rotating, the close proximity to a character takes on a psychologically completely different experience. It's powerful, but it's It's intimate. It feels wrong in a lot of ways. I don't think the brain is used to feeling present with a person or that close to a person and seeing them and knowing that they can't see you is a new experience. We just haven't had anything like that in reality before. And so a large part of the project was trying to figure out what is the viewer experience? Is this going to be like a God kind of mode thing? Or are we going to lean more into traditional filmmaking perspectives and vantage points? And then also, like, how can we push things through novel camera setups? And our cinematographer, Gal Peratt, super talented DP, was a huge part of like not only dynamically lighting the scenes in a way that it can be difficult in VR180, but also dreaming up some super unique camera moves, whether it was mounting the camera to the motorcycle or flying it through the stairwell, you know, doing the drone shot with the octocopter. We learned pretty quickly that you can move the camera pretty fast as long as you're moving along a single axis. Rotating, there's only two moments in the film where the camera rotates and it can be pretty rough for the people that are super, super sensitive. But I think the fidelity of the headset is so high that it's actually okay for the majority of people, which is interesting because it doesn't, yeah, the whole inner ear thing with the rotating move can be rough if it's too fast. Yeah, I guess, again, the animatic and testing stuff out in New York and seeing it in the headset before going to Vietnam was a huge part of gaining a better understanding of the language of this project and the language of the medium. We didn't completely exhaust what's possible, but we wanted to create a little bit more of... a voice that Currents would have and the music was a really big part of that too. It kind of has this momentum to it that I knew we wouldn't want too many locked off shots. It would be this like forward kind of propulsion to bring us through each of the different scenes and you know the Ronin was smooth enough for us to be able to pull that off but then we also used the Slypod, which nifty little piece of gear. It's like a little mini techno crane that Hugh Howe talks about that end up being like our MVP for being able to quickly set up what would normally be a very complicated shot to pull off in traditional live action with like a big cinematic camera rig. One of the reasons why we shot in the R5C with our budget, although it was smaller than like, you know, Apple immersive stuff, that small footprint allowed us to get into small spaces and have this like very fluid camera movement that would bring us closer to the character. and also being nimble, we were able to shoot a lot of scenes over the course of the shoot days. And the footprint of the R5C was also pivotal for this project. I think if we shot on a larger camera, like even some of the unreleased Apple Blackmagic cameras that some friends of mine have shot on, I think would draw a little bit too much attention to themselves. I've also heard they're a little bit unstable in terms of overheating and things, and we were shooting in the humid, you know, very hot conditions of vietnam with rain and hiking up mountains and we just didn't have the manpower to bring like three massive cameras that we couldn't rely on completely and the r5c seems to be the best model for that particular project i'm familiar with a dslr mark ii like is that around the same size or is it bigger Yeah, it's just about DSLR size, just a little bit more room on the back. And then the lens, the stereoscopic lens is quite small. The whole kit's under $5,000, which is amazing considering the quality. We did have to do some cleanup with the noise for low light stuff with Topaz AI, but pretty remarkable considering the amount of resources it takes to go the next level up. with some of the unreleased Apple cameras that we don't have access to. But I guess that'll change pretty soon with the new Blackmagic camera, I guess. Yeah.

[00:34:16.013] Kent Bye: Yeah. And so did you shoot the whole thing on this Canon camera?

[00:34:20.075] Jake Oleson: Yes. Yeah.

[00:34:21.635] Kent Bye: And then did you have to up res it? Or what resolution do you end up mastering it out to?

[00:34:26.912] Jake Oleson: Yeah, so we shot like 16 terabytes of 8K RAW in Vietnam and created proxies from the RAW. The RAW files using the EOS VR utility app, which I know is to a lot of immersive filmmakers seems like a no-no. You know, you want to use Mystica and the other programs, but we just had such a quick turnaround for this project that we needed something that could be fast. use those 4K proxies to edit the film and premiere. And then once the cut was locked, went back into the source footage and then printed the 8K raw images as ProRes, and then upscaled using Topaz AI to 16K, and then used Mike Swanson's spatial metadata injector to export at like 400 megabytes per second in the HEVC codec, and then worked with the sandwich video guys, Adam and Jose, who were... kind enough to create a custom build of their theater app to stream the film locally, play back the film here for South by Southwest.

[00:35:36.289] Kent Bye: Yeah, maybe you could elaborate a little bit about how are you using that theater app. I had a chance to talk to Adam Lisagor about the theater app just before coming out to South by Southwest. So yeah, it sounds like that they've got a little bit like a kiosk mode that you can kind of just jump in and jump out quickly.

[00:35:51.058] Jake Oleson: Yeah, so Adam and Jose, they're amazing humans making this awesome app called Theater, where you could watch both immersive films, dome films, VR180 films, and then in a myriad of different venues, just real filmmakers that are excited about what's possible with the Vision Pro in a way that I haven't seen in other people that are using the tech. And so they reached out to me about Currents and told them that we were screening at South by, but we're still figuring out like a streamlined way to play the film without having to go through the eye alignment for each person because it takes up time. And they were kind enough to develop like a custom build of the app with the current artwork and a big watch button in the center. So then it allowed people to just put the headset on and look at the center and start playing the film in VR 180.

[00:36:46.914] Kent Bye: Nice. Yeah. And I brought my corrective lenses. Did anybody else bring their corrective lenses to be able to watch it?

[00:36:52.182] Jake Oleson: That's a good question. I did not see anybody who did do that. We did have some people put the headset on with their glasses, which I also know is a bit of a no-no, but yeah, I didn't see anybody with corrective lenses. It's always, it's a bit of a, it's a tricky thing, right? Trying to share the Apple Vision Pro with multiple users. And I know that they're working on ways that it could be a more shareable device, but right now it's kind of designed as a personal device.

[00:37:20.829] Kent Bye: Going back to the upscaling in Topaz AI, because you're going from 8K to 16K, did you find any visual artifacts of certain situations where you could perceive a difference between things clearly not shot at a high enough resolution to interpolate the pixels? Or for the most part, do you feel like it did a good job? When I was watching, I didn't notice anything, but... As you're editing it and going through this process of the different resolutions, I'm just wondering if there's anything that you perceived, any downsides to that.

[00:37:53.367] Jake Oleson: Yeah. I mean, so there's a few different models in Topaz AI that you can choose. The NYX model is used for denoising. And it does a really phenomenal job at it, but it also creates this smoothing effect on skin and on rough surfaces. There's so much texture in Vietnam that I ended up really not liking what it was doing. And so there's, I think it was Proteus is the other one that we used to upscale, which was doing a good job without like smoothing some of the more fun textures that Vietnam had to offer. And in terms of artifacts and things, it was the low light scenes that you'd start to see pieces of the blacks that would be multicolored that you're like, I don't know if that quite is what it would look like, but it's close enough because it's dark that the brain isn't really hyper focused on it. There's still more denoising to be done. Someone gave a suggestion of this plug-in within DaVinci that we could use to do a little bit more denoising in the club scene. And when she's walking in the alleyway with the rain, there's still a little bit of noise left on there. But for the most part, it's pretty impressive considering you're going from 8K to 16K and you're close enough, you know, 90 degree field of vision in the Vision Pro to to be able to like see the ways in which it's doing a good job or not. Like you can really see the detail and it's lived up to that. Yeah. It's pretty remarkable. It kind of feels like magic.

[00:39:24.259] Kent Bye: What have been some of the reactions to the piece?

[00:39:27.481] Jake Oleson: The reactions have been really meaningful and sweet. A lot of people have related to this idea of leaving their home, wherever that might be to move into a city for more opportunity for a job opportunity for a better life. And, um, The universality of the piece has been cool. It was kind of designed to be resonant with a lot of different people, but I really just wanted to focus in on Lin's story, the character. So to see it have that effect was really exciting. We have all these postcards that we printed out from the cinematographer that shot 35mm stills. And after viewing, we'd invite viewers to write a letter to themselves and put it in this red box that we would mail to them in a year. Just as an exercise of... Sometimes you watch the film and you could be choked up and nostalgic and in this introspective, reflective state that... kind of want to capture and bottle it up and then give it to people later on. And so I'm excited about that little detail. But yeah, overall, it's been super meaningful. Also, the Vision Pro is so new to people. There's a lot of people here at South By. This is their first Vision Pro experience. So, you know, I wonder, it's like, is the film that good or is it just like the fidelity and the... the magic of the device that's also doing a lot of the legwork. But there have been a lot of immersive filmmakers that have come through and watched it and have really resonated with just its experimentation and I guess like the emotional delicacy of it. And that's been fun and really meaningful because I didn't really know how this one was going to turn out.

[00:41:15.563] Kent Bye: Yeah, it's funny. When I first started watching the piece, I was like, oh, this is like a documentary because I didn't read any of the blurbs or anything. And so then but then I was like, oh, well, this sounds like it's scripted. It's like, OK, maybe it's a narrative piece. And then it morphs into like a music video. And then at some point it transforms into like this stereoscopic point cloud representation with all these shaders going across the world with like this. motion capture actor that's moving through. So, yeah, it's like a really magical progression. But I think overall the rhythm and the feel of the piece is just, I don't know, I feel like it just has a really hyper-stylized and, to me, real exploration of the affordances of the medium. That is doing something that's distinctly new that I haven't really seen done to that extent before. And I think your background in both music video and advertising, I can see those fingerprints of that background in here. Yeah, I just wanted to see if you had anything else you wanted to say in terms of the point cloud scene, because that seems to be at another kind of turning point in the story, but also just a real evocative and beautiful. And, you know, it's got colors that are in there that are giving you a mood and a vibe. And so, yeah, just if you have other comments on the process of designing that scene.

[00:42:29.082] Jake Oleson: Yeah, so the point cloud scene was an idea that I had from the beginning that I just knew I wanted to do as an exploration with the Vision Pro. I've been doing 3D scanning work and experimenting with telling stories in real environments that are stylized and manipulated. And I knew that if we weren't going to have a really deep kind of story exploring the relationship between one character and another on screen, that there must be an interesting way that we could explore Lin, the main character's inner journey. So the DJ who she befriends, Poe, on the motorcycle, we were talking about psychedelic drug-induced rave experiences that we've had just like at parties and she told me this story of going to Saigon and seeing all these young people who had moved from the countryside to the city all on the dance floor and this light was in their chests and it was almost like it was trying to escape. And I've been really interested in music's ability to, within a dance floor and rave setting, to create transformative experiences for people and was reminiscing and talking about our own experiences with dance music and started to sketch out what it might look like to create the inner journey that Lynn has reflecting on her leaving home and her mom and this desire for something better that you can strive for something better in your life and that there is a place that you can make your dreams come true but you have to give something up in order to go there and what could be an abstract symbolic representation of that feeling being pulled in two separate directions. You're born in one corner of the world with a particular family, a particular set of values, where everyone is taught that to be successful, you need to have a house or have a job or a family. And I think a lot of it is conditioning. There's just this flow of ideas and of energy that move from place to place, whether it's from the countryside to the city. And I know that I feel like I'm pushing up against that flow sometimes. And although it strengthens me to fight against that current, it's hard and it feels very lonely and solitary sometimes. And Lynn's character is in the throes of that. trying to make a life for herself in the city while being homesick and in search of that deep connection and the connection with her mother and her family that she might not have in the city. So the point cloud sequence really became like an emotional exploration of that very complicated color of emotion that I'm kind of butchering right now trying to explain to you. that to me is really, really exciting to explore with choreography, with spatial storytelling, with VFX, with 3D scanning. I have a lot of difficulty synthesizing my feelings and thoughts into words. And so when a piece of art hits me, it just gets it and it bottles it like it captures it. I feel seen and it makes me reflect on what I care most about. And when I leave an installation or a movie or a VR experience, having gone through a cathartic excavation of the soul like that, I just feel a little bit closer to the reason why we're here, you know, and I don't have the answers to that. But I do think that this artistic pursuit of just trying to explore that is a meaningful pursuit and it's a worthwhile pursuit and Being able to do it in that much more fidelity, that much more immersion, I think has the potential to open up avenues for deepening the connections with ourselves and our purpose for what we're supposed to do here in this human experience.

[00:46:54.577] Kent Bye: Nice. And you mentioned that Vimeo was a part of producing this piece. So what happens to Currents next? Is it going to be on the Vimeo app? And is it going to be on the theater app? Are you selling it or streaming? Like, how are people going to be able to see this piece?

[00:47:08.840] Jake Oleson: Yeah, so the spatial cut of Currents is available on the Vimeo app, on the Vision Pro. And the immersive cut, we're working on a home for that right now. So I've been talking with a few different distributors and also Apple about what their plans are for supporting independent creators. I think the main thing for me is just how do we get more storytellers to make immersive projects. And I think the way for that to happen is more avenues for distribution. It was an amazing experience making Currents, but not a lot of people have seen it. And that's why South By was so exciting. Have a place where everyone gets to watch the project. So I'm in a lot of conversations figuring out what the next steps look like for realizing currents, VR180 distribution, and then also other festivals and things. But yeah, it's kind of a work in progress.

[00:48:07.691] Kent Bye: Did you say the spatial cut and immersive cut? And what's the difference between the different cuts?

[00:48:12.170] Jake Oleson: Yeah, so the Spatial Cut is like a floating 3D screen that you can pull pretty close to you. And Vimeo's app is able to stream spatial video, and that's the...

[00:48:24.844] Kent Bye: So it's like a window and you can see the stereoscopic effects, but it's like more of a, you can see the frame.

[00:48:29.728] Jake Oleson: Yeah. Yeah, exactly. But VR180, streaming VR180 is like a pretty complicated process that I think a lot of people are working on and ironing out the kinks in, so.

[00:48:43.857] Kent Bye: That's what you're showing here was the streaming 180, but it was more locally. So it wasn't streamed. It was all on the device locally.

[00:48:51.667] Jake Oleson: Yes. Yeah. So we locally played the VR 180 version of Currents on the theater app.

[00:48:58.576] Kent Bye: Okay. Okay. And so because Apple is doing their own streaming of that, because you can go to Apple TV and stream that content, but there's no one else that's really streaming it. Is that right?

[00:49:07.719] Jake Oleson: I think that there are some streaming platforms for VR180, but it's like they're still working out the best pipeline for it and then also building up user base. You might be able to crack that code, but do you have the users that are coming to watch the film? And if so, are they willing to pay money for projects? How much should they cost? It seems to be a lot of questions. And also, it's a very new field for me. I'm still getting a lay of the land of all the different places that this thing could live.

[00:49:40.714] Kent Bye: Okay, well, that's certainly helpful. Hopefully it'll get out there in all the different forms that people can see, with the most immersive one being the one that I would recommend. I haven't seen the other one, but I can just, it was so powerful, the version that I saw. Hopefully people have access to it here in the future. And so, yeah, I guess as we start to wrap up, I'd love to hear what you think the ultimate potential of this type of immersive storytelling might be and what it might be able to enable people.

[00:50:04.545] Jake Oleson: Oh, man. I mean, I think I don't know if I could repeat what I said before in terms of just as a storyteller and as a human who's interested in making cathartic pieces of work that explore the world and capture. The complexity of the stuff that I don't understand and attempt to synthesize that into a story or a song. Working with a medium that you're able to place a person in an environment and have them witness a character rather than watch a character through a screen is just that much more exciting and powerful, but also, you know, like we were talking about before, comes with a great responsibility So I think as storytellers exploring the immersive space with a device as high fidelity as the Vision Pro, we have this little window of time these next few years to make work that can set precedence for the public to understand how this can be used, but also what role it fits within our lives. i love watching content and art on the vision pro but i also love going out into the world and experiencing things in reality and i think that uh i'm very interested in the ways that this tech can further connect us and connect us with ourselves that much faster that much deeper And I'm just going to continue to try to push the envelope in terms of what's possible with pushing audiences. I think Currents was a, I don't want to call it a safe exploration because it's not asking too much of viewers. It's more of this meditation. But I am excited to explore stories that might be that much more difficult to relate to unless it was told with VR. Like, what does this medium mean? And where does it fit within the cultural zeitgeist? Because it has traces of film within it, but it is fundamentally a different experience watching like a stereoscopic, like immersive film.

[00:52:22.112] Kent Bye: Awesome. And is there anything else that's left unsaid that you'd like to say to the broader immersive community?

[00:52:29.316] Jake Oleson: No, I mean, I'm just excited to be a part of it. I think like doing South by and watching some of the other immersive experiences and slowly entering a community that's very new to me. I've been really surprised and excited about how open people have been and accepting within the immersive community. I think film can be sometimes very competitive and you have a lot of different personalities and with immersive i feel everyone is just super curious and so passionate we're not making a lot of money if anything off of these projects so it attracts a very particular kind of artist you know these are the early adopters these are the people that believe in something that doesn't exist yet and we get to dream up the future together That's a pretty special window of time or a special conversation to be a part of in human history. So I'm honored to be a part of that conversation.

[00:53:32.669] Kent Bye: I 100% agree because that's the big reason why I've been doing this podcast for the last almost 11 years now in May. They're just bearing witness to the medium as it's unfolded. And yeah, it does attract this type of passion driven artists who are really interested in exploring stories that are meaningful to them and also just exploring what can be done with the medium. So yeah, it's a really exciting time to be a part of this ongoing community and medium as it's evolving. And, uh, With your piece, it really resonated with me. It's something that I start to see the traces of the new grammar of the constraints of the 180. That's also opening up new possibilities for new production pipelines, new cameras, new artists and creatives, new backgrounds being fused together. So it's a really exciting time to see where it's going to start to go here from the future, especially now with Apple Vision Pro, with that resolution. I think it's really inviting, this type of cinematic innovations that we're just seeing at the very beginnings of. I'd point people to watching your piece as a good turning point in my own kind of observation of the medium. There's something different here that's worth looking at. And I think Blake was seeing the same thing. Blake Camadiner, the curator of South by Southwest, because he said he was surprised by it. It took him by surprise. And I think it's because it's a story that slowly unfolds and takes you in a direction that I wasn't quite expecting. And just all the way that the music and visuals are all kind of working together in harmony and Yeah, and just some really hyper-stylized cuts and just some scenes that stick in my mind from the piece of like a cut that you have with her in the foreground and the mountains. And it's just like, wow, just so well done. So anyway, thanks again for joining me here on the podcast to break down a little bit more about your process and your journey in this space. And it's really fascinating to hear all the details. So thanks again for kind of breaking it all down.

[00:55:21.887] Jake Oleson: Thank you, Kent. It's an honor to be on the podcast and really appreciate the kind words. Just so excited to be here at South By and speak to you. 11 years is a chunk of time to be so focused on this emerging medium. And it's awesome that you're elevating each of us to try to make more of these because I'm certainly excited too. Awesome.

[00:55:47.883] Kent Bye: Well, let's go make some stuff.

[00:55:50.064] Jake Oleson: Let's do it.

[00:55:51.601] Kent Bye: Thanks for listening to this episode of the Voices of VR podcast. And there is a lot that's happening in the world today. And the one place that I find solace is in stories, whether that's a great movie, a documentary, or immersive storytelling. And I love going to these different conferences and festivals and seeing all the different work and talking to all the different artists. And sharing that with the community, because I think there's just so much to be learned from listening to someone's process to hear about what they want to tell a story about. And even if you don't have a chance to see it, just to have the opportunity to hear about a project that you might have missed or to learn about it. And so this is a part of my own creative process of capturing these stories and sharing it with a larger community. And if you find that valuable and want to sustain this oral history project that I've been doing for the last decade, then please do consider supporting me on Patreon at patreon.com slash voices of VR. Every amount does indeed help sustain the work that I'm doing here. Even if it's just $5 a month, that goes a long way for allowing me to continue to make these trips and to to ensure that I can see as much of the work as I can and to talk to as many of the artists as I can and to share that with the larger community. So you can support the podcast at patreon.com slash voices of VR. Thanks for listening.

More from this show

Menu

#1556: “Currents” Boldly Defines a New Era of DIY 180-Degree Immersive Filmmaking with a Kit Under $5k