I spoke with Khang Lu, Quinn Heptig, & Eliza Zhang about their short film shot in VRChat called Our Precious Autumn as a part of my Raindance Immersive 2025 coverage. See more context in the rough transcript below.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.438] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So continuing my coverage of Rain Dance Immersive 2025, today's episode is with the winner of the short film competition at Rain Dance Immersive. It's a piece called Our Precious Autumn by Studio Penrose. So this is a short video that you can go watch on YouTube. I highly recommend that you go check it out before listening to the conversation. But I had a chance to sit down with the founder of Studio Penrose and some of the key contributors to this project to walk through this story that they're telling around artist burnout, which is something I think all of us can identify to some degree or another, anybody who's working in it. any creative venture, or especially if you're working in the field of virtual reality, you're bound to at some point hit a wall in terms of just being completely burnt out. And so this was a piece that was really exploring that as a theme, having different aspects of traveling back in time and talking to your former self. And yeah, just trying to find that rekindling of your creative imagination and It's a piece that I think is a pretty universal story and also is using the virtual reality technologies in quite an innovative way in terms of starting with the voice actors to lay down the voice and then having body actors that are coming in and having face tracking technologies to be able to do eye tracking, but also integrating custom expressions and emotions. And so trying to find different ways of communicating. moving past some of the different limitations for how to do these different types of choreographic coordinations within the context of these virtual films, and also creating a consistent look and feel to create this real cinematic experience. And so again, highly recommend checking out this piece called Our Precious Autumn, and we'll be diving into all the different process in which they created it on today's episode of Voices of VR Podcast. So this interview with Kong, Quinn, and Eliza happened on Wednesday, July 9th, 2025, as part of my broader Raindance Immersive 2025 coverage. So with that, let's go ahead and dive right in.
[00:02:08.817] Khang Lu: Hi guys, I'm Kong. I'm director for Studio Penrose, as well as I run the studio. I create VR films using VR tech and all that sort of stuff. I've been doing this for four, almost five years now. I haven't really counted the dates, but yeah, that's it for me. Yeah, Quinn, you can go ahead. Okay, sure.
[00:02:31.143] Quinn Heptig: Hi, my name is Quinn. I'm the cinematographer for Studio Penrose. And I've been doing VR film work for about four and a half years. So not as long as Kong, but a decent amount. And I primarily work on making the films look as pretty as they can. And I've just started getting into the realm of actually directing.
[00:02:49.349] Eliza Zhang: And my name is Eliza. I'm a film composer and product manager. I compose the music for Our Precious Autumn, which is my first time scoring a short film. I've been working with a couple other VR filmmakers as well. And I've also been involved in the VR space in different ways over the last 10 years.
[00:03:05.999] Kent Bye: Okay. And I always like to hear all the different design disciplines that people are bringing in to the realm of VR and VR art. So I'd love if each of you could give a bit more context as to your background and your journey into the space.
[00:03:18.885] Khang Lu: Yeah, of course. So I actually never had a film or like art background. I joined VR in 2019, I think it was. My first headset was like the Rift S and I just kind of just played VR chat as my main platform for fun. I also played like Beat Saber. But for a few years, that was all it was, was just playing the game as a game. But it wasn't until like... 20 whatever five years ago from for now is 2020 ish is when we started like making films for the first time in my professional setting i'm a mechanical engineer and so the creative side kind of comes into play when i go on vr and it's a lot different than my like job and so vr film was really like new to me. And I've always wanted to do film, but I could never do it in real life because it was just too expensive and too much skill or too much, I guess, time and commitment. VR film is a lot different. And yeah, that's about it from my background.
[00:04:19.537] Quinn Heptig: I guess for my end, I tried VRChat in 2019 just kind of on a hunch. I'd seen it in a recommendation on YouTube a couple years back, and it kind of just flowed into my mind again at the time. So got in there. I didn't realize it was a social platform, but kind of got used to it. I started realizing that you could create or like take photos at the time. And I had a passion for photography when I was younger and I still kind of do. And I started taking pictures in the game and I saw a group that was making films or like video work. I was like, oh, that looks really cool. Let me see what I can do. And ever since then, I've just really hunkered down on learning the idea of cinematography and seeing how those aspects can apply to the virtual realm in terms of filmmaking or video production or anything in between.
[00:05:07.759] Eliza Zhang: For myself, my journey in VR began in 2016 with the Google Cardboard headset. After I got to try the Arctic journey experience with this whale that comes out of the ocean and crashes in front of you, I knew that I really wanted to work with XR technology. But there weren't that many resources all the time. For myself, I studied at the University of Michigan. And when I started, we only had two Rift DK2 headsets available for a campus of like 50,000 students. So recognizing that there weren't that many resources and we would need more to really help grow the VR industry, I co-founded a VR AR student organization called the Alternate Reality Initiative. And that later also led to me leading the Intercollegiate XR Community or ICXR. And both of these groups were basically intended to help more students break into the VR industry through hosting events, workshops, and providing resources. From that original starting point with more community building, I got more involved in social VR. I started off in alt space VR originally, and especially during COVID, ended up using social VR quite a lot. And I think it helped me really understand the power of the social VR friendships. I kind of became interested in figuring out a way that we could help prove that these relationships were real, were authentic, and that led me into exploring academia. I worked with a professor to conduct a research study on social VR relationships in all space VR. And I originally had planned to continue with doing research into grad school. And that's around the time when I got started in VRChat, originally more focused on the social VR research side of things rather than filmmaking. But although I didn't end up going into grad school, I continued with VR chat as well as the VR filmmaking side because of the people that I met. I've done different things with body acting, helping with the marketing for different VR chat films. And so when Penrose was founded, I joined my friends and it was a wonderful way to combine my interest in VR as well as with my background in music.
[00:07:08.732] Kent Bye: Very cool. Well, it's quite a diverse background and it's always interesting to hear everyone's journey into the space. And Kong, I know that I've had a chance to talk with Ariel Emerald and she was talking around how, you know, kind of the early days of some of the early exhibitions that were happening, like Fia Bunny did some exhibitions. I know there were some other exhibitions that were happening. And then eventually Rain Dance started exhibiting different projects and Joe Hunting's we met in virtual reality. Love to hear from your perspective, you know, kind of the origin story for some of the filmmaking communities and the discord and what was the catalyst for you to be involved in the creation of some of those early VR chat, VRC filmmakers in the community around that?
[00:07:51.084] Khang Lu: Of course. That one's a fun one. I think everyone's stories are origins for their filmmaking, I guess. It varies quite a lot in VR. Mine was actually VRCon. I don't remember the exact year. It was either 2020 or 2019. So VRCon, way back when it was still around, was like a virtual convention online, just a bunch of communities showing off their stuff. And they had, I guess, a portion for VR film, because at the time, there was some new VR films that were popping up. And when I first saw my first VR film, I was like, oh, that's really cool. I've never seen something like that. I didn't know it was possible to do using VR chat. And so I watched a couple there, I made my own around that time. And that was when during this convention, I met up with some of the others that did this stuff and realized, okay, this is pretty cool. There's more people doing this than I thought. And visually speaking, these films kind of look like animations in a way. And so I just thought it was really unique. The platform of mocap combined with like the appearance of animation. And so met with a bunch of people there. That's how I met, actually, Eliza and Quinn early on. And from there, we did our own thing for a while. And then as we kept presenting our films at events, so VRCon was one, and then Thea's showcase was another one that Ariel mentioned. we just met more and more people. And it kind of became like a snowball effect of us meeting people and us kind of realizing, oh, we don't really have a community, do we? Because this stuff's very niche. And so a few years ago, me, Ariel, and Gab Jenkins, actually, the three of us made the Filmmakers of VR chat. or filmmakers of VR. And that's our little community to help kind of get people into filmmaking in VR. And that's what we host like workshops there and classes and all that sort of stuff. So that was kind of my like journey into the meeting other people stuff.
[00:09:59.500] Kent Bye: Very cool. And maybe you could then follow on and talk around kind of the formation of Studio Penrose. You talked a bit about the founding of the filmmaking community, but I'm just curious to hear what was the catalyzing point for you to create your own studio around starting to gather other folks to come on board to help create different films that were being shot within the context of VRChat?
[00:10:20.973] Khang Lu: Sure, Quinn remembers this one because it was actually, it almost might've been on the dot two years ago that we actually did this because it was like in the early July, we were writing some stuff for a different project for our past studio. And we're writing some lore to a story that we had made earlier. And we were like, oh, this is kind of fun, you know, like writing a universe, if you will, for all of our films to take place in. And it was from that point that we realized, okay, this is really nice. We have a universe, we have stuff that we want to do in the universe. And so a few months after this writing thing, we decided to go independent with our studio and kind of focus on our universe, our lore, our stories. And so a lot of us that we had already met previously, we had just continued on and moved on to the new studio. And that's where Studio Patmos was formed. And our first project as a studio was Emergence in 2023. That one was submitted to Raindance of 2023. And yeah, that was the start of the studio.
[00:11:26.564] Kent Bye: Nice. And Quinn and Eliza, I'd love to hear a little bit from each of your perspectives of kind of entering into both the larger filmmaking community, but also getting involved specifically with Studio Penrose.
[00:11:36.853] Quinn Heptig: Yeah, no, for sure. I guess on my end, Kong actually mentioned pretty well We started Studio Penrose roughly about, I think, two years ago from, yeah, July, because we had just gotten back from Eliza, me, Kong, and a few others went to Los Angeles to go check out a convention. It was just a group of friends and a few of us. And we had just gotten back and we were working on some lore for one of the films we were working on that led to what Emergence would be. And we decided, hey, I think it'd be cool to make a studio. Or Kong, you know, said that and said, sure, let's do it. And that ultimately led to studio pendants becoming a thing. But I guess in my eyes, when I started getting more into virtual production, it was still very niche and it still technically is very small. But a couple of years ago, I only knew of, you know, Kong, Eliza, and a few other people that were making films. And it didn't seem like there was too much else going on. So when I saw that Kong and a few other people had introduced the filmmakers of VR group and community, I was really relieved to see so many new people come into the field in the scene and introduce their own, you know, pieces of work, their own films, their own movies. And it really kind of expanded the idea of, wow, virtual production is really, really cool and also super unique. And I'm glad to see that there's lots of people still striving to check out VRChat because of the possibilities for film to be done there.
[00:13:03.929] Eliza Zhang: For myself, I got into VR filmmaking because I was looking for good videos that I could show to basically help explain what VRChat is and what made social VR special. And it was from some of those early videos from like like four or five years ago, that kind of led me into exploring more about like, how are people actually making these VR films? I've only heard of like 360 VR films. Didn't know people were making these like machinima style films within VR chat. And so from meeting people like Quinn and Kong, it's been quite a journey where we've known each other for going on five years now. And I've like worked together on different films in the past year. And I think throughout this process, even though VR as a whole has kind of changed quite a lot, the relationships that people have formed and the supportive community around all these passionate filmmakers has been something that's kind of brought me back continuously. And now I'm also exploring some different things in this space, too.
[00:14:02.082] Kent Bye: Nice. Well, for the film that you created for Rain Dance Immersive 2025, Our Precious Autumn, so ended up winning the award this year and is a really beautiful film, really well-told story. And we can get into the mechanics of how you're telling the story, but just curious to hear a little bit about where the idea of this first came up. It seems to be about the topic of burnout and creativity and does this interesting travel through time and speaking to your prior selves to rekindle the imagination for, you know, your original passions for doing creative acts. And so I just love to hear a little bit around the creative inspiration for Our Precious Autumn.
[00:14:45.117] Khang Lu: Yeah. So the project actually began about a year and a quarter ago when we first started the idea. The film was written off of my own personal experience with filmmaking burnout. Obviously, the tie-in is Ami, or the main character, is a manga artist, and we're filmmakers kind of experiencing the same thing. So I, at a certain point, like two years ago, before the studio was formed, was pretty much ready to quit filmmaking because it just got way too mechanicalized. It got too corporatized for me. I kind of lost sight of the creative aspect of it and became too invested with the numbers and all sorts of stuff. And I just straight up burned out. I started getting less creative and it was tiring for me and all that sort of stuff. And I realized, you know, after I got out of that pit, when we started our new studio, when we met a lot of people, we continued our first film after creating the new studio. I realized that there's a lot to be excited about in filmmaking and in creative work in general. You know, creative work is one of the most burnout prone types of work. because it's not just, oh, I'm doing a very mechanicalized thing, it's like you're yourself, like you're writing your own stuff, you're telling your stories, right? I talked with a bunch of other creators in the VR space, like musicians and artists and voice actors even, and kind of got their input and like, have they ever reached that sort of like stage of burnout? Or like, have you ever gotten tired of what you're doing and kind of like lost sight of your passion? And the kind of scary thing is that everyone said yes. like every single like creator that I talked to and so my experience wasn't just my own it was also a very common experience for a lot of people and I think it was right around that point where I was like okay we got to do this so let's let's make a story to kind of I guess connect with people most of our previous projects were more action more like I guess fictional based and more like lore based. And so I've been wanting to do a film that was more emotionally driven. And this was the one I really wanted to do because it was a personal experience, but like the VR space is very creative. There's a lot of people who do a lot of like talented things on here. And I wanted to give something back in that sort of way as well.
[00:17:15.370] Kent Bye: Yeah, and so let's go around and talk around, Quinn, your introduction to this project, because as a center photographer, I know that there's a special camera that you can use. It's also the default camera, but just curious to hear around a bit of the process of how you went about this, because usually I've seen Directors are usually sometimes also the cinematographer just because the logistics of having the director be able to give like real-time feedback, you know, is this the type of situation where you're sort of trusted to go off and shoot a number of shots or is there some sort of like screen sharing or, you know, just curious to hear a little bit about the process of coming onto this project and then working with Kong, who's the director and to try to figure out your strategy for how to collaborate in shooting this film.
[00:18:01.636] Quinn Heptig: Yeah, for sure. So with Artbrushes Autumn, Kang and I were both basically cinematographers of the film. And when it came to production, we were shooting each scene. Kang and I would both talk about what we want to have in frame and how we want to shoot the specific part of the scene. And we would both choose similar angles, but we'd all, we both have our personal kind of touches to what we want to have within the shot. So Kong, since it was, he was directing the film, he would go for one of the main shots he had envisioned based off the script. And then I would take that same idea, but also put in my own personal touch. So it may be a very similar angle, but maybe I'll have it a little bit perhaps to the left or I'll have it more zoomed in or some other parameter of the cameras changed. And with Our Precious Autumn, there's a ton of different angles that we decide to experiment and test with because we have a variety of different places we're going from the office world where Ami is working within the studio to her child room at the house that we go to in the dreamscape to the train scene to all these different places. And this is something I talked with Kong about is not being afraid to try very unique angles, for example, like Dutch shots or rule of thirds or perfect ratios. I think it'd be really cool to show not only in VRChat, but like the capabilities of having camera techniques be possible because for a long time, the default VRChat camera wasn't actually able to do anything like that. You could just do photography. And as of recently updated, so you can have like certain camera settings like depth of field, aperture, zoom in, you can turn off certain layers of the camera so you can see only perhaps the player or just the world. And what we use for our productions, we use an asset called VRC Lens by Hirabiki. And you basically take a DSLR camera and put it into VRChat. And that's basically what it is. And for this film, we actually did some modifications to the camera so we could do some ratio adjustments. So for this film, Kong, I believe, has it. But I think it was ultra wide. Is that what you said it was, Kong, for the ratio?
[00:20:02.585] Khang Lu: I can't remember the name. I think it's 23.1 dash, oh man, I can't remember that. It's a widescreen format.
[00:20:11.313] Quinn Heptig: Yeah, but we shot it in widescreen format for like large cinema theaters and for anyone to watch on like their widescreens. And with this, we actually, on our little display where we could see the camera, we actually had red lines to show us where that ratio would be. So we could actually film with a little bit of leeway in case, you know, we had an issue on set or like one of the shots didn't look as good. But we had that kind of as our placeholder and we would adjust things as needed. But this was a, our precious autumn was definitely a challenge. in terms of how much can we push the limits of cinematography and making something look as good in-game as it could look on film.
[00:20:50.252] Kent Bye: And so Kong, I'd love to hear from your perspective, because, you know, as the director, so it sounds like you're basically doing a two camera shoot where both you and Quinn are shooting like every scene from different angles. Is that, is that right? Or would you kind of do pickup shops where one or the other, one of you would do like a closeup and, but are you essentially shooting just like double coverage just so that you could have more options when you're editing?
[00:21:14.502] Khang Lu: Yeah, it depends on the scene, but for the most part, we were basically doing a double coverage and me and him would like do a lot of coordination to figure out what we want to do as well. Like before each shoot, we would have to sit down and like talk about, okay, uh, this is a scene with a lot of conversation. How do we want to go about this framing? How do we want to go about this reverse shot verse sort of thing, but yeah.
[00:21:40.024] Kent Bye: Okay. And Eliza, when you're working on this project of Our Precious Autumn, were you coming in at the end to do this music or were you involved in the production aspects or other things as the film is being produced?
[00:21:51.977] Eliza Zhang: Yeah, so I was really fortunate to be involved early with being able to read the script and see the storyboards. And it really gave me time to help understand the emotions and story that Kang wanted to tell. I know that's a little bit atypical in the film industry, where typically the composer's brought in at the very end as they're doing the editing. But because of being involved early, it gave me more time to learn and come up with ideas. Because for myself... I played piano and trombone for over a decade growing up, but I didn't go to music school and never scored a film. So I had to do a lot of learning throughout the process of working on Autumn. I read different books, watched YouTube, completed some online courses. Specifically, I started studying film scoring and orchestration through Berklee Online through the Berklee College of Music in Boston. And I was able to take a lot of what I was learning in those classes directly into creating the music for the film and drawing on all sorts of different inspiration. It was definitely a very iterative process, typical of composing music, of course. Worked with Kong, presented like, here's different ideas. And then what do you think about them? This is one too happy, this one's too sad. And throughout that process was able to then come up with the Ami's theme, which then you hear in different variations over the course of the film with different instrumentation.
[00:23:08.467] Kent Bye: Very cool. And so Kong, the topic of emotion has come up quite a bit when we're talking around this piece. And I know that in the Q&A afterwards of the world premiere screening at Reign Dance, I had a chance to ask a question around, like just noticing in the credits, how you had a decoupling of the voice acting from the body acting. And that happens sometimes within the context of animation, just motion capture versus voice acting. I think in previous works, film productions, I tended to see a little bit more of a tighter coupling from who was doing the body acting was also usually doing the voice acting, but maybe just talk a bit about, you know, the decision to go to voice actors and then just the production pipeline that you had to go through in terms of whether you were preferencing the body acting or the voice acting and just the workflow of trying to get the voice and the intonation right. And then kind of building everything else around that.
[00:24:01.823] Khang Lu: Yeah, I think every director or studio does it differently. For us, we've always done like voice actors are separated from body actors. There's like a couple of rare cases where there were people that both body acted and voice acted in a film, but for the majority, we separate them out. And there's typically two different ways that we do the dubbing of the voice acting. It's either we do an ADR, which is you film everything and then you have the voice actors dub over the footage, or we do the voice acting first and then have the body acting done accompaniment with the voice acting. And typically we usually do the ADR, but this time around we chose to do it reverse where we have the voice acting done first, because I knew these scenes, there was going to be a lot of like conversing scenes and especially like the end scene where there's a lot of like crying and more emotional line delivery. I didn't want the voice actors and actresses to be limited by fitting like a body acting or fitting a body actor's like execution of the lines. And so we actually did the voice lines first for this film because of that, mainly being just emotional impact, not limiting the voice actors and actresses. But in terms of doing that choice, it led to a little bit more complication on set. So instead of having the body actors just freely be able to voice however they want, because the mouth movements will be just... If you don't have to match to anything, it's a lot easier. We had to give the voice lines ahead of time. And so I had all the voice actresses and actors record their lines. I set them up in the way that we would see them in the film and then gave them to the body actors. And from there, they would actually verbally say the lines in the same... timing as the voice actors did. And there's another option you could do, and that's playing the voice lines through the microphones to eliminate that problem of trying to, I guess, sync up with the voice actors. But the problem with that is when you play audio and you have to worry about pressing buttons and stuff, it becomes very cumbersome. And I think it stresses out the body actors or it limits their movements. And so I elected to just go for having them try to recite the lines that they were given. And so that's a little bit of a unique experience for this project.
[00:26:24.959] Kent Bye: Yeah. The mouths are small enough. I think that it's not like some of the furry community that has dog mouths that are very articulated, like the mouths are not as articulated. So I think you can start to do stuff like that. And in a way that it was not really noticeable as I was watching it at any time that the things were out of sync. But what you're also having with the body actors, did they have like eye tracking? Because it seemed like they had pretty well articulated like eye movements within the context of the piece as well, which I think is also something that is, you know, I guess, you know, more relatively new. But just curious if you're using like the MetaQuest Pros with eye tracking.
[00:26:59.741] Khang Lu: Yes. So that's another thing that was unique about this film that we did for the first time. So all of the main body actors or main characters had eye tracking and they all had a MetaQuest prose that I sent out to them for this project. And it's kind of interesting. For a while, I was kind of against using face tracking for films because what ends up happening is if you have like full face track and that includes mouth and like eyes and eyebrows, all those, it tends to over animate your expressions a bit. Cause you know, it's really hard to completely perfectly accurately track like every little mouth movement that you make in real life and then translate that to VR on an avatar. And so I was kind of against using that for a while. For the over expression, I think it's fine if you like using avatar for other stuff, but for film, it looks a little weird when you have an over animated face expression. But we figured out that we can combine our controlled expressions with eye tracking only. So there's a way to set this up in VR chat where you can only track the eyes. And what ends up happening is we now can control our face expressions traditionally, like using animated face expressions that we already pre-made. but also have the eyes track. Human eyes, when they look around, especially during very intense scenes or intense emotions, your eyes will dart around very fast. There's not a really easy way to do that through the avatar's animation. And so using eye tracking will fix that. And it makes the expression feel a lot more lively. And so eye and blinking were tracked, but eyebrows and mouths were controlled through animation. And we combined the two to make it look more mobile.
[00:28:47.990] Kent Bye: Nice. Yeah. Yeah. It was a really great effect. And I think as time has gone on, we start to see new waves of technology and people having access to it and starting to add more and more of these capabilities. I think another aspect of this piece was that there was a lot of very specific locations. And I don't know if you did location scouting or if you went out and actually built, like custom built some of the different worlds in order to shoot some of the different scenes. So Just curious to hear a little bit about the either the scouting or world building process to create the scenes in order to shoot this film.
[00:29:20.618] Khang Lu: Yeah, all of the worlds in this film were created or curated for the project. So the office world, the apartment world, the childhood home world, and then the bookstore, those were all made for this project. The reason why is we, you know, the main character is an artist, a manga artist. And in order to kind of make sure that the audience knows that we need to have a lot of art all over the place of hers. And especially since she's overworked and burnt out and tired, we wanted to do things like making the apartment very messy looking. know plastering her art to remind her of what she's doing all over the place the posters all that sort of stuff and so we had to make these worlds from like we had to make them for the project and so yeah it took a bit of time that was actually one of the reasons of why the pre-production for this project took so long but all the worlds were modified or custom
[00:30:17.691] Kent Bye: Yeah, and Quinn, love to bring you back in to talk a bit about like creating the look and feel in terms of the, like I noticed that it seemed like there was some color grading that was happening, at least to give it a consistent look and feel and have maybe some slight variations when you have like the dream sequences. And I think taking the video right out of VRChat, I mean, sometimes you can do that, but I guess I'm assuming that there's a little bit more of a color grading of creating the aesthetic. Just love if you could elaborate on that a little bit.
[00:30:47.113] Quinn Heptig: Yeah, for sure. So when we were shooting between all the worlds, they all have their own unique type of lighting. But of course, we bring that video outside of VR chat and into our program to edit. We have a few programs that allow us to do color grading specifically. And Kong can actually answer this a little bit better than me. But we want to kind of give a certain, I guess, atmosphere for each location. For example, for the office world, it feels very... like work you know you have like that kind of natural warm lighting in a way but also it's not too i guess protruding to the idea of oh you know it's very cozy it's very nice to be here and then we go over to let's say the train scene you know very dark and very moody but you can still see what's going on and let's say uh child ami's room you know it's a very nice orange hue it's very cozy very warm you feel like you're actually in a place where you can relax and uh be comfortable but kong actually i'd like to elaborate a little bit more on this as well
[00:31:41.181] Khang Lu: Yeah, so I did all the color grading for the project through Adobe Premiere. And so we typically heavily rely on color grading for our projects because in order to kind of remove the differences between each world, like Quinn mentioned, and also kind of the feel of it being a VR platform that's not specifically made for film, we have to do certain things through color control to make everything consistent, especially if you're going from scene to scene or world to world. things will be different between each. Like highlights and shadows are the two biggest things that we had to focus on color grading because they vary a lot between world to world. And even though we made all these worlds for the project, there's still going to be like differences between each of them. For the dreamscape scene, so like, or all the dreamscape scenes, we kind of wanted a more warm hue to them because they're her memories. And I think that's kind of like a very typical film thing to do. And for like the office scenes and then the nighttime office scenes and then the apartment and then the train scene, we went for a very cold feel because yeah, it's her current reality. It's very cold. So yeah.
[00:32:57.947] Kent Bye: Yeah, as I think about this piece, there's kind of like these different emotional peaks and climaxes. And Eliza, I'd love to hear you speak around the music in terms of like how you were trying to match the narrative climaxes and thinking particularly the... the scenes where there's like a breakdown and then but also there's train sequence with like some sound effects and so there's kind of larger sound design that's being involved but on top of the score as well and so just curious to hear a little bit about how you were thinking around like how to match the arc of the story with the music kind of swelling and trying to amplify the emotions that are already there, but also the broader sound design in terms of like, if that was something that you were also involved in, in terms of creating all the other Foley effects and everything else that was involved in the film.
[00:33:48.634] Eliza Zhang: Sure. So starting off with the score itself. So in terms of like the overall sound we were going for, I was fortunate that Kong did have some like musical inspiration that he was able to share. One of the most prominent ones was Autumn Leaves, specifically the cover by Ava Cassidy, which also has the London Symphony Orchestra playing in the background. I also knew, given that this story was so personal to Kang, I wanted to try to draw little bits of pieces based off of his past as well. And given that Kang also is formerly a violinist, I definitely wanted to feature the solo violin and strings in general throughout this film. And so I originally came up with just the piano version of it, which is what you hear at the beginning of the film, just introducing the theme to the audience without too much additional jazz or anything to it. And then throughout each of the scenes, basically, was trying to understand how it would potentially build up to the emotional climax that happens when Ami breaks down. And so I The different approaches that we took was with the bookstore, the dreamscape one ended up being just a more simplified version of the theme, given that we wanted to kind of just let the environment speak for itself. the version where she is walking home from work is slightly more different from the main theme but still kind of playing that similar type of role of just helping convey those emotions of like she's extremely exhausted extremely tired but then also still having to make sure that that could be that type of sadness but not conflict with this type of sadness that we wanted mommy to feel when she does chat with her child self. And so that's why you hear both the music box version, which of course is very emulated childhood, and then you hear the more larger one, which is with the piano, solo violin, and solo cello. So it's a lot of trying to figure out which versions of the theme would go where, and we actually even swapped some of the places a couple of times to try different ideas. And then when it came to how that would then interact with the sound design, the sound design for the mixing mastering was done by our amazing team member, Scarlet, or Scarj. And so we were able to kind of collaborate together. She also had sought feedback from many of us on the team as well about how different things sounded. So it was quite a collaborative process. And I think that's what allowed us to be able to combine all these different elements together into the final film.
[00:36:24.574] Kent Bye: Yeah. And as I think about the film, there's also these like moments of call it special effects where, you know, just even crying, you know, there's things that you can do with like, you know, swapping out an avatar where you have tears that are flowing down the face or, you know, have like the water drops falling onto a piece of paper. It's something like in physical reality would be pretty inconsequential. You just, you know, could be do it in any other number of real physical effects that you could do in film. Um, But in VR, you kind of have to construct a lot of these things, even when the main actor is on the train tracks and then having to pull up another actor, what you can't really grab onto anyone in VR. And so kind of create the illusion that they're being pulled up. I imagine there's lots of coordination to make sure that. that looks realistic, but also like play space manipulation. So I love if you could maybe just elaborate on some of the more challenging shots that you had to do, or just the extra effort that you had to do in terms of special effects within this piece.
[00:37:24.224] Khang Lu: Yeah, of course. So I think starting off with the crying, because that was a fun one. So it was half practical and half VFX. For the avatar, we have like the vertical tears down her cheeks. Those are actually like a prefab that you can apply to the avatar. to make it look like she's crying. But we wanted to also show tears welling in her eyes and then also dripping onto the sketchbook. And so those were actually VFX'd in by a friend of ours named Dozing Cat. They do a lot of Adobe After Effects things, and they're very talented with it. And so they did stuff like that for the project. And as for the train sequence and all sort of stuff, that falls under choreography, which I feel like the majority of our struggle in VR filmmaking, you mentioned that we don't get to actually hold each other and can't actually pick someone up and so on and so forth. And so essentially, for all these scenes that involve character to character interaction, they have to be choreographed. And with that scene, we basically would time The train, the two characters that were pulling her off the platform, and then her play spacing up or moving her avatar upwards together, which there's already four parts of that equation and it gets really difficult. That is probably our biggest difficulty with VR filmmaking, I would say, is maneuvering around the lack of ability to interact with each other's characters.
[00:38:54.428] Kent Bye: Well, I know that VRChat has implemented some of that, but it seems like that it's still not robust enough or, you know, to really, you know, I guess, you know, you don't want to be having someone put their hands through someone else because they don't have any like tactile, captic feedback. And so it's just kind of a choreographic move at that point, because you can interact on some virtual interactions, but you still have to not put your hand through their body and stuff like that.
[00:39:18.649] Khang Lu: Yeah, so there are things you can interact with, like you mentioned. For instance, avatars, we can push each other's hair around or grab it, like that sort of interaction. But you can't fully interact with body parts, like your arms, as an example. You can't grab someone's arm and then pull it. And even if you could, there ends up usually being what's called clipping, where our avatars will kind of cut into each other, go through each other. It ends up being very difficult that way. That's one of the problems with the hair interaction that I mentioned a little bit ago. You can still cut through avatars. I actually have talked about this with some of the developers of the game. It'd be very difficult to fully introduce a system that allows, say, for me to grab someone's hand and shake it without having any sort of clipping. through the fingers or anything like that. And at its core, VRChat is a social platform. It's not made for what we're doing here per se. And so it's not like an expectation that we can have to have all that sort of stuff ironed out. But I think it's one of those things where as technology progresses, we may see less and less problems or more cool tools that are introduced to, I guess, fight that problem.
[00:40:36.547] Kent Bye: Yeah, I'd love to hear some reflections on the theme of this project, it being about burnout and each of you being involved in creative process. Kong has already spoke to this a little bit, but love to hear from both Quinn and Eliza around your own personal experience with burnout and some of the themes that were explored within this piece.
[00:40:54.488] Quinn Heptig: Yeah, no, for sure. I guess on the topic of burnout, my time in VRChat is not only put to making films. I work with a lot of other people who do content creation and help produce videos on YouTube, on Twitch, and also create musical experiences, for example, like concerts and virtual productions. And trying to put all that together roughly around similar time stamps and having film on top of that. And of course, I'm also attending school and work a full time job. It really kind of all comes together and is very difficult to balance. So there have definitely been some days where I have been kind of burnt out from just trying to work on these projects, get these deadlines done, or just try and stay up and standing to continue working on everything. And I'd say part of putting that into Our Precious Autumn came from Ami when she's sitting at her desk and the manager calls her in to talk about what she was working on about Seeds of Life and how they mentioned that the sales were not doing as good and they asked to try a new idea. And it was really kind of disheartening almost in the sense of Ami being pushed away from something she was working on already and she thought would do really well. And I've had that happen myself for projects I've worked on for people and whatnot. And it's been really disheartening at points. But even still, I still continue on. I still want to be passionate and create things that people will enjoy. And yeah.
[00:42:22.002] Eliza Zhang: For myself, the creative burnout has kind of emerged in different ways. I think the most prominent one I can think of would be just in my early professional career, where I started off doing marketing in the games industry and games industry itself has its own reputation regarding burnout crunch time and some of the different pressures that come from that. And I think having worked in that environment for several years, it's definitely Had somewhat of a toll, I suppose. And I think as a result of working on Autumn, though, it was an opportunity for me to reflect on my own journey and go back to like my younger self and some of the things that I was interested in. Similar to Ami meeting her child self for myself, it was going back to the childhood self that wanted to be a professional musician. I was extremely passionate about trombone, piano. I wanted to be a professional trombone player in orchestra and such, but due to various different factors, didn't pursue that type of creative career. And then also there's certain scenes like Aiko and Yuta's argument that I think especially resonates for people who have had to deal with some of those challenges and thoughts about pursuing your passions and also just the realities of like working in some of these creative industries but throughout the process it's kind of also been sort of a healing journey for me and as a result of working on this film i've been exploring more about exploring film scoring more seriously as well as learning about other skills like filmmaking and cinematography myself hmm
[00:43:54.758] Kent Bye: Yeah, one last thing around the themes of the film is that there's some playing with time and other concepts of time, like you're going back and time traveling within this dream sequence where the main character is able to talk to her child self and her child self be able to re-inspire her with the source of her creative imagination, but also just this precognitive hit of something that might be possible in some potential possible future that has not been realized into actual reality yet. And then you have what's possible, then by the end, you have this kind of realization of that possibility into actuality. So love to hear any reflections on the nature of time and kind of using this piece to jump around in time and explore these themes.
[00:44:43.632] Khang Lu: Yeah, that's a good question. I think generally speaking, I'm a pretty sentimental person. So I hold on to my memories and the things that were good and also bad in the past pretty tightly. And I think with the film, I wanted to really hone in the idea of memories. Something that I realized that I do, and I think my other studio members do this as well, or know that I do that at least, is I will look back at the films that we've made previously. And that scene where she interacts with her child self was basically me looking back at our previous films. And almost every time, I'll talk about them being very like, oh, it was really fun to do. I was really happy with the product and all that sort of stuff. And then I always follow that with the question, where did all that go? Where has the time gone? And so in terms of the film's exploration of time, it's more so using time as a means of memory, if you will, not as a huge player in the story itself. Is that the question?
[00:45:53.846] Kent Bye: Yeah. I think as I was rewatching it, there was just kind of some of the other themes that were popping up. And I think with VR, you can start to play with time in a way, but also film as a medium does that as well. So I don't know if Quinn or Eliza, if you have any other thoughts on kind of the feelings of nostalgia and memory or just jumping around in time as you think around this film.
[00:46:18.422] Quinn Heptig: Yeah, actually, with Our Precious Autumn, when you have the transition from Ami being on the train, the train stop going into the dreamscape, I've actually had something like happen to myself in a way. I tend to reflect a lot on past experiences I've had, whether it be good or very bad. And I do shoulder both of them on both sides because it gives me good, I guess, experience to look back on and to learn on or learn from. Whether it be really good advice or a mistake I made when trying to do something I thought was right, it taught me a lot. And I think when it came to Our Precious Autumn, I like that we had that scene incorporated because it shows people that you can look back at, you know, perhaps your past self and learn, you know, what got you to where you are now. And it's okay to make mistakes and you can learn from it.
[00:47:13.709] Kent Bye: So awesome. So finally, I'd love to hear from each of you what you think the ultimate potential of virtual reality and VR filmmaking might be and what it might be able to enable.
[00:47:24.952] Khang Lu: Sure. I think there's a lot to look for in the future. I mentioned some of the things that could help us with the filmmaking aspect. And a lot of things, even technology like hardware, face tracking, as it gets more advanced, we could see more versatile facial expressions. We could see a lot of the over animations that I was talking about earlier get fixed. And as all these things improve, the hardware and software side, we'll see the films improve as well. With the filmmaking stuff in particular, so we've submitted this film to a bunch of film festivals. We've already gotten it accepted into three of them. And for a lot of people, that was the first time they've ever seen a VR-created film. So I'm already very excited about all that. And I think in terms of the VR filmmaking in the future, as more people see it, I think it'll just get more and more attention overall. Even us uploading the film onto YouTube, a lot of the comments in the film itself were like, hey, I've never seen something like that. What was this made in? Like, oh, this is a nice animation. Like, oh, we're not animation, right? But people thought it looked like that. And so I think in that aspect, our medium has a really good potential in the future, especially as, like I said, tech improves itself. So more films, right? That's all I got.
[00:48:49.774] Quinn Heptig: i guess i'll go next um in regards to i guess the future of virtual filmmaking i think we're at the very beginning of what's possible because currently we're using vr chat as a medium to produce films which i think is a very cool and neat concept and it's beautifully done as of right now because we were able to produce our precious autumn which is i think really really just phenomenal with how much went into it and the reward that came from well over a year's worth of time and effort and i definitely know that there will most likely be much more advanced tech in the future that allows to be even more in depth with production in that scale But I have no idea what it will be just yet. But I'm very excited to see where it goes. And I hope that I can continue to help produce and also perhaps in the future direct my own films in VR or perhaps something very similar to it.
[00:49:46.912] Eliza Zhang: Yeah, I think I'm most excited just to continue to see more stories being told. I think it's been fascinating just to see how accessible this technology has been to allow people who don't necessarily have that traditional filmmaking background to be able to share their stories and use this technology in really interesting and creative ways. I think also in general, It's been wonderful being in such a supportive environment of these different VR filmmakers through a lot of the collective efforts of people like Kong with the Filmmakers VR Chat and such. Because I think it's helped us remind ourselves that even though there's many different things going on in the world, at the heart of like what we all care about. It's the fact that we're making these really amazing stories with people that are passionate, that care about what they're doing. And that's overall been a very inspiring process.
[00:50:41.509] Kent Bye: Awesome. And is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:50:46.633] Khang Lu: I think it's going to be the message of the film, actually. A lot of people on the VR community or in the VR community are very creative and talented. And remember to keep doing what you're doing. It impacts us all whenever someone makes a cool event or makes a new film, makes music. I think having everyone creatively free around us really helps us continue what we're doing. So keep it up.
[00:51:14.608] Quinn Heptig: Yeah, I'd say with those who are listening and are thinking about making something new, whether it be a passing project or just an idea you have in your head, go for it. Because I think it's always worth taking the chance to make something extraordinary. And I think you'll have a blast doing it. So please just take the next step.
[00:51:36.249] Kent Bye: Awesome. Well, Kong, Quinn, and Eliza, thanks so much for joining me here today on the podcast to break down the film Our Precious Autumn and your journey through Studio Penrose and creating these films within VRChat. And yeah, it's a really beautiful story and it's well told. And I think it also just kind of starts to piece together all the different components from the eye tracking to the set of photography, the look and feel. But overall, it's just like a heartfelt story that I think really resonates with a lot of people within the community and Yeah, it was definitely my favorite piece this year in the filmmaking category. And yeah, just also really enjoyed hearing a little bit more context for your journey into the space and where you want to take this all here in the future. So thanks so much for joining me here today on the podcast.
[00:52:17.910] Khang Lu: Yes, thank you for having us. Thank you for having us.
[00:52:20.848] Kent Bye: Thanks again for listening to this episode of the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
