Raqi Syed and Areito Echevarria’s Minimum Mass is an emotional, immersive narrative that explores grief of miscarriage. They apply their visual effects knowledge gained from working at WETA Digital in New Zealand to push the real-time game engine to go beyond the cartoony, flat shader look. Syed is interested in exploring more photorealistic lighting in VR with a film noir style inspired by Todd Hido and dynamic lighting inspired by Lumia artist Thomas Wilfred.
Echevarria is also interested in researching how the proximity to characters in VR is correlated to the emotional impact of a story, and they’re experimenting with a rotation mechanic to rotate these table-top scale scenes within the experience. Having a table top scale allows a lot more agency for the viewer to walk around and act as a sort of cinematographer choosing the perspective that is the most appealing to them.
Minimum Mass was a part of the Tribeca Showcase at CannesXR 2020, and I had a chance to catch up with Echevarria and Syed to talk about their experiential design process, exploring metaphoric embodiment of grief through a sort of tentacle smoke, how they worked within the limitations of a real-time game engine of Unreal Engine, their process of working with actors in VR, their experimentation with using the world rotation mechanic to get the best perspective, the philosophy behind their lighting and fragmented black hole world to invoke a personal dream-like quality, and how they took inspiration from Jungian psychology and the alchemical principles of the reconciling third to resolve the tension of opposites.
It has one of the most distinctive styles that I’ve seen in VR, and it’s a powerful personal story about the trauma and grief of experiencing a miscarriage. Syed said that it’s the process is both retraumatizing, but also cathartic to be able to create a piece of art that becomes a point of conversation in something that is otherwise a pretty taboo topic. She says that good art requires that you have skin in the game, and that this work is a result of putting themselves out there to be public and vulnerable about a very difficult experience. Given that they were an independent production, then they were also freed from the overplanning that can happen in big film productions and they were able to follow their artistic intuitions in an iterative fashion. They described their process as a sort of deep, intuitive listening of what the piece was telling them what it wanted to be.
Minimum Mass is still available to see for free until July 3rd as a part of the Tribeca Showcase within the Museum of Other Realities.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. Today's episode, I'm going to be doing a deep dive into minimum mass and covering lots of different aspects of both the lighting, character performance, and some of the theory and practice of experiential design. If you're listening to this before July 3rd, I highly recommend going to Museum of the Realities to look at the ConXR exhibition. There's a Tribeca showcase that this specific experience is a part of and is available for the next couple of days. highly recommend that you go check it out and actually try to check out all the different experiences from Tribeca and you'll get a lot more out of this conversation and hopefully if you're listening to this after July 3rd that at some point the experience will become available you'll be able to watch it but there's still a number of different theoretical aspects around lighting and experiential design and also it's a very personal story specifically around their experiences around miscarriage and trying to create art that is trying to express this trauma in a cathartic way so That's what we're covering on today's episode of the Voices of VR podcast. So this interview with Areto and Rocky happened here in Portland, Oregon on June 28th, 2020, but there in New Zealand on June 29th, 2020. And it was a part of the larger Tribeca Film Festival showcase that was happening in CanXR, which was happening within the Museum of Other Realities online. So with that, let's go ahead and dive right in.
[00:01:37.801] Areito Echevarria: My name is Areto Echevarria. I guess I would call myself a researcher slash artist. I work in the area of virtual reality on the narrative side and I'm also an academic and I do some research around emotions and character performance and how those two concepts fit together. How people express emotions, how people understand them, primarily through performance.
[00:02:00.653] Raqi Syed: I'm Rocky Syed. I am one of the co-writer directors of Minimum Mass. I'm also a researcher and an artist. I come from a visual effects background as well. That's how Reto and I met. We worked at Weta Digital for eight years. Before that, I was at Disney. And my area of focus is because I was a lighting technical director for a long time, I'm really interested in the art direction of virtual reality, specifically real-time lighting, and how we can use a lot of the offline rendering techniques that we are so familiar with, that we know work and look beautiful in big-budget visual effects. How do we bring that polish and sensibility to really independent storytelling and do it in virtual reality? So one of the things we're doing in Minimum Mask is like telling a story that we think, like the kind of film that might have been made in the 1970s, a very independent, pretty personal story, and do that with the technology of big budget visual effects, but with the budget and pipeline of really independent gaming. So there's like a bunch of design challenges with that. And that's the area that we want to push into with Minimum Mass. And also on a storytelling level, like using our personal experiences to drive genre. So like with Minimum Mass, it's like miscarriage, but also black holes and speculative worlds. So really wanting to bring those two things together thematically in a visually interesting way.
[00:03:31.121] Kent Bye: Yeah. I'm wondering if you could maybe set a bit more of that context of, you know, your journey into VR, but also how this story came about and why you decided to tell it in this way. Yeah.
[00:03:42.111] Areito Echevarria: Well, we'd both been, as Raka said, both been working in visual effects for a long, long, long time. You know, it felt like this very long extended film school experience that went on for like 20 years or something like this. So we were really honing our craft, I think. and then kind of got sick of working for the man and felt that we had enough life experience and things that we wanted to say that we were like ready okay ready to talk about stuff and we've got the craft there that we can do it in a way that satisfies our tastes I guess and the story is about a couple who are trying to conceive a child and it's a personal story so it was Something that we were kind of going through at the time and I don't know I guess it was like one summer and we were writing a lot of stories that some way we wrote I don't know seven or eight or nine or twelve treatments or something like this and That seemed to be the one that really stuck out and the one that Resonated with people and the one that started to gather a bunch of momentum. So we were like, all right I guess we're doing I guess we're doing this one And that's how that got started. And the VR angle, I don't know, I guess a lot of people were talking about, our friends were talking about VR at the time. I was quite interested in it. I had tried the Kickstarter and thought it was like super, incredibly compelling. I put it on, I was like, this is amazing. Felt very sick, had to lie down for three days because it didn't have the sick stuff at the time. But through that sickness, I kept thinking that I could really do something with this. And I think I was Maybe even I think I was in the shower one day and it just sort of very suddenly came to me that, yes, this is what we should do, because I think we could do something really interesting with this medium.
[00:05:15.313] Raqi Syed: Yeah, after I left Weta Digital, I made a 360 short. It was just like a weird little idea that I had. And I wanted to understand, like, how do you do that? At that time, it was Chris Milk had said, oh, VR is an empathy machine. And how do we tell stories? Like, there were all these, like, really basic questions about can you edit in VR? And so we were using a lot of us were using 360 to put together these really experimental camera rigs and try to get the sound to sync and then figuring out how do you create a witness camera? What do you do with the director in the room? Like these super rudimentary things that seemed very important like five years ago. I figured stuff like that out. I was trying to understand that through that film. And that film is like, you're not going to see that because nobody, it was just, it doesn't need to be seen. It's done. But I submitted that project to the Sundance lab, and it got long-listed. And then it didn't make it through the last phase. But Kamal sent me a personal rejection, which was really nice. And I took that as an invitation to try again. And so that's when we submitted Minimum Mass the following year, and we got invited to the lab. So the form, the medium itself, was calling to me in this way that seemed like a good fit at the time. Also, like with 360, I realized I wanted to learn stuff about the medium, but I couldn't really exercise my craft. Like we knew that we have to go into real time and we had to use computer graphics to create these stories because that was where our superpower, so to speak, our craft could actually move further. And with 360, I just felt like that wasn't a good fit because I didn't have that live action experience behind me to begin with.
[00:06:57.143] Areito Echevarria: Yeah, we did go into it with a certain amount of hubris, I think. We know how to do this. It'll be fine.
[00:07:03.995] Raqi Syed: Yeah, we found ourselves wondering, like, why does so much virtual reality look this way? Like, why does it look so cartoony? And why are the renders so simple? Why is it all non-photorealistic rendering? Surely we can resolve this. Then we realized what the engine can and cannot do. And we've talked about this a lot, like that computer graphics in VR is like this stubborn material that we're constantly trying to wrangle. to get it to do a thing. And I think that we've had a certain amount of success with minimum masks, but we didn't get everything we wanted, for sure.
[00:07:41.448] Kent Bye: Yeah, well, there's a lot of different aspects that I find interesting about this piece that I wanted to kind of unpack with you as the creators. One is the locomotion type of mechanic where you're rotating the scene around. There's amazing lighting throughout. We can talk about some more and also the story and the ways that you were metaphorically showing aspects of that story and then being set in a black hole. But let's start with the decision to do what is kind of like a dollhouse tabletop scale What I found is that the story is actually very intimate. And in talking to Paula Weiss, who does a lot of writing about VR, her comment was that she felt like she wanted to be at the same scale because she felt like she was looking in from the outside. But you have this mechanic right when you go in where you're taught how to like move a box up and down and rotate it. And so right off the bat, you get thrown into like this car going down this fragmented road and you have to like learn how to rotate around. Otherwise, you know, you're going to be lost in the experiences like right off the bat, you're like, hey, you got to learn how to actually use this mechanic that we just taught you. But throughout the entire experience, I found it staged like a theater. And so as I move around, I'm a bit of like the cinematographer, but there's one moment where I'm asked to switch rooms, but I'm just curious to hear a little bit more about the thinking and the design behind this rotation mechanic, because I haven't seen it in a lot of other places. I'm just curious what your thesis was and what you found in the process of adding this to this experience.
[00:09:11.094] Areito Echevarria: Yeah, that was, um, I'm glad you picked up on that, Kent, because that was, that was a long road to get there really, because I think in the end it's not perfect, but it feels fairly simple or natural or something like this, but it was, it really took a really long time to figure that out. So I think the genesis of that was. playing around in Tilt Brush and just really liking this action where you can pick up an object and move it around in your hand as if it's a real thing. And I really enjoyed that interaction. My thinking was, well, what if you could do that with the world? I just love dioramas anyway. I grew up making things in shoeboxes, and I kind of love this idea. So I was like, what would happen if we could do that in VR? And then, of course, you do it in VR to begin with. If you have the whole world, it's just like completely nauseating if the whole world is spinning around in your hands. So then it's like, OK, well, we've got to scope this down. We've got to make it a smaller space. Still really uncomfortable. So then the next step, I think, was locking down the horizon so you couldn't tilt it at all. And at that point, OK, that was starting to sort of get somewhere useful. And also, there was another mechanic to it for a really long time, which was we were allowing people to scale. make it bigger and smaller. And I think to us, that felt quite natural. But as soon as we play tested it with people, many people found it very, very confusing. Like, why is it getting bigger? Why is it getting smaller? I want to move forward, but then I have to move this thing. There was like this weird co-location problem where people would forget that they could move with their feet. So they were trying to swim everywhere and scaling all at the same time. And it was kind of like this user interface mess for a long time. And then I think at some point we made the decision, oh well let's just lock it down vertically as well, so you can't actually translate the thing at all. If you want to get closer or further away, you have to physically move. I think that was really the light bulb moment, because then it became much more personal. So, okay, now if I want to be in the space of these characters, I have to physically move closer to them, or I have to physically move away from them. And then it suddenly felt like it really gelled with the performative aspect, I think, that suddenly it wasn't just a mechanic, it was actually your physical relationship with the characters. And I think that is what I found most compelling about VR is this intimacy factor. Your proximity to characters seems to somehow modulate your emotional response to them. So being able to control that with a natural motion, like moving my body, as opposed to something kind of abstract, like moving my hands, I think that's when it really started to feel like it was going to work.
[00:11:46.755] Raqi Syed: As filmmakers, we often joke about how we miss the camera in virtual reality. Like, I wish I had a camera, but I feel that scale works like camera, like playing with scale. And that's something that we want to do more with our next projects is work with the miniature, but then also with maybe a half 0.5 scale or room scale and mix them up to varying effect. I also think with this interaction mechanic that we have, because we're a team of VFX artists for the most part, we spent our life in 3D. Like I have spent so many man hours just sitting in a 3D world, manipulating things. And that's what you're doing when you're making visual effects is like you have these environments and you have assets and objects and you're just rotating them all the time. And that feeling of like holding an object or a world or world building being immersed in the space is very intuitive to us. Like, I like being able to do that. So I found that when I was in the Museum of Realities, I loved being there, but I want to talk to Robin about this. But like, what if you could go up to the art and pick it up and manipulate it like a Minimum Mass style thing? Like, that would be super powerful. And that's what we always want to do in the museum, so we're not allowed to. So yeah, I think that ability to manipulate an object and the intimacy that you get with it, and also the sense of control of like cinematography, all these things are kind of working together.
[00:13:20.333] Areito Echevarria: I think the other point about that is that, or for Minimum Mass anyway, is that one of the themes is point of view. So there's two characters and it's kind of about the male and the female point of view of miscarriage. And that's something that we wanted to explore because you know, miscarriage itself is something that doesn't get talked a lot about in society and males point of view of miscarriage certainly doesn't get talked about very much at all. And actually remember, maybe this is a bit of a too personal story, but I remember my brother in law sending an email at one point saying, you know, I went through the same thing and, you know, empathizing with me. And I found that really touching and actually kind of unusual. But anyway, that's also part of the reason why we chose this mechanic, because we wanted you to be able to see the story from anyone's point of view. Like, I'm going to follow Skye for a bit, or I'm going to follow Rabia for a bit. So I guess it's a thematic choice as well.
[00:14:15.864] Kent Bye: Well, I found that the use of this black smoke three times throughout the piece to be really an evocative and powerful metaphor for the grief and the loss and Usually when you think about like the hero's journey. There's like things about loss and grief that you don't actually see a lot of stories about the filmmaking medium has a certain amount of pacing that you expect and There's something about VR that takes you to a place that I think you're able to be into these different contexts that maybe, you know, in a theatrical production, because there's like these extended takes that you can move around, but it's essentially like these one shot takes, but to go into these different contexts and to see how you use both the smoke, but also this black hole world, which is like this fragmented world already, and then it starts to crumble around. There's ways in which that context and those visual effects were able to somehow metaphorically describe different aspects of this experience. Maybe you could unpack that a little bit more in terms of how you were able to translate those emotions into that set design, as well as some of those special effects that you were using.
[00:15:23.677] Areito Echevarria: Yeah. Sorry Rocky, the void itself was an interesting one because Rocky had for a long time envisioned that as like a light space made out of light more than anything and then as part through the production process we figured out that that just didn't look very good so it became these tentacles at some point and then we went through a phase where we were like really fighting the tentacles we were like This is not what we wanted. The thing that we wanted wasn't working. And then at some point, we just sort of realized, oh, we're inside a womb or something like this. It was kind of like it told its own story somehow. And I really like this way of making things where I kind of reject, in my thesis, kind of reject this classical approach to design, where you have some base form and then you add detail to it. I'm not really a big proponent of this. I'm much more in the sort of Gothic style, where the structure of the thing is actually what's important in Gothic architecture. You have the ribs, which are kind of like walls, but they're also windows, and they're all sort of part of the whole structure. And I feel like design that uses that process is really interesting, where the detail somehow actually becomes the structure as well. And I think that's what happened in that specific case. It's sort of through the working process, it told us what it was, which was really cool.
[00:16:42.011] Raqi Syed: Yeah, and I think that's an exciting aspect about us now being indie artists is that we get to work intuitively, whereas when you work in big budget projects, you don't get to work intuitively like artists are always making intuitive decisions but with franchise cinema it's all very prescribed and you know at every stage you sort of are taking notes and doing things iterating but with Minimum Mass we allowed ourselves to work intuitively and this is something I'm kind of like I'm really interested in alchemy as a methodology like from a psychological point of view and I guess this is a thing that Jung and the Jungian alchemist psychologists talk a lot about which is the black sun and the idea of the dark and the light, so the shadow and the light. And for me as a lighter, I find this really compelling because I find it interesting on a thematic level, like what's happening. As an artist, you're allowing yourself to explore your shadow self, which is what minimum mass is. There's like the shadow world, which is the void, and then there's the real world of loss, miscarriage, and these two people need to integrate. They need to integrate those two parts of themselves in order to be whole. And then from a very practical, like as a lighter, I find all of this very compelling, like how do you sit in the darkness? How do you allow the darkness to bring about illumination? And so from a formal perspective, this is what I was really interested in with Minimum Mass, with the art direction. And we were looking at, there's a photographer in particular, an American photographer, Todd Hedo. who does these really beautiful, noir, suburban landscape photos. And it's noir, but it's not dystopian. And that was really important to us with Minimum Mass, is like, we don't want to go down this dystopia. Even though there's these things that could be dystopian, like the world is fracturing, there's black holes, it's not dystopian. It's all about moving towards utopia or redefining what utopia is. dark and noirish and it's exploring the shadow self, but there's always like hope and possibility there as well. So in a very practical level with the light, that's what I wanted to do as well.
[00:18:51.638] Kent Bye: Yeah, and there's this world that is either like a brain, neurons, or a womb area. And the lighting there was very dynamic. And I was moving my hands around and seeing the dynamic lighting. And the lighting in those scenes are very evocative. But what was that describing? Was that the womb? Or is that the light versus the dark? Or maybe could expand upon those polarities between the light and the dark and how you conceived of those contrasts between those two?
[00:19:16.745] Raqi Syed: Well, it's really about this very basic human desire to return to the womb. And we're finding this playing out in an interesting way with our son, who's now three years old, and he's very nostalgic for his baby self. And he's like, he wants to, this is kind of funny and weird, but he like wants to go back into my tummy. He's always saying that. And I think that he's just giving voice to a thing that it's like the first human loss, right? This loss of like the womb and the safe space. And working with our visual effects designer, Sunny Teich, she really spent a lot of time figuring out what is the visual design language of these tendrils? Should they be angular? Should they be sharp? Should they be rounded? And then also how we animate them. So we initially wanted them to move in a particular way that was becoming too expensive. for virtual reality. And so she did this vertex animation where it's like, almost like this undulating, which was the efficient way to animate it. So like the design kept being, we had to keep coming back to the limitations of VR and that formed the design itself. So the movement, the animation, that very like undulating kind of form, viscous, like we kept talking about how it's a viscous space. We knew that we didn't want the black hole to be a scary place, actually, because this is where they're going to integrate their shadow and ego. So it had to be a place of warmth and invitation, of mystery, but also comfort. Yeah, I don't know if that answers the question, but that's really what we were trying to do there.
[00:20:51.757] Areito Echevarria: I like this brain synapse idea, Kim. That's cool.
[00:20:56.328] Raqi Syed: I guess the white light going off, like, yeah, it does have a synaptic quality.
[00:21:01.816] Kent Bye: Well, in the transition, it sort of like has the rush of white. It's almost like the synapse firing. But, you know, as I was listening to you, also evokes the baby floating in the womb.
[00:21:11.805] Raqi Syed: I do, I do want to add one thing like to Areto's earlier point about what we wanted to do initially, like, I don't know if you're familiar with Thomas Willifred. He's a, he was a light artist, like in the 1940s. And you've probably seen Tree of Life, the Terrence Malick film. Yeah. And you know, in the beginning, it starts with that. It's like a flame that's dancing. That's Thomas Willifred's art, like Lumia art. And He never allowed anyone to film it until the Malik estate came to him and said, look, we have the technology now. We want to film this because it's really beautiful. We want it to be the beginning of the world in this film. And what he did was he was very informed by what was happening in the 1920s and 30s with the popular culture starting to understand how time and space were working with the publication of Einstein's theory of relativity and all this stuff. And so suddenly artists had this anxiety about where we sit in the world in terms of the vastness of it in time and space. And what Thomas Willifred was doing is he was creating these physical lightscapes in museums that you walk through and the light is warm and it engulfs you. And that's what we wanted to do initially is we wanted to create a Thomas Wilfrid light scape. So you're embodied in light, but transparency is terrible in virtual reality. And, you know, once we started working with Sunny, she was like, look, from a simulation and a shader and a material point of view, like, I just don't think we can pull this off. So we got to try another approach.
[00:22:46.278] Areito Echevarria: Yeah, I mean, I think this is something we found again and again, as Rocky mentioned before, this idea that VR is kind of like the stubborn medium and you go into it with a kind of background and you obviously want to take your past experience and push it into the medium, but so it just doesn't want to do some things. It's just like, no, not doing that, sorry. But then some things it is incredibly inviting and wants you to pursue further, I feel like performance, for example, works wonderfully in VR. But yeah, some things it's just like, Nope, not gonna work, sorry.
[00:23:19.278] Kent Bye: Yeah, the lighting in this piece, that was something that I was struck by. Not only the dynamic lighting in the womb scenes, but also the baked lighting in the different scenes seemed to be very intentional. I mean, I've seen a lot of scenes that don't have good lighting and it makes a big difference when there is some consideration to the lighting.
[00:23:35.712] Raqi Syed: It breaks my heart, the lack of lighting that I see in virtual reality.
[00:23:41.295] Kent Bye: But it looked like a lot of, a lot of like baked lighting in there. Um, some dynamic lighting within the womb scenes, but maybe you could just sort of expand a little bit about your lighting strategy when you were going through this piece.
[00:23:52.742] Raqi Syed: Yeah, well, what I wanted to do in the beginning was looking at the Todd Hedo photographs and what we know Unreal does really well with open world games is we wanted it to be a very volumetric space, like volumetric lighting, misty and emissive light sources. And we had to stop with the volumetric because it was becoming too expensive. And we then sort of like just rolled back to the most basic thing, like what is happening in the Todd Hedo artwork? In the Todd Hedo artwork, It's noir because we always understand where the light is coming from. It's nighttime. It's single source lighting. There's emissive windows that illuminate these very dark spaces. And that actually is something that allows us to work to the strength of the real-time engine. So we redesigned all the lighting so that there was only ever one dynamic or stationary, actually not movable. So Unreal has these different categories of lights. What you see in the womb is a true dynamic light, meaning The light is moving, objects are moving, shadows are casting. That's super expensive and you don't want to do that as much as possible. We said we were going to do that in the womb and everywhere else we were going to have a movable light so that when characters move their shadows are being cast but everything else is static and baked. And it lent itself very well to the noir aesthetic because noir is so underlit and it's so single source. And then out of that, you get something that's quite stylized. So that was really the reference. And then from a practical point of view, like the genre was the cue of how to keep consistent with that across each sequence.
[00:25:34.663] Kent Bye: Do you have anything else to add about the lighting?
[00:25:36.864] Areito Echevarria: Yeah, I mean, I was just admiring Rocky's lighting, because I do think she did a fantastic job with the lighting on this project. It looks really cool. Rocky's in a different room right now, but in her office, the whole wall is covered in lighting scripts of what it was like.
[00:25:51.150] Raqi Syed: Yeah, I can show it to you.
[00:25:52.811] Areito Echevarria: Every day I go into the office, it's always very inspiring, because I see them as I walk through the door, and I'll be like, ah, yes, I'm a mess. Here we go, we're doing it.
[00:26:03.689] Kent Bye: What is the lighting script? Like, is there a notation for lighting or what's that mean?
[00:26:07.270] Areito Echevarria: Yeah. So like a visual reference of this is what the scenes are going to look like. These are what the lighting values are going to be. This is what the colors are going to be. So you can see what it's going to look like.
[00:26:17.093] Raqi Syed: On a technical level, we got really detailed on it. So looking at a lot of references, like Todd Hedo was the main reference, but then also Because we have this cinema language that we're constantly working with, because we love movies, we created mood boards, visual mood boards, for every single sequence and a color space. So to get really deep into the lighting, there's the warm light, which is the color of the room, and there's a specific HSV value for that, like a warm light temperature. But then there's also this green mercury vapor light. So mercury vapor is a particular kind of electronic lighting that used to be used a lot where like mercury gets heated up and then when the environment is cold, what you get is a very green palish cast. So when you look at like throughout the history of art at paintings like Edward Hopper, you look at vertigo, you look at, of course, the matrix is the one that really pushed this into our popular consciousness, but the green light is like the light of urban alienation. And it's a real light. It comes from mercury vapor, which went out of style because people didn't like the way it made skin look when it's green. But in minimum mass, you'll notice that like in the hospital scenes, in exteriors, you see this green light, and that's what that emotion is that it's trying to capture. And so, you know, working with that as the reference as like the visual script as a ratio said to make sure that when certain things are happening you're getting the color cue lining up right down to a specific hsv value right down to a specific exposure and then following the basic rule of cinema lighting which is that like beauty lighting you always have to light for the character so like When Rabia and Sky have an important moment, they step into the light and they get this beauty light, which is like Rembrandt lighting, whatever you want to call it. And then everything else we allow to fall away from there, which is why it's quite naturalistic. Like it's okay that they sometimes move into shadow. I don't need them to be perfectly illuminated all the time. And that adds a layer of realism that I think as audiences, hopefully you're not aware of it, but that's what happens in cinema. And that's what I think people are often afraid to do in computer graphics because they've built these elaborate worlds and there's all this detail. want to make sure people see it so then they over light and I think over lighting is like the scourge of computer graphics because you don't get shape when you over light you actually flatten so you're getting the opposite of what you think you're doing you add too much light.
[00:28:52.297] Kent Bye: Fascinating. Yeah. And I noticed, Aredo, you were talking about the performance because you do have two actors in each scene and sometimes three, but I noticed that it was very fluid. It was like motion captured or in a way that it felt like it was captured from humans. It had a human quality, but there was no facial capture, or at least it was so small, I couldn't make out anything. So it was almost like you were using the body for the performance rather than from the equivalent of the different shots and film, it was like everything being shot from a wide shot, without any of the close ups up until like till the very last shot where you get to see the close up shots. But maybe you could talk about the technical aspects of the performance and how you're able to pull off that aesthetic.
[00:29:32.682] Areito Echevarria: Yeah, so first of all, it was really down to Frankie Adams, our lead actor, and Alan Henry, our lead actor, so Ravi and Sky, and Kerry Thiel, who played the Doctor as well. They're all fantastic performers and they were just a joy to work with. That's where the bulk of the work came from, is these amazing actors and their wonderful performances. So yeah, it was all motion capture. There is facial actually, however, due to budget, it's not all like super high quality facial capture. So some of the scenes have high quality facial, like for example, the most of the date scene is fairly basic facial, but then towards the lighter part of the scene, the end of the scene where they come close together, that kind of switches into high quality facials. If you peer in, you can really see the performance coming through. And I think on the proposal scene is like the high-res facial as well. But yeah, so That was a really interesting process. We had actually, we'd blocked the whole thing a few times before we actually did the shoot with our actors. So with our research assistants and ourselves as well down on the stage, we blocked the whole thing out a few times to try and figure out what the staging of these scenes would be and where the sets and props would live and this kind of thing. And that was a good process. We tried to do it in this kind of iterative fashion. mainly down to budget, I guess, which was like we didn't have time or money to spend weeks and weeks and weeks with Frankie and Alan. We only had them for a week in the end. So we wanted to get as much of the staging worked out ahead of time. And I think like the philosophy that we used was to keep the action playing fairly close to the perimeter of the volume so that when you have these scenes in your hands the main action should always land up so that the characters are fairly close to you so instead of being at the back of the stage they're on the near side of the stage to the participant so that you could then choose to move in or move away depending on how close you as a viewer want to be to the drama so that was kind of like how we ended up staging everything, really just kind of keeping the main beats to the edge and then allowing them also to move in and out of the volume to give some sense of depth and feel like they're actually moving around a real space. I mean, I love working with actors. It's one of my favorite parts of the whole thing, just seeing them take the material and transform it into something really real. Frankie and Alan showed up on the Monday and they'd never met each other before. So that was one of the things I was worrying about in the back of my mind is that okay you know they had this intense intimate relationship in the story but they've never met each other so how's how's that going to work but in the date scene if you recall they have this list of questions that this guy is just kind of like reading this thing some list of questions they got off the internet or something like this which is an intimacy study How to make someone fall in love with you instantly or something like this. And this is actually based on a real psychology study.
[00:32:31.312] Kent Bye: 36 questions to fall in love. I think it is.
[00:32:33.013] Areito Echevarria: Yeah, yeah, yeah. There was like a pop science article or something written about it. But the actual paper is really interesting. If you read the paper that that material is based on. And it's this long list of questions. I think there's like 50 questions or something like this. And they get progressively more and more intimate and the idea is you sit with someone that you've never met before and you ask these questions back and forth and through that process you begin to empathize with them or fall in love with them if you want the pop version of it. So I had Frankie and Alan do those questions when they first met and I hadn't done that before with actors so I wasn't sure how that was going to work but then you know within 15 minutes they're just like sitting there telling each other their like deepest secrets and all about their mother and their father and it was really cool and they just like clicked like that after that experience and then they were Sky and Rabia and then they like really understood each other right from the get-go.
[00:33:30.068] Raqi Syed: And I also think that part of the secret sauce of the Frankie and Alan thing that worked was part of it is luck and part of it is just where we are in New Zealand. But we got connected with Liz Mullane, who's pretty much like the best casting agent in New Zealand, if not the world. Like she cast all the Lord of the Rings films. She cast all the Hobbits. Like she's super well respected and amazing. And through a friend, She gave me some time one morning. I went and had a coffee with her. I told her about the project. She's like, right. Okay, I'm gonna help you. And she brought Frankie Adams to the project, which was just like amazing. And then also she said, because Alan Hendry is really well known at Weta because he's done a lot of performance motion capture. So if you watched Avatar, he's played all the characters in Avatar. He's doing stuff on Avatar now, so he's been around for a long time, and so she knew Alan as well, and she said, yeah, I think Alan and Frankie will have amazing chemistry, and they'd be really good together. And that was her experience and intuition as an art director that really made that happen, and then her generosity in bringing them to the project. I think that was an important part of it.
[00:34:45.538] Kent Bye: I'm curious to hear a little bit more of your own experiences of creating a piece of art that is in a topic that not only very taboo to talk about in public, but also filled with just a lot of emotion and grief and You know, I've had an experience of doing art like that. And it's almost like you get so lost in the technical details for so long. And then there's a certain moment when you can actually sit back and watch it for the first time and then take it in. But it's a long process to get to that point. But I'm just curious to hear your own process of creating that art and what it was like for you to maybe really fully receive it for the first time.
[00:35:26.103] Raqi Syed: Well, we should talk about what Mark Monroe told us at Sundance.
[00:35:30.424] Areito Echevarria: Mark Munro is a wonderful man. He's a writer slash editor, you probably would have seen. He wrote The Cove and some amazing documentaries.
[00:35:39.468] Raqi Syed: Icarus, yeah.
[00:35:40.609] Areito Echevarria: Yeah, so he early on gave us this, it wasn't really advice, it was more like this- Caution.
[00:35:45.631] Raqi Syed: Caution.
[00:35:46.011] Areito Echevarria: Advice caution. Which was like, he said to us, well, this is pretty intimate material here. Are you sure you want to spend two or three years talking about this in public with people? And we were like, uh, I hadn't thought about that.
[00:36:00.988] Raqi Syed: Well, everyone else at Sundance was like, really into the story. And because Sundance is so like, it's run by women, all the women were like, yes, you need to lean into this. And Mark Monroe was like, are you sure you want to live with this for two years?
[00:36:17.159] Kent Bye: This is the Sundance Labs, right? To be able to workshop it. Yeah.
[00:36:21.793] Areito Echevarria: Yeah, but I think and then we sort of brushed it off a little bit and we're like, it'll be fine. And then it was really hard. It was like super hard talking about your personal stuff in front of people, friends and strangers for a really long time. It's incredibly emotionally draining. But I think at a certain point, I felt like I just had to do it, like it needed to be made, needed to be finished. And I think ultimately, I want people to feel that they can talk about this stuff. And I think that's one of the purposes of art, really, is it's like a cultural document that allows people to talk about things and have continued conversation about what the piece that you're viewing is about. And I thought that was important because nobody does talk about this stuff. So I kind of just, you know, I had to wear it. I mean, I'm kind of a bit of a fighter, I suppose. I did boxing and Muay Thai and this kind of thing, so I'm used to getting beat up. But you've just got to stand up, you know. You've just got to get up and get on with it and move on and finish it. And then I think the next thing that sort of happened is, especially after we'd worked with Frankie and Alan, we had the actual performances down, and we really had a clear idea of how it was going to feel to experience it. And I was playing back the scenes or watching the scenes, and I was getting emotional and crying watching it. So then at that point, I knew we sort of had something. It's still really affecting me after two years or whatever of living with it. I'm still getting emotional watching this stuff. then I felt like, okay, this is good. This is good work. Yeah. And I think I've sort of made my peace with it a little bit. I don't, yeah, I feel good about it. I feel good that we went through that process as painful as it was.
[00:38:10.828] Raqi Syed: Yeah. I think it's a combination of catharsis and trauma and constantly It's a cycle. That's how I felt working on the project, that it was allowing us to get this stuff out there, but then re-traumatizing yourself by having to talk about it and live it. Literally inhabit the world day to day is re-traumatizing. But then I also think as an artist, this is what people mean when they say you have to have skin in the game. This is what was at stake. It is incredibly vulnerable and difficult to keep putting yourself out there and you're like, ripping off the band-aid over and over again. But that's why I think it's compelling and hopefully good. Hopefully it's good because that's where the skin in the game is. It's like you're actually putting yourself out there and opening yourself up and it's difficult.
[00:39:02.228] Kent Bye: Hmm. I'm curious to hear your thoughts on, of all the different mediums that you could have told the story. Why do you think VR worked the best to be able to tell this particular story as opposed to film or theater or any other medium? But like, what do you think it is about the VR that allow you to do things that maybe you couldn't do in other mediums?
[00:39:23.156] Areito Echevarria: I think it's certainly like a lot that we don't know about that question. There's a lot left to be discovered. But for me, the main thing is this proximity effect. When you're close to a character in VR, there's just a certain emotional response that at least I've never felt with any other medium. As soon as a character in VR gets into my personal space, the hairs on the back of my neck stand up. And it's very, very impactful, I think. And as much as I love film and games, have those kinds of experiences far less often, or at least not to the same intensity that you do in VR. And somehow VR has this intensity, a motion intensity slider that you can move around very precisely, that I think is much more difficult to do in film or games or theatre. In VR, it seems you can be very precise about how you want someone to emotionally respond to a piece, which I find really interesting. So I guess it's just a different kind of instrument, but it's one that I feel like, yeah, it's special in that way. It's got some special notes or something that you can play.
[00:40:31.067] Raqi Syed: I think that people have historically talked about cinema like the power of cinema is that it's a collective dream that we're all together in this place traditionally in the cinema and we're collectively experiencing the same emotions and so when we leave the theater and we walk out onto the street and it's bright it's like we're all waking up from a collective dream and that's really powerful because When you're in a movie, when you've had a great movie experience in a theater, it's like you've shared something with someone else, and there's a very emotional way of processing that. And VR, I think, it has a similar effect, but the opposite. It takes a totally different tack, which is, it is the singular dream. We really thought about, this is like something that Sacha Anselm told us at the lab, that you've got to think about the volume as a black dreamscape. And that was one of the decisions that we made is thinking about that, like not building out the world and having these almost like these forms emerge from it is like that feeling of like this is your psyche and this is the dream and the forms can come out of it and really using the negative space to your advantage. So I feel like when you go into the headset and you have this singular experience, you're like actually allowing yourself to be invited into a dreamscape. And you need to be by yourself in order to do that. In order to have that emotional connection with this world, you have to be by yourself. And it's like an amazing privilege in VR, too. You can't second screen VR. You can second screen almost everything else right now, particularly in the narrative space. But with VR, we still have the privilege of 100% attention. And I think that's really special and unique. And that's why I'm much more interested in virtual reality than augmented reality, because I think from a storytelling position, it gives you power that no other medium does right now.
[00:42:30.041] Areito Echevarria: I was just going to say, I mean, I know there's some really amazing social interactive angles to VR that possible. You know, like the Museum of Realities we were talking about earlier is an awesome example of this. You can go into this virtual space and talk to strangers, which is really cool. But there's also this very personal side to it, which I find fascinating, which is, you know, it's just you. It's just you and the piece. There's nobody else there. And I think that's really interesting.
[00:43:00.894] Kent Bye: Well, you both said that you were researchers and that you're both artists as well. So you're kind of using art to research. So were there any specific research topics that you were exploring in the process of Minimum Mass?
[00:43:12.891] Areito Echevarria: Yeah. I mean, there were some technical ones initially, like we were, this manipulation scheme was like some technical research we were pursuing for a long time. But I think ultimately what we found more interesting was just understanding the medium itself. Like what this idea of like the stubborn medium, how does it like to be manipulated and trying to understand that as part of our process. Cause I do feel like it's kind of like this, VR is like a, I don't know, it's like some kind of puzzle where it's like a rock that's hard if you hold it one way, but if you turn it a different way, it becomes very malleable. And if you move it quickly, it becomes hard. If you move it slowly, it becomes soft. It's like got these really weird counterintuitive properties to it. And that was, I think in the end, the thing that we became most interested in from a research point of view is trying to understand that form a little bit better. And we've practiced based researchers for the most part, meaning we're using the outputs to understand things more clearly rather than trying to solve generally for any kind of particular problem space. So yeah, I think that was like, ended up being the main thrust of what we're trying to figure out.
[00:44:20.699] Raqi Syed: Yeah, one of my areas of research in addition to lighting and figuring out how do you teach real-time lighting, how do you apply the principles of traditional visual effects to real-time lighting is, and I think you and I can have talked about this before at DevLab, is like the media archaeology stuff that I do. And I'm really interested in looking at the history of visual effects and how visual effects as a medium has shaped storytelling, has shaped cinema. And so George Lucas is this interesting person, like you can keep kind of like finding layers of meaning the further back you go into his history. Expanded cinema, the history of expanded cinema, I think is particularly relevant to VR because VR is expanding cinema in a way that hasn't really happened since like maybe the 70s and the 80s. And so there's these great interviews that I found between George Lucas and Gene Youngblood, who wrote the expanded cinema book, where very young George Lucas and Gene Youngblood are wandering around in like Malibu. talking about cinema and what cinema could become. And, you know, George Lucas says like, well, I think what needs to happen is that people need to have their own way of exhibiting and distributing this stuff. And then once that happens, the studios will just have to listen and audiences will vote with their wallets and you can kind of dismantle the structure of Hollywood. And I guess what people don't really understand about George Lucas is that he was very radical when he first left film school and he really wanted to dismantle Hollywood and it's this interesting irony that he reified Hollywood through the dismantling of it. by creating his visual effects empire at ILM. I mean, there's all kinds of interesting things about labor and the studio system and how that played out within that world that he created. But his ideas of expanded cinema at that time have really come to bear. And I think he and Coppola and all these young artists at that time were striving towards what's happening now. I don't think George Lucas knows about the Museum of Realities, but if he did, I think he would be so jazzed, because this is what they were trying to get at, like, in the 1970s and in the late 60s. They just didn't have the technology, and we weren't there yet. But that's one thing that I'm really interested in, and with Minimum Mass, like, it's a practice-based thing, and we're doing it for a lot of reasons, but how it sits into this, like, trajectory of expanded cinema and the work of our peers, I think is like really fascinating.
[00:46:53.331] Kent Bye: Yeah, I know we were talking a little bit on Twitter, just about the Maori and Maori design. I don't know if you want to just quickly expand on what that is, because there was something else that was taking some of these design principles from the Maori and maybe you could describe what that is and how that fits into everything.
[00:47:09.756] Raqi Syed: Yeah, I mean, one of the things we were talking about early on with the project was the Maori mythology about how the world was created. And there's this really beautiful story about a mother and father. They're lovers and they're locked in an age long embrace and their children are born into the crevices of their bodies. And over time, the children get really crowded and they're being suffocated. And so In order to live, they have to destroy the closeness of their parents. They have to push their parents apart, and that's the creation myth. That's how the universe is created. And Tain, one of the sons who, with his legs, pushes his father and his mother apart. is the god of the natural world, of the green world. And this is like an amazing story. It's not a story that we felt we had ownership over because neither of us are Maori, but it was a very beautiful and informed like how we were thinking about the story when we were writing the script at some point. And so for both of us as teachers, as academics and researchers at Victoria University, we have this responsibility where you are really encouraged to integrate the Treaty of Waitangi into your course material. And so from an art perspective, what we've been looking at is what does that mean from an art perspective? And for me as a lighter Traditionally, for the last three years or the first two years I taught my class, like, let's begin with the Renaissance and let's look at Rembrandt lighting and the principles of beauty lighting and portraiture and how that translates to Hollywood styles of lighting. And what I'm doing now is I'm going back and I'm teaching myself like the history of lighting and realism and portraiture in the New Zealand context. So There's these painters, these European painters like Lindauer and Goldie that came over during the early colonial period and they created this amazing body of work. It was very Rembrandt oriented in terms of the style, but they were painting the Maori chiefs. And these are now the documents of that period of history of all these people that exist through these portraits. And so trying to understand like, because we're in New Zealand and we have this responsibility to decolonize the subject matter and teach it in a way that is resonant with place, I want to integrate that history more into the subject matter.
[00:49:34.563] Areito Echevarria: One other anecdote related to that, maybe it's kind of serendipitous, so the Ranginui and Papa were the Sky Father and the Mother in the creation myth that Rakhi was talking about. So that's actually where the Sky character name came from, Sky, and Rabia Sky meaning the Sky Father. And it turned out just kind of like coincidentally that Alan Henry's middle name is Ranginui, which is the sky father.
[00:49:59.234] Kent Bye: Oh wow.
[00:50:02.836] Raqi Syed: Yeah and then of course like Rotorua as a place, like Rotorua is a very beautiful place in New Zealand. It's a real place and it's an important place to me because it's the first place that I went in New Zealand after coming to Wellington and spending all my time working at Weta Digital, the first trip that I took outside, and Areto and I went together, was to Rotorua. And it's like, when you go to Rotorua, you are immediately struck with this sense of its otherworldliness, that sulfurs, that steam that you see in minimum mass in certain scenes, like in the opening scene in their house. Then in the date scene, like that's everywhere. There's just like mist sort of like growing up out of the ground. And that was really important thing to convey because it was part of my, I guess, origin story of like coming to New Zealand and starting to understand what this landscape is about, that this landscape is not Middle Earth. I know people want to call it Middle Earth, but it's uniquely New Zealand and it's tied into the Maturanga design and the New Zealand Maori mythology. All that stuff is so evocative and amazing. That's what it really is.
[00:51:15.749] Kent Bye: I noticed that you had put that in the description where it takes place, where it could really take place anywhere. It's to some extent, but there's certain things that you're adding in there. But it's sort of a dual place because it's also in the black hole. So it's in a real place in New Zealand, but also in this black hole, and you're kind of fusing these two worlds together, but trying to fuse in different aspects of that context. And as you're Describing it, I can see all the different threads of, you know, the little mist that was coming up in that scene. I noticed it, it was very evocative, but now it's actually reflective of that place. So just the concept of trying to actually situate yourself into somewhere and try to expand onto that context within the virtual reality is a very fascinating concept.
[00:51:55.126] Areito Echevarria: In the doctor's office, if you put your head in and look at the TV screen, you can see that it's the Geyserlands medical center.
[00:52:04.855] Kent Bye: Nice. Well, just to kind of wrap things up here, I'm just curious if each of you could share what you think the ultimate potential of virtual reality and immersive storytelling might be and what it might be able to enable.
[00:52:19.140] Areito Echevarria: I mean, I think it's kind of endless. It's a bit of a cliche, but I do feel like we're at the train arriving at the station period of VR. I'm super excited to keep working in the medium and see what we can do with it. It feels like the innovation kind of comes in waves, like people think they've worked everything out. There's like these preconceived ideas of how it should be. And then someone does something that's totally left field and reconfigures people's understanding of virtual reality. I think that's really exciting. I mean, me personally, I'm most excited about doing more character based stuff and dealing with this proximity, this intimacy question a bit more detail. That's personally what I want to work on next.
[00:53:00.098] Raqi Syed: Yeah, I mean, I think that a bunch of things are happening that make right now a really unique and special time. We've talked before about how the difficulty of visual effects is that there's so much friction at every part of the production process. It's hard, it's expensive, it's very technical, and there's all this specialized knowledge that is required. to make something look good and there's still a lot of friction but it's easing up and so most of the tools that we use we use specialized knowledge but we used off-the-shelf tools for the most part like we didn't write any of the software ourselves we didn't use a ton of code to like create bridges between different pieces of software we tried to use as much off-the-shelf stuff so that we can teach our students this is where the research comes in is like we figured a thing out and now we want to share that knowledge and help other people young artists be able to use those same techniques and tell their own stories. And I think that's what's cool is that we're actually at a time where we can start to do that. So like the digital human stuff, for example, which, you know, there was a keynote by the digital domain, one of the digital domain guys about this. And people are always talking about digital humans, but the reality of it is, is like those digital humans that the studios create are very expensive and very time consuming. But, and I think to his point, like he did say, like you mentioned character creator, which is a tool that we use. And we want our students to be able to use these tools. And I think a question that no one at the studio asks that I'm really interested in asking for my next project is not just how do we create digital humans or why, but who gets to be a digital human? That, I think, is the most important question that we should be asking right now. I think digital humans is a kind of privilege that we haven't unpacked in terms of who gets to be one, who gets to talk about it. And we can actually start to now, in some ways, like democratize that process. And I think that's going to be very powerful.
[00:55:02.140] Kent Bye: Wow. Great. Is there anything else that's left unsaid that you'd like to say to the immersive community?
[00:55:09.984] Raqi Syed: I just want to say thank you. You know, when we started this journey in the beginning of 2018, we started at the Sundance Lab and then at the end of the year, we went to the Dev Lab and then to CanXR in 2019. And along the way, like, this is my favorite part about going to festivals is like meeting our other creators, people like you. People like Floreal Films, like Kate Yoon, for example, is someone that we met at Dove Lab. And then at Cannes, she came up to me and said, I want to work with you. And that's how it happened. So we always go to these festivals thinking, I need funding. I need distribution. I need to talk to someone who's going to help me out. But in fact, we talk to people at our own level, and we help each other out. And that's what I think is super cool about this community.
[00:55:55.898] Areito Echevarria: Yeah, I think the same thing. Actually, the most thing about working VR is the community. I think there's like so many interesting, fun, loving, giving, wonderful people in the community. And it's just, yeah, that's really fun and really inspiring. It's just like, everyone's trying to do cool stuff. They're all cool people. And yeah, I love that.
[00:56:19.496] Raqi Syed: Yeah, I feel like we should really quick just give a shout out to this international community of this village that made this project happen, right? The Sundance folks, the Dev Lab folks, Rene, L'Oreal Films, the CNC has given us funding, the New Zealand Film Commission, Epic, all these people from all over the world have come together to help us make this project and to deliver this project into the world.
[00:56:48.561] Areito Echevarria: And Eskimo for his fantastic score.
[00:56:51.546] Raqi Syed: Yeah, that was a huge coup actually convincing him to be involved in the project. He's amazing and the music is like so powerful, I think.
[00:57:01.924] Kent Bye: Well, I'm going to try to get this podcast out before July 3rd for people to go see it in the Museum of Other Realities, download the DLC for Tribeca, and you can see it at the end of the hallway. But after that, what's your plan? Are you going to do more festivals? Are you going to try to distribute it onto Steam? Or like, how can people in the future see this if they don't have a chance to see it at CanXR?
[00:57:23.811] Raqi Syed: Well, that's our vision. All those things are our vision is like maybe a handful more festivals, but We're really thinking hard about what our distribution strategy is now. That's our goal is like we want everyone to be able to see the project.
[00:57:40.436] Kent Bye: Yeah, well, there's certainly a lot of really innovative stuff that's happening in this piece. And the visual style is very striking. And also just the story is very heartfelt. And I could tell that it was coming from a real authentic and deep place of grief and trauma and catharsis, as you're saying. So yeah, thanks for making the piece. And also just for joining me on the podcast today to be able to kind of unpack it and talk about all dimensions of it. So yeah, thank you. Thank you, Ken.
[00:58:07.299] Areito Echevarria: It was wonderful being here.
[00:58:09.535] Raqi Syed: Yeah. Thanks for inviting us. This was really fun.
[00:58:12.756] Kent Bye: Yeah. So that was Erato Echevarria. He's a researcher and artist who's looking at different aspects of emotion and character performance, as well as Raki Syed. She's a researcher and writer, lighting director, and artistic director, looking at how to use real-time lighting in immersive experiences. So I have a number of different takeaways about this interview is that, first of all, Well, this is a very personal and intimate story and really resonated with what Rocky is saying is that kind of reminds her of the time period in the 70s when people would make really personal and gritty films on the independent budget. But in this case, they're using a lot of the pipelines that they used at Weta Digital, which, you know, was working on Lord of the Rings and The Hobbit and all sorts of other big films like that. So this piece has got a lot of things going on in terms of the set design. You're set in a black hole, and so you're in this fragmented world. The thing I think is probably most striking is that you have this mechanic where you're actually rotating this space around, and the size of the characters are fairly small. It's like a tabletop scale. And I think they wanted you to peer your head into this scene and walk around, personally when I saw it I was sitting down and I wasn't really moving around much and so I felt like I was actually pretty far from the distance and actually had to watch it a couple of times just to be able to you know know that I could actually put my head in and it was a little bit uncomfortable sitting down to be able to actually do that and so I would just recommend they continue to tune this because at the very beginning they tell you to stand in the center of your space, you push a button, it calibrates it. And that distance, I think you could stand to be maybe a little bit more closer to the action, or maybe think about raising it up to something beyond the tabletop scale, maybe something that's a little bit more of a medium version of that. But it was a little bit difficult to see the facial capture and the performances there. And they said that due to budgetary reasons, they didn't do like an equal amount of high resolution facial capture. And I think I would recommend starting off with a lot of high quality facial captures so that it's at least a cue to me that it's something that I might look for later, especially if it's appearing later in the experience and I'm sitting in a chair where the characters are fairly far away and it's actually hard for me in a low resolution vibe, which is what I watched this in instead of my index, just because I had some technical difficulties on my main computer, but the point being is that, you know, you could think about what scale and your closeness and proximity because Areto essentially said that, you know, your closeness and proximity can dictate your emotional reaction. And if that, by default, if you just sat in the center and you watch this piece, then you're actually pretty far away and you're expecting the audience to kind of go against their intuition and kind of stick their head into this like shoebox environment. And when I watched it, I could see, you know, it's actually pretty compelling, but I'd like to see maybe the default settings set so that, you know, the center is actually closer to a lot of the proximity of the action without having to kind of like stick your head in. But that's all stuff that, you know, by the time this actually gets released, that could be sorted out. But that's just my initial reaction. but the lighting here i think is probably one of the more striking things and there's different inspirations and that they were taking from they wanted to do something like really dynamic lighting from like thomas williford which is the beginning of the tree of life with a lot of his lumia but the dynamic type of lighting is very expensive they're able to do that a little bit within like the womb scenes where it creates this very evocative like feeling when you're in that it's a bit of a contrast between the green mercury vapor lit neon space that you know Rocky's saying is really trying to metaphorically describe that urban alienation and the warmness that happens within the womb and then the noir lighting that you see inspired by Todd Hito as a photographer. So There's lots of different inspirations that you're taking from different aspects of cinema and noir and art, and they're trying to fuse that together. They said they had a whole lighting script with mood boards and HSV values and color cues and all sorts of ways to be able to create these different contrasts. One of the things that Rocky said is that she's very inspired by Jungian psychology, specifically the alchemy aspects. And just to elaborate on that a little bit, you know, Jung was a lot into seeing how there are these different polarity points and you kind of have to like sit in the tension of the opposites until you have this third that comes and resolves that. And that was very similar to the Hegelian thought of the thesis and the antithesis, and you kind of sit in those two aspects and you kind of, as Ken Wilber says, you transcend each other limitations and you or including the good things, and then you take the thesis and antithesis and essentially you have this synthesis. And so I think that's a bit of what they're trying to do with having these different aspects of contrast and using the lighting from the green mercury lit alienation to the more warmly lit in the womb. And as you go in between these different scenes, then you're able to kind of like feel this tension that's a little bit more of a subconscious level. And I had to go back and actually watch it again to be able to pay attention to it a little bit more now that I had heard that that's what they were doing. And I did see that some of the characters are kind of stepping in and out of lights at certain times and just noticing what is being dynamically lit and when the characters are being featured and just how the lighting in general is used as this way of subtly highlighting different aspects. And so they're very interested in looking at, you know, how can you do these different types of real-time lighting that are going to be taking a lot of the lessons they learn to be able to teach them their story. But also this aspect of performance because they do have two actors that were acting and they're being motion captured and they're trying to create these theater staging like you're in this one scene and there's no cuts and so there have the different blocking they're moving around and in and out of the lights and Yeah, again, my sort of takeaway of that is that given the proximity and the closer you are, the more intimate that you feel, it does feel a little bit far. And so is the scale too small? Would it be better to kind of have it more of a medium scale? They want to have this sort of agency, but sometimes I found that actually rotating the world around was a little bit of a distraction. They could just set you where they think is the optimal point, but I think they wanted you to actually have the agency to kind of move around where you're essentially becoming like this cinematographer and you have to pick the best perspective. but at the same time there's sometimes where maybe there's just the optimal perspective that they want you to be at and you don't have to do as much work because sometimes you know you would start a scene and that the next scene would be like above my head and I would actually just have to pull it down to find it and so there's a little bit of that friction where I would prefer just to be set into the optimal space rather than having things jump around and having to kind of like reorient how high things is just to be able to see what action is having. And when you think about those user actions, then what is that really giving and adding to the story? If anything, it's just kind of like a distraction to really being immersed into this world. There was a scene at the beginning where you have a choice to be able to decide whether or not to go into the bathroom to follow the female protagonist or to watch the male protagonist. And the guy isn't really doing that much. If you just watch it, there's nothing happening. He's not saying anything. And so, and at some point, the lights go out, you know, cause originally it was like, okay, there's nothing happening here. I'm going to see what's happening with this woman who's getting in the bathtub. There's actually just a lot more stuff that's happening in that scene relative to the other scene. And when I watched it the second time, when I watched the alternative scene, it was just kind of like, okay, I'm not sure. Does there need to be a choice here? Because I'm not actually getting anything extra from seeing that. And I understand they're trying to show the variety of different perspectives, but I don't know. I think there's a lot that you can do with cutting into an entire scene and maybe designing an entire scene around it rather than having a beat track and you making a choice and a decision, because there's going to be 95% of the people who are going to make a certain choice. And so unless it's really given a cue where there's something happening or progress that's happening, then as a viewer, then I'm going to go over to where I see that there may be something that's building up towards something, because I don't want to miss the biggest action. So I think there's still work to do there, but you know, just thinking about when you make a choice, then what is it adding to the overall story? And if you make the wrong choice, then is it going to be just as good? Or maybe there's going to be situations where making that choice, you're able to get something different from that experience. There was a thing that from Sashka and so talking about how the VR experience to be able to use the negative space to your advantage and to be able to think about it as if you're entering in this dreamscape and you know, how can you go from what is typically in cinema, like the collective dream into like a very personal dream where you're able to really be immersed in it. It's like your personal dream. And the stuff about the media archaeology and the history of special effects and the history of expanded cinema, that was really fascinating to hear a little bit more about that because George Lucas back in the 70s saying that, you know, just wait until we're able to have our own ability to create all of our movies and be able to distribute them. Well, we're kind of getting to that point with where the Unreal Engine is and Unity and being able to create your own. immersive experiences with all these different special effects. And then the distribution, you know, there's obviously like YouTube and Vimeo and all the different online distribution channels that if you produce a piece that's good enough, you could actually get it into Netflix or Amazon Prime or Hulu or potentially Disney Plus or Apple and, you know, lots of video on demand places to be able to sell yourself. So we are kind of getting to that point that George was imagined. And Rocky's point is just that Lucas was trying to deconstruct Hollywood, but yet for the whole special effects taking over, he was reifying Hollywood to be able to double down and strengthen the institutions of Hollywood themselves. But I do see that some of these immersive real-time graphics and machines could have another effect of democratizing the access to a lot of these different tools. Because as they were producing this piece, they were trying to do all commercial off-the-shelf technology. So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listed supporter podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So, you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
