The Man Who Couldn’t Leave is a emotionally-evocative 360 video and powerfully-told story about political prisoners during Taiwan’s White Terror trying to communicate with their families. It took home the top prize at Venice Immersive 2022 for it’s many innovations of storytelling grammar and fully using the spatial medium of VR to find new ways of telling this story and moving the audience. Be sure to catch the behind-the-scenes video in order to fully appreciate the ambitious scope of the production, and the various multi-perspective theatrical staging, lighting, and art direction on top of many abstracted metaphors and poetic interpretations of the story that cultivated a dream-like quality. Funique also did an amazing job with the stereroscopic production & post-production of this project. I had a chance to chat with director Singing Chen through interpreter / distributor of the project Poshan Wu during the Venice Immersive Festival.
The first half of this interview is in English, and the Mandarin Chinese portion starts at 42:52.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. You can support me on Patreon at patreon.com slash Voices of VR. So today's episode is with the best immersive experience from Venice 2022. It's the man who could not leave. So this was awarded with the top prize at Venice and it's a 360 video. And as I was watching all the different experiences here on my shortlist, I kept recommending people to definitely check out. The Man Who Couldn't Leave and All That Remains. These two pieces from Taiwan that were on this whole other level in terms of like experimenting with the structure and the forms of immersive storytelling within VR itself. And so lots of innovation in terms of like lighting and staging and use of animation and blending different scales together. And it just was a real masterful piece and also just really emotionally evocative and a powerful, well-told story. So I had a chance to sit down with the director of Singing Chen to unpack a little bit of the process of creating this experience. I wanted to give a little bit of maybe a context before we start to dive in and also to point people to this video that is a behind the scenes video that if you don't have a chance to check it out again, I always recommend people to try to see the experiences first before really unpacking and listening to it. But if you don't always have an opportunity to, then always welcome to listen to these creators talk about their process. But the larger context of the story is about the white terror that happened in the 1950s in Taiwan. This was a period of martial law in Taiwan that lasted 38 to 43 plus years. From Wikipedia, it says it started on May 19, 1949, and then that martial law was lifted on the 15th of July, 1987, but others cite the repeal of Article 100 of the Criminal Code on the 21st of September, 1992. So, anywhere from 38 to 43 years of martial law in Taiwan, and in that context of this white terror, this story takes place. I'm just going to read the description because I think it helps set a broader context for the story and where it's taking place. So, within the walls of the former Green Island prison, political detainee A Quinn tells the stories of imprisonment and persecution happened in the 1950s in Taiwan. among fellow inmates. Frozen in time, he recounts his own experiences and those of his friend, A. Ching, who never made it out. Experience the time and place, the waiting, and hope for a chance to keep the stories alive. The Man Who Couldn't Leave integrates the stories of numerous political victims of the White Terror and told through the form of an undelivered family letter, an immersive VR experience of hope, fear, and camaraderie. There's a number of different letters that were confiscated because they were being censored. Imagine being a political prisoner trying to write a letter back to your family. It doesn't ever reach your family. You may not ever know that. But these letters are made available after this white terror had been lifted. Singing Chin got a hold of some of these letters and visited this wax museum in Green Prison and started from that point to be able to expand this out into an immersive experience. And again, the mastery of telling the story with 360 video medium, collaborating with Phonique, the VR supervisor also had a chance to do an interview with because, believe it or not, he had three projects that were at Venus Immersive and they were all launching at the same time. All That Remains, The Man Who Couldn't Leave, and Redtail. VR supervisor, Mingyuan Chen, Yeah, just also had a lot of interesting insights into the process of creating this, and I'll have that as a separate interview and digging into his journey of working at Phonique, which the 360 videos that they have at Phonique, some of the best stereoscopic VR I've ever seen, including a lot of the stuff from Felix and Paul. Yeah, just some really amazing post-production processes that they have in order to really make this a piece that stands out in terms of pushing forward the language and grammar of immersive storytelling. So that's what we're covering on today's episode of the Voices of VR podcast. A quick logistical note is that this interview is actually in two sections. The first part is going to be the English version that is going to have singing Chen being translated through the interpreter of Poshan Wu. Then we're going to have my takeaways in English. And then the last section is going to be the Mandarin Chinese version. So if you want to listen to the Chinese version of this conversation, then you can skip forward to it'll roughly be around the halfway point. So this interview with Xing Chen happened on Friday, September 2nd, 2022 at the Venice Immersive in Lido, Italy. So with that, let's go ahead and dive right in.
[00:04:40.898] Poshan Wu: Hi, everyone. My name is Boshan Wu and I'm working with director Xing Chen for this project, The Men Who Couldn't Leave as the distribution coordinator. I'll also be translating for the director. Yeah.
[00:04:57.007] Singing Chen: Hi everyone, my name is Xinyi Chen.
[00:05:05.433] Poshan Wu: I'm the director of The Man Who Couldn't Leave. And I used to direct feature film and documentary. And The Man Who Couldn't Leave is my second VR film. My first VR film is called After Image for Tomorrow. And it's my pleasure to be here in Venice this time.
[00:05:24.190] Kent Bye: Okay, and so yeah, maybe you could give a bit more context as to your background and your journey into making 360 videos.
[00:05:35.418] Poshan Wu: I used to direct and produce feature film and documentary. My documentaries were mostly about the theater, contemporary dance, and contemporary artists. When I directed my first VR film, I realized that VR is about space, just like in theater. But on the other hand, movie, feature film, it's more about time. And that's how I think of VR as a medium.
[00:06:09.982] Kent Bye: In your piece, you have photographs of people right before they're going to be executed in Taiwan. And so maybe you could give a bit more context as to the story that you're telling here and where it began. So,
[00:06:26.661] Poshan Wu: The background of the story happens in 1950 when in Taiwan we had a like white horror during which there were a lot of like a political victim or like prison they were like persecuted. The background of the story was shot in Green Island where there's like a During the White Terror, there were many political... So the period of white horror lasted around like 30 years and many years has passed after the white horror ended. However, like not until lately the stories are being revealed by people and are being discussed The reason for that is also because in Taiwan recently we're talking about transitional justice. That's sort of the reason why people are like talking more and more about the stories that was happening in the past. So in the film, there is a letter that was written to the family of the person who was executed. Those kind of like historical documents related to Waihoro were locked up before, and it was not open to public access until recently. So there are a lot of documents that were locked up for like 70, 80 years until recently. Yeah.
[00:08:18.495] Kent Bye: So it sounds like you got access to new documents, but there was also the scene that you starred in with the wax museum, where you have a lot of figures that seem to be recreating one of these prisons, and you have an actor that comes in. And so maybe you could talk about how that location of that wax museum may have also been an inspiration for what you created in this film.
[00:08:43.453] Poshan Wu: So the jail, the prison in Green Island was later turned into a human rights memorial park and one of director's friend was involved in planning and organizing this human rights memorial park. and it was about like 10 years ago so like she has always known about this place so when she was commissioned to make this project it was like within seconds she thought of the space yeah and because they're like wax statues in the space so the director was thinking yeah maybe there were some moments that those wax statues could represent those people who were really there and also like it could like reflect their stories. So when I went to Green Island to do the scouting for like spaces and the shooting location was a bit like a dormitory and with wax statue like sitting in the space and so the director was sitting there. Although those statues are made of wax and they're fake but like she feel that she could sort of like hear their talking about their stories and stuff like that.
[00:10:04.852] Kent Bye: Yeah, and we're here at the Venice Film Festival where your piece is premiering here. But there's also been over the years as I've gone to the film festivals, there's been a lot of 360 videos that are coming out of Taiwan and also Immersive Works and HTC Vive is located in Taiwan. But maybe you could tell me a bit more about your first VR experience or how you got involved into 360 video filmmaking. And what was the point where you made a transition making 2D films into 360 videos?
[00:10:36.001] Poshan Wu: The reason why I started to make VR film was because Kaohsiung Film Lab invited me to produce the director's first VR film. And after getting the invitation, the director spent a lot of time doing research and watching a lot of VR works. And then she realized that in a lot of works, the body, the characters, seems to be kind of far from the viewer. And so in her director's first film, After Image for Tomorrow, her attempt was trying to make a space and also make the viewer feel that in the VR the characters are very close to them and also make the viewer feel the space is surrounding them just like they're in the real space.
[00:11:34.545] Kent Bye: Yeah, and so as you were producing this piece I watched the behind-the-scenes video and the thing that I was struck by was the staging that you have and the lighting and the way that you were having to the left or right you could see a scene and then something in front of you and it seemed to have a lot of influence from theater and insights from theater so I'm wondering if you could talk a bit about When I was making my first film, After Images for Tomorrow,
[00:12:11.391] Poshan Wu: So like, okay. So actually in the first few film after image for tomorrow, the director was already like using some of the elements like staging, like from theater, for example, there's no like cut editing. Yeah. So like in the space, unlike other VR, maybe you have like one shot and then transition and then another shots. Yeah.
[00:12:37.752] Singing Chen: It's no editing. No editing in VR. No cut editing. Some VR is cut, then change the space. But I don't want this. I want in stage, the light change, then space change.
[00:13:01.251] Poshan Wu: In the first film, the approach was still like the audience was standing in the same position while the scene changes around the viewer. But this time, the director wanted to push a little bit further. Like she was trying to make the scene changes while the audience is also looking around. And back to the question, the director has always felt that VR is just like a theater space. So maybe that's why for her, how she uses it is sort of like similar. So like she said like one of the challenges in both VR works that she has done is how to balance between like the realistic feature quality or elements in the film or like between reality like realistics and also like those abstract elements that were usually used in theater.
[00:14:08.884] Kent Bye: Yeah, and there's scenes of people who are in prison and then they are hung up and executed. And you'd said in the behind the scenes video that you had seen these photos of people that were political prisoners right before they're about to be executed and that you were really struck by how they were smiling. And I'm wondering if you came across some of these photos and how that played into expanding out into the larger context of what was happening leading up to those moments.
[00:14:41.023] Poshan Wu: So when the director was writing script, she read a huge amount of letters that were written by the prisoner to their family. And then she gradually realized that it is sort of like human nature or like innate desire for people to fight for a better future or a better living condition. and that's the moment that she realized that that's the universal language that people would like fighting for like even like in the 1950s up until like 2020s like what's happening like recently like people are still like doing the same thing like fighting for their future and yeah
[00:15:32.041] Kent Bye: Yeah, I'm wondering if you could connect a little bit of looking into the past of what happened and then what's happening now with Taiwan and China.
[00:15:46.055] Poshan Wu: okay so in fact the history of taiwan is very complicated and so is the relationship between taiwan and china and over like this long period of time a lot of people they sacrificed themselves to fight for the better future or what they believe and that's happening in China in Taiwan in Hong Kong and all over the world for example in the ending scenes you can see those statues of people and strike and demonstration and like those statues like some of them they have western facial feature and some of them they have like asian facial feature and that's kind of like representing that it's a universal issue that we all face and also in that scene you can also see umbrella which is like a very direct reference related to what was happening in hong kong and also like the song that was used in the ending like it's called like el pueblo unido jamás será vencido That song is also used in Taiwan when they're doing demonstrations and also around the world. So the director put these universal elements that could be relevant to people in the ending of the film.
[00:17:18.174] Kent Bye: Yeah, okay, so that's the people united will never be defeated is the Spanish translation that's being translated into Taiwan and the same type of revolutionary chance, I guess. So that last scene I think was really striking to me because you start with very similar to the wax statues where you have these people who are still but there's on one side the police and on the other side there's the people who are fighting for truth and justice and freedom and that as the camera moved through it you have the people that are still still but you get this feeling of animation because you get the feeling of movement of what's happening and so it was a really provocative way to end the piece by having these two sides going against each other and by having the camera movement move through it it really gave a really unique feel so I'd love to hear about the process of designing that scene.
[00:18:16.106] Poshan Wu: So in the beginning of that scene, those 3D models, they were peaceful. As the camera was panning, the scene, the stride, the demonstration become intensified. And up until that moment, when like the statue of a police was going to hit the people on the other side and the viewer was like placed in between this so like the viewers they could turn their head either as if they were going to be hit by the police or like they can turn their head around looking at the people and so yeah that's how it was designed.
[00:18:58.729] Kent Bye: Yeah, and throughout the course of the story you have the political prisoners who are about to face execution and they try to send some last messages to their family by hiding notes, writing stuff down, sewing them into the fabric, and then eventually getting back into the hands of the family. And I'm wondering if, as these new documents became available, if you found some of those notes, or if this was something that you created to tell the story, to explain the dynamics of the relationships of the husbands and wives and families, to have that as communication. So, were those notes something that you had found, or is that something that you created for the story?
[00:19:36.471] Singing Chen: This family letter contains
[00:19:40.953] Poshan Wu: So in that letter, it combines text from at least five real letters from the past by those political prisoners. And some of the texts are the same as they were written, and some were actually modified by the director. herself and at the end of that letter there is a poem it was like written by like the deceased political prisoner however that letter like that poem it was not like given in a written form that person like read that poem to a friend in prison and that friend memorized it and he left the jail and then tell his friend's family about this poem.
[00:20:35.202] Kent Bye: In the film there's a scene where when you look to the left and to the right you see these boards with writing on it and the wife of the political prisoner is walking and looking at those and maybe you could explain what those boards were and if they were a part of something that was back then or what was happening in that scene when she was looking at those writings.
[00:20:56.867] Singing Chen: At that time, my family...
[00:21:01.774] Poshan Wu: so like those letters written in prison they were being examined and if they're like sensitive they would not be like sent to the family so that explain why a lot of letters were locked up like 70 80 years yeah so like a lot of like family member of those prisoners they didn't even know their family are going to be executed Yeah, so that's the board that would have the name written who were executed. And so some people would not even go there to check if their family were executed. But there are some family wives, they would go there every day to see if their family are still alive or not.
[00:21:52.266] Singing Chen: Back then, a lot of people were killed, so it's quite hard for the family too.
[00:22:09.720] Poshan Wu: to recognize their husbands or family members. So that's why in the film, the wife, like in the letter, told her husband he has to wear the shirt that she gave him and put his hands in the pocket so she could recognize him after he was killed. So in that scene with the board, it was a little bit more like abstractized or modified in a theatrical way. In reality, there might be just a board with a piece of paper with a few names on it. But in the film, the director sort of like make a scene with a lot of boards to represent that wife has been like looking for a long time to represent that process.
[00:23:18.607] Kent Bye: Yeah, and there is that contrast between what's going from the left and right with those boards and then in the center is this build-up to the prisoners about to be executed and another really striking moment in this piece was to have the prisoners hanging from the ceiling and a number of the prisoners hanging from their hands. And so I'm wondering if you could elaborate on the metaphoric or symbolic representation of those executions. And it was a really striking image to see all the people hanging. And I'm just wondering if you could explain the process of deciding how to depict those executions in that way.
[00:23:59.397] Poshan Wu: so that was like also like a theatrical like more abstract way to address that scene actually like with their hand tied up and hand up it was used as a way to torture those political victims yeah and That scene was also being drawn by a painter who was also a political victim and he was lucky that he didn't get executed so he was able to paint it out and so people could know what was the torture that they went through. Yeah, and so with that scene, you can see a lot of people were being like hung up. So that scene represents that a lot of like elder or like deceased fighters, like political victims, they were being treated that way back then.
[00:24:54.707] Kent Bye: Yeah, and in this piece, you have a lot of this journey of the imprisonment and torture and death and connections to the family. Also, the ending with the larger context of the universal aspects of this conflict. And I'm wondering if you could reflect upon these themes of death and then what hope or joy comes out of that or optimism from what you're trying to convey in this piece and the message that you're trying to tell.
[00:25:25.280] Poshan Wu: Okay, so other from like those like imprisonment and torture things like the message the director wanted to convey was actually like this kind of things like political oppression doesn't just happen in 1950 in Taiwan, it happens all around the world and even now like it's happening and there are always people working and fighting for a better future for a better world and like she wants that like people even in the like virtual world in VR they could feel you know like How do you call that? Like hot-blooded? They could be ignited to fight for what's better for them if needed. It's sort of like encouragement. So the main character in the VR, Akhun, the director sort of kept him in the space of VR. So when the film started, he said, I've been waiting for you for a long time. and he want to finish his story. So that sentence was also a reminder to people do not forget the stories and keep the stories being told to make people in the future know about what was happening.
[00:26:57.449] Kent Bye: And finally, what do you think is the ultimate potential of virtual reality and immersive storytelling and what it might be able to enable?
[00:27:10.259] Singing Chen: So there will be some more.
[00:27:15.461] Poshan Wu: Okay, so like VR for the director is a new medium on its own, just like photo, video, there are like different medias, transitioning. And so it is important to like view VR as its own category. And we have to find its own language. I think in the VR, what's important is the existence of the body. It might sound a little bit strange, but the director, it might sound a bit strange, but she wants people to still remember their body and virtual reality world. Because as we can expect, in the future, people will spend more and more time in the VR. But body is kind of important and we should not forget how our body exists in this medium. Nowadays, all those images, media, we're trying to provoke, stimulate people's sensory. But the director doesn't want to do it in a stimulating way. She wants to use VR as a space to bring out people's emotions and feelings. For example, for her first VR film, After Image for Tomorrow, there are like two versions of this piece. One is just like a standalone 360 version. Another one is a on-site version. with the audience standing in the space. For example, one time the piece was exhibited in a courtyard in an open space. People were viewing and there was wind and breeze and things that sort of created a new corporal experience in VR. and that would be the direction that she would like to experiment and continue trying in the future.
[00:29:35.220] Kent Bye: Yeah, and I'm curious, you were talking about the language of VR and there's just a lot of influences from theater, but I'm speaking English and you're speaking Mandarin Chinese, but the language differences between English and Chinese as a language is quite different from how there's a lot of context for Mandarin Chinese, the things being related to each other. And it feels like VR as a medium may be closer to like a Chinese language than English language. in the sense of creating a relational dynamics within the film. So I'm wondering if you feel like speakers of contextual languages like Chinese have particular insights for how to develop the language of VR based upon the language differences of Mandarin versus English.
[00:30:21.695] Poshan Wu: Is it the language? Is it the real language?
[00:30:23.937] Singing Chen: Because I just said that we need to find a new language. It's actually not that language.
[00:30:31.374] Poshan Wu: Could you repeat your question? Because one director said that language VR is sort of like a format or it's not really like a spoken language, but it's a style or something like that.
[00:30:43.777] Kent Bye: So your question was in different... Just my understanding of the Mandarin Chinese is that when you speak a language, sometimes when you say the same thing, but if it's in relationship to other things, it changes. And so there's more of a context that is taken into consideration. And so VR as a medium seems to preserve some of those contextual dimensions more than the other previous media that may be more linear. There's more of a spatial, relational, contextual dimension to VR as a language. Yeah.
[00:31:18.281] Poshan Wu: 我懂了 他應該是說像中文英文有些是高語境語言的問題 So your question is like, I'm sorry, like if like VR... There is a... I mean, language is something we speak and we swim in.
[00:31:32.173] Kent Bye: It's invisible to us. But if you feel like that there is certain aspects of Chinese culture or Taiwanese culture, like Chinese philosophy or other ways that are trying to see the the larger context, thinking all about the relationships between things, it seems like, for me at least, VR as a medium is mirroring more of the Chinese contextual relational dynamics than it is the sort of more linear aspects of the English language. And so, yeah, just any thoughts or reflections on that?
[00:32:04.850] Poshan Wu: So yeah like the director agree with what you just said because like she also work on like 2D film and she feels like film usually it's more like a closed system because like on the audience they're looking at what the director want them to see and with the editing and shooting and stuff like that However, in VR, it's more like an open system. Of course, it's not like the audience. They just see what they want to see. It was also organized and presented by the director. But there are a lot more elements. And in VR, that could make the viewer relate their experiences with all those different things they see.
[00:32:53.074] Singing Chen: To me, VR is more interesting and creative.
[00:32:58.747] Poshan Wu: So because of what she just mentioned, the director feels VR is more interesting and in a way more creative. For example, when she was writing the script, like sometimes she would like sort of like enclose her like views and to like look around to think what the audience might be seeing, thinking. So she would try to incorporate I think people's perception is easy to fall off. Okay so the director, she feels that like people's sensory are usually taken for granted or like we get used to this for example when she just like try watching VR when things like come into her like she would just like her body would like move to dodge the flying object or something like that but after viewing Many VR films like your body gets lazy or like it used to that and you're stopping active So just as what you mentioned before like she wants to keep like trying to create this like a new corporal experiences in VR
[00:34:40.850] Kent Bye: Awesome. Well, congratulations on having your piece here. I really, really enjoyed it. I thought it was really poetic and beautiful and really strong message. And so thanks again for joining me here on the podcast.
[00:34:52.759] Singing Chen: Thank you very much.
[00:34:54.640] Kent Bye: Thank you. So that was Singing Chen. She's the director of The Man Who Couldn't Leave, which won the Best Immersive Experience prize at Venice, the top prize there, as well as Po Shan Wu, who's the distributor and also served as the interpreter for Singing Chen. So I've a number of takeaways about this interview is that first of all, I mean, for me, the thing of the interpreter potion who getting choked up of just thinking about the emotional impact of these political prisoners who there's these photos that they show in the behind the scenes videos, and I think that actually might be in the experience as well. But It's just beaming smiles of people that are just expressing this joy and happiness and knowing that they may actually be the last images that are taken of them. And so to connect that sense of hope and optimism into the future, just really powerful to reflect upon this totalitarian martial law that was taking place in Taiwan for 38 to 43 years, however you measure it. There's a director's statement that I want to just read as well, because I think it ties into the character of this piece. So the director's statement from Shing-Chen, the background to the man who couldn't leave derives from documents recounting the experiences of political dissidents imprisoned on Green Island during the White Terror of 1950s in Taiwan. It is narrated from the perspective of four symbolic characters, Ae Kwon, who recalls with regret his friend Ae Ching, the man who couldn't leave. There's also Ae Ching's wife, who held the secret of her husband's incarceration for many years, and Ae Ching's daughter, who now inherits that story and is determined to keep it alive. It also lets the audience feel the suffering and hopes of these people and conveys their desires for a better society, hopes that transcend as universal values pursued by identity groups and new generations. In the end, it is only if stories are constantly told and the ideals and sacrifices of earlier generations are remembered, can the dead souls rest in peace and the spirit of an ideal society can be continued from generation to generation. So yeah, there's a lot of like symbolic dream logic that I think is used in this piece that is really evocative imagery. And if you watch the behind the scenes video, which if you haven't already, I highly recommend checking that out just to get a sense of some of the different scenes, if you haven't had a chance to see it, or even if you had a chance to see it, to really appreciate the production quality of all the stuff that they had to do. I think they mentioned in there that it's probably one of the most complicated 360 videos that have ever been made. Probably, you know, right up there is the ISS space explorers experience of sending a camera up in space. But in terms of like production here on terrestrial Earth, this is definitely up there in terms of like one of the most sophisticated and complicated orchestrated experiences that I've seen in 360 video. So yeah, just the way that seeing Chen is holding up her hands over her eyes as she's blocking out all the staging and talking to Mingyuan Chen, who's the VR supervisor, and also talking about their process of trying to block it out. So it's a powerful piece. And just the way that they're combining all these things into this, again, going back into like, okay, is this something that is a literal translation or a symbolic translation, and time and time again, taking that liberty of not trying to be photorealistic or authentic into what literally happened, but to use the medium of VR to create these dreamlike sequences that transport you into this realm where you see all these images of people hanging from the ceilings, and you get the sense of that they're being tortured. And so, it's a metaphor and a symbol that makes it feel like you're walking through a dream. And I think in their behind-the-scenes, they were really going for that dreamlike imagery. And as I reflect more and more about the medium of VR, I keep coming back into this symbolic dream logic of the ways in which it is like a dream, like you're waking up for a dream. And in dreams, you use these symbols. And sometimes the symbols are very connected to what's happening for you as an individual, sometimes there as a collective as a culture, or sometimes they're universal, archetypal symbols. And so it's always this challenge of trying to take this poetic interpretation of this abstraction and trying to reach the type of imagery that is able to translate the feeling that you're going for, Into the thrust of the story and I think that's again and again a successful interpretation of that type of symbolic dream logic into a narrative and it just works so well so powerful and I was recommending people to Not sleep on the 360 videos a medium because a lot of 360 videos in the past They've been off in the corner like you can see the interactive experiences Which is what everybody thinks is going to be the future and frontiers of the medium and then kind of off to the side is a 360 videos I'm really glad that the curators this year, Liz and Michelle, got away of all this bifurcation between the best immersive story versus the best interactive story and just put it into the best story. I agree that this is definitely up there in terms of one of the best stories that was told here, regardless of however the mediums and the technology that was being used. But to reiterate the power and the difference of the 360 video as a medium is that you're able to really get into the emotional expression of the actors. I think that was a big part of what really made this experience land, to see the suffering, see the hope, see the grief, the sorrow, the pain, the death and grief and torture. All these things that were acting and trying to really convey all these things, as well. That's really lost when you are translating into a computer-generated environment. There's still a lot of power of being able to see another human's face and to activate those mirror neurons. I just wanted to mention a couple of things that we didn't talk about here. There's some scenes where they're trying to convey these aspects of surveillance. They have this room where the wall is being opened up and you see this giant face looking in. The use of macro photography to give this sense of the surveillance state that's looking over everybody that's in that room. And also, the ending sequence of having what looks like to be, think of like the aesthetic of a super hot type of low-poly figures, and you have this dividing line between the protesters and the police, and there's a tracking shot that's happening from one end, and you're kind of zooming through this, what is a static statues, but as you move forward, you see this static actions be animated And again, it's not actually moving. They're all statues still, but it just, from the camera movement, it gives you this sense of motion and how the scene is progressing into something that is this polar opposite between the state of police versus the protesters. And to see how this polarization, you're just right in the middle, in the trenches, going right into the midst of this type of totalitarian control or surveillance. And so, as we reflect upon what's happening in Taiwan and with China and the different tensions and all the different aspects of that complicated history and the potential for what's happening with Russia and Ukraine with this invasion that's happening there and there's uncertainty as to whether or not there will be any type of military intervention from China into Taiwan or anything like that, you know, could potentially escalate into something like that, some signs that we've seen. And so there's particular interest of trying to look back into the past totalitarian history of this martial law that happened for 38 to 43 years. looking into the past to be able to tell these stories, but also how those stories are reflecting as to what's happening in the moment. And I think that's another dimension for why The Man Who Couldn't Leave, the story within it is resonating so much more as well. And yeah, highly recommend getting a chance to check it out if you haven't already. And yeah, just a powerful use of the medium and happy to be able to sit down with Singing Chen. So yeah, that's all the takeaways that I have. The last section here is going to be the part of the interview that is in Mandarin Chinese. And yeah, if you enjoy the podcast, please consider supporting me on Patreon at patreon.com. And for all the other listeners, enjoy the Mandarin Chinese version of this conversation.
[00:42:57.647] Poshan Wu: Hi everyone, my name is Boshan Wu and I'm working with director Xing Chen for this project, The Men Who Couldn't Leave as the distribution coordinator I'll also be translating for the director
[00:43:12.027] Singing Chen: Hello everyone, I'm Xinyi, Xinyin Chen. I'm come from Taiwan. This is my second VR video. I used to make plot videos and documentary videos. The last video was After Images for Tomorrows. I'm very happy to bring my new work to Venice Film Festival.
[00:43:37.087] Poshan Wu: Can you tell us about your background and your creative journey as a whole?
[00:43:48.323] Singing Chen: I used to make film and documentary films. Most of my documentaries are about theater and dance. They are also about contemporary artists. So when I was making VR, when I made my first film, I thought VR was an art about space. It's very different from movies. Movies are about time. Later, I used a lot of theatrical methods to deal with the VR space.
[00:44:19.374] Poshan Wu: In your piece, you have photographs of people... In the film, there are some photos of people who are about to be executed. Director, could you please explain the background of this story?
[00:44:36.683] Singing Chen: This story is set in the 1950s, when there was the so-called White Terror in Taiwan. During the White Terror, some of the political prisoners were locked up on Green Island. They were already locked up on Green Island, but they were still charged with rebellion. About a dozen political prisoners were executed. That's the story. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed. During the period of the White Terror, many political victims died and were executed In the story, there is a very important thing, which is the will, the will hidden in the clothes. Like this will, it's only recently that there is a slowly... In the past, these were files that couldn't be read, files that were locked up, and these files were only recently dug out. So these wills have been passed down for so long, maybe almost 70 or 80 years,
[00:46:02.636] Kent Bye: So it sounds like you got access to new documents, but there is also the scene that you started...
[00:46:10.437] Poshan Wu: So those newly published documents are the inspiration for my work. There is also a wax museum in my work. Can the director talk about the space of the wax museum as an inspiration or inspiration for your work?
[00:46:33.528] Singing Chen: When the prison on Green Island was about to become a human rights park many years ago, I had a friend who was in charge of planning the Green Island Human Rights Park. After they built the LASAM, I felt that the space was very... Actually, it is very good for VR, because it is a virtual image. It has physicality, but it is also virtual. When I was working on the second VR, I immediately thought of this scene. I thought that if those people really move at a certain moment, the audience can completely travel through history and space at the scene of the 1950s. So I really wanted to make use of this space.
[00:47:28.010] Poshan Wu: You said that senior is your friend and... Actually, the person who planned the Green Island Park is my friend.
[00:47:34.553] Singing Chen: So you asked him at that time, or this friend and this space... No, what are you asking?
[00:47:37.835] Poshan Wu: I'm just guessing the relationship of the translator. I mean, that space has already... Oh, because he planned it, so you thought of this space directly.
[00:47:54.805] Singing Chen: Okay, so... Yeah, and we're here at the Venice Film Festival where your piece is premiering here, but there's also been
[00:48:50.098] Poshan Wu: In the past, there were a lot of 360 videos in Venice. Now in Taiwan, we even have HTC Vive. They also make their own 360 content. There are also a lot of creators in Taiwan making 360 content. Can you share with us how you started from a flat screen movie to a VR work?
[00:49:19.253] Singing Chen: When I made my last film, After Images for Tomorrow, it was an invitation from Kaohsiung Film Lab. They invited me to make a film about dancing. During that time, I had watched a lot of VR films, but not many. After that, I started to collect a lot of VR films. I think there is a phenomenon that it is very similar to the human body. 所以在我上一支影片的時候,我想要做的事情是如何把人的身體給抓進去那個VR的空間裡 所以在我上一支影片的時候,我想要做的事情是如何把人的身體給抓進去那個VR的空間裡 因為我看了很多VR,它其實有時候你知道那個VR在講什麼 可是其實你跟它是很有距離的,你沒有真的身歷其境在裡面 所以我上一支片的重點就是如何讓觀眾可以被抓進去那個空間 你是真的不管是你的裡面或者說你的身體都有感受這樣 那個身體感的部分
[00:50:23.228] Kent Bye: Yeah, and so as you were producing this piece I watched the behind-the-scenes video and the thing that I was struck by was the staging that you have and the lighting and the way that you were having to the left or right you could see a scene and then something in front of you and it seemed to have a lot of influence from theater and insights from theater so I'm wondering if you could talk a bit about the staging that you have in terms of how you're arranging things and the influences that you're bringing in, bringing theory to practices and lighting into the process of making 360 video.
[00:50:55.594] Poshan Wu: He said that when he watched the VR, he felt that the lighting on the stage, including looking to the left, to the right, to the left, to the right, to the right, to the left, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right, to the right,
[00:51:23.840] Singing Chen: When I was working on the first film, After Images for Tomorrow, I had already used some of the theatrical methods. I didn't really have a cut in the film. Actually, I think VR is very similar to theater, because theater is the same space, but with lighting and various background changes, the properties of the space are changed. So in my VR, I also used many similar methods. 你要先翻嗎?我再往下講。 沒有cut就是一鏡到底嗎? 就是... 不是,它就是沒有edit,它沒有cut,它不算是有真正的剪接 哦,那也不算一鏡到底 不是一鏡到底 You interrupted me, so I kind of forgot what I was going to say. What was the question? I was going to talk about the theater. So this time, you're more ambitious in trying to break through. Because the last time, the audience was still there. and the 360-degree space is transformed. But what I want to do this time is to make more use of the displacement of space. When the audience is in displacement and change, the space is slowly changing. This is the ambition this time. 在上一支影片的時候其實我們就很掙扎說 到底是要實景拍攝 還是說要像劇場一樣稍微有點抽象虛擬這樣子 就是比較抽象的美學的表現的形式這樣 這在上一支就已經有做很多的掙扎 然後延續到這一支其實也有 就是你如何在真實跟抽象的劇場中間取得一個平衡 這個也是這兩次都有在做的
[00:53:14.354] Kent Bye: Yeah, and there's scenes of people who are in prison,
[00:53:21.946] Poshan Wu: In the creator's video, you said that you saw some photos. They were smiling before they were shot. How did the director expand the story from these photos to this work?
[00:53:45.930] Singing Chen: When I was writing the script, I read a lot of letters from the victims. I read a lot of letters from the victims to their families, or to their relatives, or all kinds of texts. We can slowly understand that everyone wants to have a better life. They want to have a better life for themselves, their friends, and their families. They are just fighting for their ideals. 所以後來我抓到這個點之後 就覺得這個其實是全世界的共通的 所以後來我抓到這個點之後 就覺得這個其實是全世界的共通的 那當這個東西是全世界共通的時候 那當這個東西是全世界共通的時候 其實從現在的2020 穿越到1950 再回到現在2020 全世界都還有很多在發生這樣的事情 那那些笑容對我來講就是 我覺得那些前輩之所以會笑 是因為他們知道到了現在都還會有人 就是繼續好像有了這些理想 繼續為了這些事情在奮鬥這樣
[00:54:51.574] Kent Bye: Yeah, I'm wondering if you could connect a little bit of...
[00:55:09.165] Singing Chen: In fact, Taiwan's history is complex. Taiwan's history is complex. The relationship between Taiwan and China has always been very complex. In fact, since such a long time ago, in different eras, there have been many different victims because of different issues. Although this story is based in Taiwan, I think it's happening all over the world, not just in China, Taiwan, and Hong Kong. So in the final 3D model, there are foreigners and Asians, and umbrellas. Of course, it's easy to think of umbrellas in the umbrella movement. The other one is the song we use for the ending, which is the song, People of Unity Will Never Be Defeated. The people united will never be defeated is the Spanish translation that's being translated into Taiwan and the same type of revolutionary chants, I guess.
[00:56:28.460] Poshan Wu: So that last scene, I think, was really... Okay, so he said that the last scene was very shocking to him, or very surprising to him. The models of those two rows were a bit like wax figures. They were lined up into two rows, and as the camera was panning, It's like they're fighting each other. He thinks it's pretty provocative.
[00:57:05.233] Singing Chen: In the beginning of the film, there was peace between the two sides. But as the camera moved back and forth, the fight became more and more intense. There was a conflict between the police and the people, a violent conflict. I stopped for a moment. I don't know if the audience noticed, but at that moment, the police had to fight. Okay.
[00:57:49.099] Kent Bye: Yeah, and throughout the course of the story, you have the political prisoners who are about to face execution.
[00:58:19.008] Singing Chen: There are about four or five, maybe five or six articles in this book. It's a book written by a lot of seniors. Some of the words are exactly the same, and some of them were revised by me. So it's a combination of a lot of seniors' books. But the last poem, he wrote a poem for his wife and children. The last poem was really written by a senior like this. 那首詩不是他寫下來給他的小孩 真實的狀況是他要死掉了 他要去槍決了 然後他把這首詩告訴他一個朋友 獄友 然後這個獄友帶出去 用voice的方式帶出去給他的家人
[00:59:08.869] Poshan Wu: In the film, there's a scene where when you look to the left and to the right, you see these boards with... There's a scene where the wife is looking at something like a notice board. Can you explain what the wife is doing in that scene?
[00:59:47.927] Singing Chen: At that time, except for the family members who sent the letter, but sometimes they couldn't send it to the family members, so sometimes they couldn't get the letter. So they couldn't know, and the gunman wouldn't notify the family first. So they had to go to the notice board every day to see if there was their family name on it. And if there is, then you know that you've been forced to do so. So at that time, a lot of families would directly... Some people wouldn't dare to look, but some families would directly look at the name on the notice board to see if there was my family's name on it. At the time, many people were executed, so it was difficult to identify the body. So the letter that the wife wrote to her husband said, of course, it wasn't necessarily the right way to identify the body, but she wrote to him, asking him to put his hand in his pocket and wear that piece of clothing, so that he could identify the body.
[01:00:47.363] Poshan Wu: Oh, can you explain it again?
[01:00:49.964] Singing Chen: The wife wrote a letter to her husband, and in the letter, she wrote that she wanted to wear the shirt I made for her, and put her hands in her pocket, so that she could recognize the body as you. Otherwise, she might not be able to recognize it. Yeah, and there is that contrast between what's going from the left and right with those boards and then in the center is...
[01:01:31.943] Poshan Wu: In addition to the scene of the notice board, there was another shocking scene, the scene where the hand fell off. How did the director choose to make this scene as the scene where they were about to be shot?
[01:01:49.636] Singing Chen: The person who was hanged was a political prisoner. He was not executed. He was a painter. He painted a lot of things in prison. He was hanged in various ways. 所以那樣被吊起來被刑求的方式是 前輩們畫出來我們才知道有這樣子的方式這樣 所以那樣被吊起來被刑求的方式是 前輩們畫出來我們才知道有這樣子的方式這樣 然後那當然那個被刑求我用了 那個也是一個象徵的手法 就是很多本來是一個 然後三個 然後越來越多個 那個是有點象徵很多前輩被這樣子對待 它也是一個比較抽象的處理的方式這樣
[01:02:32.278] Kent Bye: Yeah, and in this piece you have a lot of this journey of the imprisonment and torture and death and connections to the family.
[01:02:41.320] Poshan Wu: So in this film, there are a lot of scenes where people are being tortured and so on. And in addition to these re-enactments of painful memories, the director also wanted to convey other messages.
[01:02:59.643] Singing Chen: I think what I just mentioned is that these things didn't just happen in the 1950s or in the past. Even now, many places in the world are still experiencing the same things. People are suffering because they want to pursue a better world or a better life. So in the end, the singing scene and the 3D modeling scene, I hope that everyone, even in this virtual world, can feel that your blood is boiling, and we can do something. No matter what country you are in, anyone can fight for a better world or life. Because of the role of Uncle Kun, I kept him in the VR for a long time. He seems to be locked in the VR, locked in the scene or history, waiting for someone. When the audience enters, he says, oh, I've been waiting for you for a long time. I want to continue telling this story. There is also such an intention that we do not forget the stories of these predecessors in the past.
[01:04:09.167] Kent Bye: And finally, what do you think is the ultimate potential of virtual reality and immersive storytelling, and what it might be able to enable?
[01:04:18.935] Poshan Wu: What do you think is the ultimate potential of VR in the future? What potential could it bring?
[01:04:34.905] Singing Chen: I think VR is a new form of media. It's like moving from a photo to a dynamic image. It's a new form of media. Of course, it can be used in many ways. VR is not a movie or a dynamic video, but itself is a new category. We have to find a new language for this new category. Do you want to go first? It's okay, I can do it myself. It's okay, you go first. Okay. I think in VR, the most important thing is the body. In the future, we will only get more and more used to being in that world. So how do we make the people in the world not forget that the body is important? No matter what, whether the body exists or not, I don't want everyone to get used to forgetting the body in that world. This is a bit strange, but this is how I feel. Of course, in the VR world, it will only stimulate our senses more and more. Just like in movies, it stimulates our senses more and more. But I want to go the opposite way. In the way that I don't stimulate your senses, is it possible to bring out the feelings that you have forgotten? Like my first video, After Images for Tomorrow, it has a pure 360 video version. It also has a version with dancers on site. In many situations, with VR, we are in a It's like looking at the Central Pavilion. When you look at it, the wind blows. At that moment, you will have a new sense of body. This is what I want to try or look for in the future.
[01:06:37.236] Kent Bye: Yeah, and I'm curious, you were talking about the language of VR and there's just a lot of influences from theater, but I'm speaking English and you're speaking Mandarin Chinese, but the language differences between English and Chinese as a language is quite different from how there's a lot of context for Mandarin Chinese, the things being related to each other. And it feels like VR as a medium may be closer to like a Chinese language than English language. in the sense of creating relational dynamics within the film. So I'm wondering if you feel like speakers of contextual languages like Chinese have particular insights for how to develop the language of VR based upon the language differences of Mandarin versus English.
[01:07:25.443] Poshan Wu: I think I understand his question. But it depends on the director's understanding. Just now we talked about the language of VR. For example, he speaks English and we speak Chinese. Then, for example, under different languages and contexts, will there be different languages in the dynamic of the relationship?
[01:07:49.729] Singing Chen: Is it the real language? Because I just said that to find a new language is actually not that language.
[01:07:56.225] Poshan Wu: Could you repeat your question? Because one director said the language VR is sort of like a foreign mat or it's not really like a spoken language but it's a style or something like that.
[01:08:08.608] Kent Bye: So your question was in different... Just my understanding of the Mandarin Chinese is that when you speak a language sometimes when you say the same thing but if it's in relationship to other things it changes and so there's more of a context that is taken into consideration. And so VR as a medium seems to preserve some of those contextual dimensions more than the other previous media that may be more linear. There's more of a spatial, relational, contextual dimension to VR as a language. Yeah.
[01:08:41.827] Poshan Wu: I get it. It should be like Chinese, English, some are high-pitched languages, some are low-pitched languages. It's like VR, compared to other media, it's probably... What does it mean? Oh, there are some, for example, Chinese is a high-pitched language. Sometimes when we talk about something, you, you, you, you, you, it actually has a lot of connotations. Then you have to rely on a lot of So your question is like, I'm sorry like if like VR
[01:09:31.450] Kent Bye: Language is something we speak and we swim in and it's invisible to us, but if you feel like that there are certain aspects of Chinese culture or Taiwanese culture, like Chinese philosophy or other ways that are trying to see the the larger context, thinking a lot about the relationships between things, it seems like, for me at least, VR as a medium is mirroring more of the Chinese contextual relational dynamics than it is the sort of more linear aspects of the English language. And so, yeah, just any thoughts or reflections on that?
[01:10:01.493] Poshan Wu: Oh, I understand. So Chinese is more advanced, right? And he thinks that VR is a medium that can provide a lot of clues and information. So, director, what are your thoughts on his observation?
[01:10:24.338] Singing Chen: In this aspect, I agree with what he said. Because I also make 2D movies, and 2D movies have cutscenes. It's a very closed system. The movie itself is a very closed system. The director tells you what to watch or how the plot goes. It's actually a closed system. But VR may be an open system. But this open system does not completely allow the audience to watch whatever they want. But we can provide a lot of things that they can think about. He grasped a way that he could understand, a way of inner narrative. To me, VR is more interesting and creative. I'm not just telling a story. When I'm writing a script, I often imagine what the audience might feel. I have to bring in the audience's sense system. I find this very interesting. I'm also imagining different people. what different bodies can feel. This is actually very complicated. But for me, this is what I think about when I'm writing the script.
[01:11:47.012] Kent Bye: Awesome. Is there any final messages that you'd like to share with the broader immersive community?
[01:11:54.736] Poshan Wu: Do you have any final messages for the VR community, the VR audience?
[01:12:05.098] Singing Chen: I think the perception of a person is easy to fall off. What I mean by easy to fall off is when I watch VR for the first time, there are two people talking next to me, one on my left and one on my right. When this person is talking, I want to step back. When this person is passing by, I also want to step back. I have a sense of body. But after I watch more and more VR, when something passes by me, I don't feel anything. I just got used to that feeling So I think people's body and senses are easy to get used to or fall off Yes, so I really hope I can be in this VR world What I was just talking about is how to get your fallen body back I am very interested in this matter So for example, I hope the audience can stand in my film Because when you stand, your body has a kind of dynamic Awesome. Well, congratulations on having your piece here. I really, really enjoyed it. I thought it was really poetic and beautiful and really strong message. And so thanks again for joining me here on the podcast.
[01:13:17.083] Poshan Wu: Thank you very much. Thank you.
[01:13:33.835] Kent Bye: So again, that was singing Chen, the director of the man who couldn't leave, which won the best immersive prize at finis 2022, as well as the interpreter, which is push and woo, who is a distributor who is distributing the man who couldn't leave. So I have my takeaways. earlier in the podcast back for the first part of the interview if you missed that but i just wanted to thank you for listening to the voices of vr podcast and if you enjoy the podcast then please do spread the word tell your friends and consider becoming a member of the patreon this is a listener supported podcast and so i do rely upon donations from people like yourself in order to continue to bring this coverage so you could become a member and donate today at patreon.com slash voices of vr thanks listening
