Play episode

#1452: Immersive Documentary about Chinese Censorship “All I Know About Teacher Li” Tops Venice Immersive Audience Favorites Survey

September 16, 2024by kentbye

I interviewed All I Know About Teacher Li director Zhuzmo at Venice Immersive 2024. Due to the sensitivities of the topic of Chinese censorship, then the director Zhuzmo has requested that he remain anonymous. I did an interview face to face with them in Venice, got a transcript, and then fed the text back into a text-to-speech AI at Eleven Labs in order to mask his voice. See more context in the rough transcript below.

All I Know About Teacher Li also happened to top the unofficial Venice Immersive 2024 Survey results that I did in collaboration with XR Must for both the Top 5 as well as for the Audience Favorites. I found last year’s survey results to be really useful in organizing the order of discussing pieces in our critic’s roundtable, but also helping to understand the audience’s reaction to the selection. Last year there were 50 respondents, and this year there were 82 respondents who on average saw 21.9 experiences each.

Here’s a list of the Most Seen experiences from Venice Immersive 2024, which is sort of a proxy for popularity and buzz, but also throughput.

The next result is the audience favorites (no limit), which is calculated by how many people listed it as a favorite relative to how many saw it. This tends to have the most robust overview of the entire selection, and is also an interesting calibration process to see what you personally liked vs what might have been favored by the zeitgeist. 

Finally the top 5 ranking usually is a proxy for seeing which projects might have been in the running for a jury prize, but there was a lot more variance this year, which was reflected in the range of projects that people were predicting to be a prize winner. The top prize from the jury was awarded to Ito Meikyu, which was ranked 10th by the audience. The second place from the jury went to Oto’s Planet, which was ranked 5th. And then the third place from the jury went to Impulse: Playing with Reality, which was ranked 2nd. I find it increasingly difficult to predict what will resonate with the jury as even the audience favorites can vary widely from my own personal favorites.

Again what resonates with you may not resonate with others, and there are so many other qualities and nuances of each project that can’t be reduced down to numbers. Nonetheless, I found last year’s survey results to be quite an interesting cross section and sometimes confirms an intuition and many other times can be surprising. Either way, it’s additional data to help make sense of the selection and the industry.

Here’s my top 10 that I published after finishing watching all of the pieces on Wednesday, August 28th. I about watched 1/3 of the pieces ahead of the festival, 1/3 on the press preview, and the remaining 1/3 on the press & industry day and the first day bookings are available.

Also, the essence of each project can’t be reduced down to numbers, which is a big motivation that I have to record over 30 hours of coverage for Venice Immersive 2024 over the course of 5 days in order to unpack more of the design process, experiential design tradeoffs, and some of my own embodied experiences and impressions.

Here’s a trailer for All I Know About Teacher Li:

This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

Podcast: Play in new window | Download

Share this
FacebookXRedditEmail

Rough Transcript

[00:00:05.458] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So this is the last episode of my series of looking at different immersive stories from Venice Immersive 2021. I might actually end up doing a little bit more remote interviews with a couple of projects that I missed. I only interviewed one out of the three winners of the Venice Immersive this year. And so Otto's Planet as well as Ito and Maiku are two pieces that I didn't have a chance to talk to folks at this year's program. And I might end up doing like a whole kind of wrap up with critics roundtable that I've done in the past, but I at least wanted to get my first draft of all these different interviews out there just because, you know, I'm going to be traveling and going on to some other things as well. And I wanted to clear out my queue and open up for some of these other trips that are coming up as well. So this last episode is kind of unique just because it's a piece by a person who is anonymous. Jusmo created this piece called All I Know About Teacher Li. Teacher Li is this Chinese student who lives in Italy. So he's outside of China, but posted about bad news in the context of China. But because he didn't live in China, then he could continue to post these different series of bad news and which is kind of increment up his username. So Teacher Li ended up catalyzing a whole protest movement that went viral and ended up potentially having some impact on the Chinese government to change their really strict COVID regulations and policies with this kind of white paper protest where people were just kind of standing up and holding up these blank pieces of white paper that didn't say anything. And so it kind of slipped through the censors and people had enough context that it was a protest around the COVID lockdowns. And then it kind of like spread like wildfire. So Teacher Li is somebody who has been tracked by the Chinese government, and there's specific considerations for people who are talking about these different topics to not have their identities revealed. So the way that I did this interview is that I had a face-to-face conversation with Jusmo, and then I took that conversation, produced a transcript, then cleaned up the transcript, and then fed those text transcripts back into 11 Labs with an AI voice to voice it to be able to mask his identity and identity. From there, then I kind of stitched it all together into this kind of conversation with the AI version of Juice Mo. And All About Teacher Lee was one of my favorite experiences and actually ended up being like at the top rank in terms of the number of people that saw it and then voted it for to be at the top five, but also at the top of the favorites list. Last year, there was a survey that happened by XR Must and XR Crowd, Andre Lunev. And they asked people like what we were able to see, what were your favorite experiences? And then if you were to rank the top five, what would you rank them? And then it was from those answers, they're able to get some survey results from the audience just to get a little bit more of a calibration for what folks are thinking about this year's program. And so this was the last interview that I did at Venice Immersive. And I just decided to kind of air everything chronologically in the order in which I recorded everything. And then All I Know About Teacher Li just happened to also be at the top of the list of the favorites and also the top five in terms of the response for people. It was something that is using these kind of like interactions that are very simple, but is using the same interactions over time to really powerfully tell the story around Chinese censorship. So yeah, that's what we're giving on today's episode of the Voices of VR podcast. So this interview with Jusmo happened on Monday, September 2nd, 2024. So with that, let's go ahead and...

[00:03:43.661] Zhuzmo: Dive right in. I'm Jusmo. I'm a game designer, programmer, and 3D artist. I mainly, like, mostly make narrative games, but in all kinds of formats, like flat screens, VR, AR, or, like, installations, actually mostly depending on the needs of the project. So I don't really have, like, a journey into VR, per se, because I think me personally, and also, like, the whole, like, indie game dev and, like, digital art communities have been using, like, game engines for a long time. And those skills, they're pretty like translatable between just like flat screen and immersive media. So me personally, and I think a lot of friends around me in the community, started experimenting with VR. I think just around 2016 or 17 when the commercial Oculus Rift came out. But the whole thing is that even though the skill is really like translatable, it's all in-game engine and stuff. But the design thinking is really different. So it actually takes time to just test different projects to see what works well in VR and what just doesn't work. And from there, like every time I have a project, either a commercial project or just an indie project, I'm trying to just test what I can do in VR with that subject matter. And if it works, it's going to be a VR project. Like AR VR project. If it doesn't work, maybe I will try to make an installation of flat screen games. So there's not really a journey into VR, the never coming out. It's mostly just depending on the needs of the project.

[00:05:11.344] Kent Bye: Well, when did XR virtual reality first come onto your screen then?

[00:05:15.466] Zhuzmo: I have to say that was the first time.

[00:05:17.427] Kent Bye: When was? Just kind of orient me in space and time.

[00:05:21.089] Zhuzmo: Okay, okay. That was like 2016, when Dear Angelica came out. That's 2016, right? Something like that. Yeah, I saw Dear Angelica at a friend's place because he owned a DK2. And I'm not even sure it was a full version. It was after the Sundance release. It just blew my mind because I'm a game nerd, also a film nerd. Just in general, I feel just like Dear Angelica or just VR storytelling in general just combine these two things really well. And I think that just happens a lot when you're watching a lot of movies or playing a lot of games. You kind of started to see the similarities and see the patterns. And immersive storytelling is something that's really new. You don't really see patterns, even after these 10 years. So yeah, that's just something that really keeps me excited.

[00:06:05.211] Kent Bye: Okay, so it's probably around 2016, 2017, sometime around there. And then the story that you're showing here at Venice Immersive 2024 is sort of set in the pandemic. But before we start to dive into that, was there anything that happened between like 2016 to 2020 that you continued to get into or explore the potentials of interaction design or game design or immersive storytelling within the context of VR?

[00:06:30.842] Zhuzmo: Again, I go to a lot of game jams, and I mostly just work within the indie game community. And then it just really depends on the project opportunity. Sometimes there's a project coming up, maybe someone has a commercial project. I'm kind of like, I'm a programmer, I'm a designer, I'm also a 3D artist, so I can do everything. Not all the best, but I kind of fit in every role. So whenever there's an opportunity, friends reach out to me, hey, do you want to work on this project? If it's a VR project, then I'm just going to work on the VR part of the project. Maybe sometimes it's programming, sometimes it's 3D modeling, sometimes it's more on the design side. Then, yeah, I've been working on projects also shown in previous film festivals. either on AR and VR, they have both been on the narrative heavy side and also on the very interactive gameplay heavy side. So it's kind of like the full spectrum, because that's what happens when we work on indie games. The production cycle is really short. You work on it. You just finish one thing, then you just work on another thing. So yeah, that's... I have just all kinds of, like, immersive experiences just making indie games.

[00:07:35.751] Kent Bye: And so the project that you have here at Venice Immersive 2024, were you the only developer and basically doing everything from the whole thing?

[00:07:46.124] Zhuzmo: Yeah, I'm the only designer developer. Like, because there's so many titles. Again, I can kind of fit into any role. So I just call myself the main creator. Then I have another friend who helps me with the music, because that's the thing I absolutely cannot do. I have zero taste in music, and I just cannot play any instruments. So it's like two of us making these projects.

[00:08:07.843] Kent Bye: Okay. So you and I are sitting here in Venice and part of the Venice Immersive Festival. And when I did an interview with Liz and Michelle, Michelle actually said, you know, there's one of the makers of All I Know About Teacher Lee who has to remain anonymous. And so... And even publishing that information, I was like, oh, I hope we're not like putting an extra target on this filmmaker by even saying that, because this is a serious situation where he's making a documentary that's critical of how China is censoring this bad news. And that's pretty verboten for folks who have Chinese heritage. So we're going to be doing something unique and different here in this interview where I'm going to be taking your answers that I got and then translating them into AI. But just to help give people a little bit more context for what is the sensitivity around this as a topic and a little bit more of the larger context for the fact that putting this out there is putting your neck out there on the line.

[00:09:09.312] Zhuzmo: Hey, I'm Jusmo. I'm going to be turning into AI. Nice to meet you all. So my piece is called All I Know About Teacher Li. It's about an online Chinese art student activist who lives in Italy with the username Teacher Li, who started one of the biggest protests in China in the last 30 years. So he has more than 1.6 million followers on Twitter. And every single follower of his has been identified and trying to be connected by the Chinese authorities. And it's still an ongoing process where if that follower lives in China, they're going to get a phone call by the Chinese police and personally asking them to unfollow Teacher Li. If they didn't cooperate, they're going to be taken into the police station. And this is still an ongoing process. And that's why I'm making this piece anonymously. I don't want to put anyone who is involved in this piece in danger, anyone who have friends and family in China in danger. So I feel like staying anonymous is really important for this piece.

[00:10:06.058] Kent Bye: Okay, okay, well that helps set some context. And so we'll be conversing with an AI version of Jusmo. Well, maybe you could give a bit more context for how this project, All I Know About Teacher Li, how it came about.

[00:10:19.744] Zhuzmo: The context is gonna be long. I'm gonna do a simplified version because there's multiple like geopolitical layers. There's like the pandemic, like the whole world was crazy back then. But in short, I think probably most people know that China has very oppressive internet censorship. And one thing that's absolutely the red line is bad news within China. I mean, there are so many reasons, but one could argue that bad news is the result of bad government policies. So posting bad news is kind of like you are criticizing the government, which is just a red line. So posting bad news is kind of like criticizing the Chinese government, which is something you don't really want to do within the Chinese internet. And Teacher Li is kind of like a Chinese student, but he studies and lives in Italy. He started just to only post bad news on his social media feed within China, which is a Chinese equivalent of Twitter called Weibo. It's kind of like a self-protesting or just resistance against the censorship. And obviously, if you post things that the government doesn't want you to post in China, your posts are going to get deleted. So he just keep posting new things, and then his account gets deleted. So he just keep creating new accounts. And throughout the process, he attracts followers because he lives in Italy. He doesn't have to worry about the step that follows after getting your account deleted because it could get really worse in real life. And he doesn't have to worry about that because he lives in Italy, outside China. So he just keeps posting things and getting his things deleted. Others saw that saying, hey, he has a way. Teacher Lee has a way to get around the last step, which is the really scary punishment in real life. So people started to DM him saying, I have this bad news happen to me or happen to my friends or families around me, but I don't want to post it myself. I don't want to get trouble in real life.

[00:12:05.239] Kent Bye: Yeah, just one quick question. So he's got a username. And at least what you showed in the immersive story, there's like a number where he's at. And so because he's getting banned, does he just increment the next number? And people, if they can't find the one number, they just start following him again?

[00:12:22.105] Zhuzmo: Yeah, that's exactly how that happens. So every time his account gets banned, he creates a new account. Then he increments the last digits of the number. So it's Li1, Li2, Li3. And towards the end, he has Li50 or something. That's how his followers could always find his new account. So they don't actually follow him. Well, it doesn't matter anymore because his account is getting banned daily. People get up on Weibo in the morning. They just type his name, trying to find what is his latest account. That's how people find him. And when I said before he couldn't, like, in the piece, there's a part where we showed that he couldn't really create a new account on Weibo anymore. That's not because no one could create a new account. It's because the first initial, Lee, the name is banned. So you couldn't really create any account with the username Lee in it. So his followers couldn't find him. That's why he had to move to Twitter. That's why I said there's so many contexts. It all happened in 2022. And in China, there happened to be a very extreme and unsustainable lockdown, just very extreme COVID measures around the country. And people were suffering, and there were a lot of hardships. So people started just to send Teacher Li bad news happened to them during the lockdown. And the two things coming together collide into the final moment, where one person was kind of showing Teacher Li his protest or their protest. We don't know their gender yet by holding up a white piece of paper. It just says nothing on it. And that kind of a clever way because that's the thing. Even though you could post things on Twitter, like most people in China wouldn't know because they don't really have access to Twitter. You have to use a VPN and only a small group of people use VPN to access Twitter. So like whatever Teacher Li did on Twitter didn't really have any effect back in China. That all changed with like one blank piece of paper. Because that thing can go back to the Chinese social media without any problem. Because it's a blank piece of paper. It could go past the censorship.

[00:14:19.096] Kent Bye: So we'll get into how you're illustrating all of this in your immersive experience and kind of the experiential design element of it all. But just a quick clarifying point. So Teacher Li is outside of the boundaries of China. And then he's able to receive DMs from people who are following his accounts and He's able to receive information, and so he's becoming a conduit for this underground information that's being censored. So then someone stands up with a white sign that normally, if the sign said something, it'd be a protest sign, but it'd be very easy for the AI to understand what the sign said and then automatically ban it. But it was essentially just someone standing there with a sign. It was the act of a protest, but there was no content to the protest. So it sort of slipped through the censors so that they could get it through. So he was posting on Twitter. So then were people with VPNs copying it and then posting it on their own social media? Or how were people seeing it? Was it only through VPNs and they were looking at it on Twitter, which is now X, but at the time it was still Twitter. So how were people receiving it if it was on Twitter? How did it get back into the Chinese ecosystem?

[00:15:25.353] Zhuzmo: That's actually the part that no one really knew for sure. Because it all happened within one night. Again, even though it's just a blank paper, it could bypass the automatic AI censorship. AI sensor. It doesn't bypass a human censorship. So by the next morning, people actually started to review all the posts. The posts with the white paper protests started getting deleted within Inside China. So, like, nobody knows for sure, like, what exactly happened. But what happened is that Teacher Li posted one person holding up a white paper in the form of a protest. Next day, he received thousands, maybe hundreds of thousands. Same thing. People protesting with a white paper around China. Then the day after that, it's going to be tens of thousands, hundreds of thousands around the world in Chinese-speaking communities. So it's just kind of like a, people call it white paper protest. It's kind of a thing that breaks out in two days from one person holding up a white paper to a global protest thing.

[00:16:21.229] Kent Bye: And there's enough of the intensity of the lockdowns that people kind of intuitively know if there's someone that is standing up with a piece of paper that he's likely protesting against these lockdowns, which even though there's these automated systems, it sounds like this was a bit of a hack that got through the automated filter. It got out there, spread and went viral. And then was enough that people how did people know that it was from Teacher Lee?

[00:16:44.562] Zhuzmo: That's the thing because, again, people had the habit to send bad things that happened to them to Teacher Lee. So it's kind of like a habit thing. So whenever there was a protest, people just took a picture and sent it to Teacher Lee. So they don't post the protest themselves. They ask Teacher Lee to post it. So that's why I felt like it's actually two separate things happening. There's like people holding white paper in China to protest against the extreme lockdown and COVID measures. They're just Teacher Li living outside China, just receiving people's bad news, helping them repost bad news. These two things kind of just collide at the same point. And then it became like a global phenomenon, like at least within the Chinese-speaking communities.

[00:17:25.297] Kent Bye: Okay. yeah so okay so the the thrust of the story is that like there's chinese censorship and there's these really oppressive lockdowns people are protesting and sending them to lead because they know teacher lee's spreading the bad news and that some of it gets through the filter and then there's a floodgate of other images teacher lee starts posting those and then it basically goes viral and it starts an entire movement of people that Throughout the entire country, it seems like the genie's out of the bottle and they can't stop it, even if it's a bunch of police. So there was enough of the public protests that actually caused the change. Even though China's not a democracy, they were still able to find a way to have this mass public movement to bring about some change.

[00:18:06.676] Zhuzmo: Yes, I want to clarify one thing, because again, it all happened within two days. So nobody knows for sure if Teacher Lee actually started the movement. And like he had done many interviews, he wanted to distance himself from starting the movement. He sees himself more as like an information hub, or like a newsroom to spread the information about the movement. But somehow like he just became the symbol of the whole movement. because all the information go through him then go to other places and another thing that's really interesting is that it really got a lot of people excited because like we all know like the censorship in china is really everywhere and that's how this started a mass gathering of mass protests like things happening because they would normally be able to just try to intercept the information You know, like, where the protest is being planned to be located and to stop it before it actually started. That's why people got really excited about the white paper protests. Because it just magically got through the censorship within a day. Then it's kind of like a floodgate thing. Once the water is out of the gate, you cannot stop it. And pretty much... Well, again, like there's no real, like you cannot, like legally, you cannot say that white paper protests forced the Chinese government to do a complete 180 and change their policies. But these two things happened within the week. There was a white paper protest across the country and across the globe. Then one week later, Chinese government basically kind of did like a 180 on their COVID measures, all their COVID policies.

[00:19:35.413] Kent Bye: Right. Right. So when you say you can't say that legally, that means if you do say that the protests caused change that you would like again be arrested or what do you mean?

[00:19:43.358] Zhuzmo: Well, again, it's just like that's why, again, going back to Teacher Lee's point that he didn't want to be considered as the starter of the protest because, you know, we're all we're sitting in Venice talking about this. We don't really have to bear like any physical harm like done to us later because we talk about things that shouldn't be talked about in China. But people in China will actually face actual danger in real life. So if you are considered a starter of a mass protest, there's a lot of people who went to the white paper protest in China who are still in jail or their family are still in danger. So it's just kind of like you are also responsible for lots of people and their sufferings after the protest.

[00:20:23.217] Kent Bye: So, right, right. So he wants to, is he trying to say, look, these people are doing more sacrifice. You shouldn't be putting the focus on me. Or is it just like he's trying to distance himself because he doesn't want to have to think about the consequences that the people that do put their lives on the line are going to face. But it sounds like he's just trying to put more of the emphasis on the people who are actually having stake in the game and doing something to sacrifice something in their lives.

[00:20:48.260] Zhuzmo: Yeah, I mean, in my opinion, it's definitely the former. He's still doing exactly the same thing, receiving bad news and posting bad news for people up to today. He has done a lot of interviews anonymously to other news outlets, so I don't want to speak for him. But in my opinion, it's definitely the first. He has explained it way better than me because, again, I'm just trying to rephrase what he said. Okay.

[00:21:10.836] Kent Bye: Okay. Well, I think we got enough context to the basic bones of the story that you're telling here to start to get into the way that you're telling the story in VR. Because you just told me what the basics of the story were here and right now on the podcast. But how did you start to try to think about visualizing social media, information flows, censorship? These are all abstract realms where it's all kind of like invisible signals going through the ethers of our computer network age, but what were the primary spatial metaphors that you wanted to start to use in order to have a pattern language to be able to tell the story symbolically?

[00:21:51.508] Zhuzmo: So that's actually the interesting thing. So like you saw the piece, you know, the piece contains a lot of people's archival footage from Teacher Li's account converted into 3D, seen in VR. It's kind of like a regular Twitter post, like Twitter images or videos being spatialized. It's kind of like the same spatialization system that Apple Vision Pro uses. So it shows a lot of hardship during the extreme COVID lockdown in China. That was actually not the intention. Because again, like another part of the piece is about origami because it's all about, that kind of is like a straightforward idea. It's a white paper protest about papers. So I kind of just connect the dots there. How about I just make everything all the Twitter posts origami? So in Weibo, there are like paper planes. And on Twitter, because Twitter's logo is a bird, I'll just use a blue bird. And blue bird in the game is actually in the representation of paper crane. That actually has another meaning in China because people have been saying this all the time. It's kind of like a folklore thing. If you put 1,000 paper planes, they make a wish. All the wishes will come true. So visually, you're sending paper planes. You're sending paper birds. That's the visual representation. And once you send the thing, it opens up, and it contains the 3D media. That was actually not the plan, because initially I didn't really want to focus too much on people's hardship under the lockdown. Because it's personally just not my interest. Because whenever people talk about stories in China, they always focus on the government, not the people. And if you just tell a story about people's hardship under a government that they cannot choose, it just kind of removes their agency. You need a lot of intricacies to actually show the agency of the people. That's also powerful and important, but I just didn't have the faith that I could pull it off just within a VR piece in 20 minutes. Then I was trying to see if I could just do other parts of Teacher Lee's stories without showing lots of COVID lockdown videos. I guess that was the same time there was like a lot of AI technology around spatializing 2D content, making them stereoscopic. I tried one on one of the COVID lockdown videos, and it just... I really felt that I was there. So that was the moment I decided, okay, this is a really powerful media spatializing all the 2D content that was sent to Teacher Lee. That was when I decided... Okay, whenever people send a paper crane, they're going to open up, then they're going to reveal 3D content about people's life in China.

[00:24:18.789] Kent Bye: Yeah, I found that whole sequence to be really, really quite powerful. Teacher Lee wasn't on my radar at all. I hadn't been tracking that story very closely at all. And so this was all new for me. And so it was a real joy and pleasure to be introduced to it through this VR piece because the story is actually anchored by some really subtle but really powerful interaction design where you as the participant and the viewer and the person who's going through the experience are actually the one who's kind of metaphorically hitting the send button where you're either grabbing hold of the paper plane or the crane and you're kind of throwing it out and it's kind of either going into this portal where it gets into the Chinese media ecosystem or it gets blocked and it gets rejected by the automated systems that are trying to filter out content that has been deemed to be too dangerous and it has to be censored. And so there's this kind of underlying metaphor of like you're ripping out a layer of the reality. And you can think about it as like VPNs or other ways that people kind of slip out of like into another channel that allows them to start to communicate. But that overall you have this insular nature of the Chinese internet and then these more outer layers of the global internet. And it's kind of the finding a way to illustrate the boundaries between information flow between these. And because you had created these really elegant metaphors with both the paper planes and the paper cranes, you were able to visually show the separation between these two networks with the paper planes that are flying inside of the Chinese firewall and then the paper cranes, which are the kind of the tweets and expos that are outside of the ecosystem. So yeah, maybe talk through that process of trying to figure out how to illustrate the boundaries between these two media ecosystems.

[00:26:06.403] Zhuzmo: I think that's actually the benefit of wearing so many hats. Because when I was working on an indie project like this, because you know if it's a bigger team, someone's in charge of writing the story. Someone's in charge of interaction design. Because if it's all within a small team, and in this case only me, because initially I know I only want to do two worlds. I want to do a Chinese internet, and I want to alter the actual internet. But at the time I was writing the story, I didn't think of a way to combine the two worlds together. But again, I really wanted to do this metaphor like where everything's paper-based, it's all origami, it's all because it's, again, it's a white paper protest. So I was trying to say, hey, how do you censor a paper like in real life? You tear it apart. So I was just basically trying to see if that worked in VR by literally tearing a piece of geometry in VR, then reviewing the geometry behind it. That actually felt really powerful. And it actually conveys exactly the sense of censorship. Or like going through, we call it the Great Firewall, I guess, between the Chinese internet and the actual internet. It just really translates this idea really well by having a hole between two pieces of paper, then there's a plane flying up through it. So then I kind of just designed a whole censorship idea. Like when your post gets censored, it's actually getting torn off. By like, I guess, like a monster that's made of the logo of Weibo in the Chinese media. then it's reviewing more information behind it. And when the sensor keeps tearing through the paper, when it reaches the last page, that's when you reach Twitter, which is kind of like the outside of the Chinese internet.

[00:27:47.558] Kent Bye: It's like you've broken through the matrix and you're free now. You've escaped the matrix.

[00:27:51.520] Zhuzmo: Yeah, yeah, that's pretty much it. You have to break through so many layers. In the case of Teacher Li, he actually has more than 50 accounts banned before his username Li couldn't be registered. Then he moved to Twitter. That's also kind of like the similar story.

[00:28:06.502] Kent Bye: Gotcha. Yeah, that makes sense. So one of the things that I've seen a lot with an interactive narrative is that you can have a film that you're otherwise just watching and it's kind of progressing at its own pace and that sometimes there's interactions where you actually have a lot of agency where it's actually dictating the outcome of what's happening and then you have different moments of interaction that are just kind of like essentially pushing a button to continue the story and And I feel like this is a piece that on the surface, it's like the interactions are very simple. And it's basically like the thing that you do to push the button to continue the story as it moves forward. But I actually felt that like using my hands because I use the hand tracking to then actually kind of pick up things and throw them. It just gave me a lot more. sense of being immersed within this world rather than using the abstraction of a controller so it just gave me this tactile sense of feeling my fingertips against my thumb as I'm picking up a paper airplane and throwing it and as I throw it and to have it like either go through or not go through and then have it fly around with all the other paper airplanes or the cranes that it feels like this is a visual metaphor for like participating in the polis of the culture the The people who are participating in the process of creating a culture or in the case of democracies, then actually creating the democracy. But in this case, it's like the aspirations towards having your collective voice heard. And so there's like a really powerful visual metaphor that it's just like a simple matter of these voices. planes and cranes flying around each other, but gives me a sense of what it feels like for me when I compose a post on X, formerly known as Twitter, and putting it out and feeling like I'm sending stuff out and putting things on the public record. So there's just that process of like throwing it in made me feel like a part of my voice being heard even though i'm not saying anything in particular but i'm sort of a courier as it were for these messages that are very important to get out around what's actually happening it was like a very simple thing that like normally would be something that i would consider to be just more of a a thing that was like interactivity for interactivity's sake in order to like progress the story. But it actually just felt like I was participating in it because there was a building up to the moment where you're throwing things in and then all of a sudden it doesn't go through. So then there's kind of an expectation and then it hits me a little bit more of like having my voice not be heard at that point because, you know, it's being censored and that kind of being spurned of having my voice be silenced and in a very simple metaphoric way, but still getting that feeling of being exiled or being silenced.

[00:30:50.888] Zhuzmo: Yeah, that's actually, so I'm generally a fan of, because again, I'm always a narrative game designer. And so I'm generally a fan of just using one type of interaction, but keep giving a new meaning throughout the story. And it's also used a lot in just narrative games outside of VR. But I feel like that's my special interest. Again, giving different meanings to a simple interaction throughout the story. It works really well within immersive media because when people put on a headset, they get overwhelmed immediately. You don't want to teach them 10 different gestures. Hey, you need to do this to defend. You need to do this to attack. It's kind of like combining two of my interests really well. I like designing simple interactions, and in VR you cannot do, at least for now, it's really hard to do very complex interactions and have people onboard it easily. So again, I'm just trying to combine these two things. You're always throwing something, sometimes a paper plane, sometimes a paper crane, and sometimes the paper goes through, sometimes the paper gets shot down, and sometimes you just couldn't throw the paper at all. So I'm just trying to combine all those narrative elements within this interaction, and I'm really glad that you liked it. Heh.

[00:31:59.232] Kent Bye: Yeah, maybe you could outline the meaning of that evolves because in story you have these constant dissonance cycles where you're building and releasing different levels of tension. And so how many times are you throwing it before it goes through? And just refresh my memory a little bit for the same interaction of this motion of the throwing of either a paper plane or a paper crane and then like how it kind of progresses corresponding to the story as it's unfolding.

[00:32:24.324] Zhuzmo: So on that, well, I don't want to brag, but I'm actually really proud of my method. I'm not sure if it came from me, because again, I came from flat screen game design. So even though I'm working in VR, I still try to have the interaction simulated without the VR headset. So you could literally press a button and then have the thing send away. Then have the thing being shut down. Because again, it's a narrative piece. I could just play the game. Then just hitting a button on my keyboard. Record the whole thing. Then you think of that recording as like a reference video. Just send it to friends. Say, hey, do you feel like this story is dragging at this moment? Do you think there's too much paper plane flying through before it gets shot down? Then just simply from a narrative perspective, and I would just ask my friends for feedback. They haven't even put the headset on yet. Again, they're just watching a screen recording. But then they could just kind of like try to feel the narrative tension. Hey, this is too slow. You need to hurry up. Oh, the pacing is good now, but then like it's too fast later. So I was trying to play with that. Again, it's trying to, especially in terms of the narrative part. Actually, I want to give credit to one of my friend who told me this way. They are designing a whole battle game or like action fighting game and basically recorded all the cut scenes and put them in Premiere and trying to do just video editing. If they feel this cut scene is too long, they just cut the whole thing. Then they go back to redo the cut scene within the game. I just drew a lot of inspiration from that process and I'm trying to adjust the pacing of the game this way. And so, yeah.

[00:33:55.858] Kent Bye: Okay. And so what's the order of which things are rejected or not rejected? Is it like accepted, accepted, accepted and rejected? Like how many times are you doing that interaction? Because you said you're doing the same interaction over this whole experience and I'm just trying to remember how it progressed throughout the course of the story.

[00:34:13.348] Zhuzmo: It actually depends on the story. So initially I was trying to onboard the player or the audience into this idea of throwing things to send postings onto social media. I was trying to have them make the connection between the two. So obviously when you start throwing things, initially it's all going through. Then the story has to introduce censorship at some point. Then when the censorship is being introduced, things get blocked down. And again, we just talked about the story earlier. Teacher Lee had to kind of like using him as like a self-protesting tool by recreating accounts, reposting things. And throughout this process, Paper Plane kept getting shut down. And when he moved to Twitter, he started to send Paper Crane because that's, again, the bluebird, the symbol of X, formerly known as Twitter. Sad face. It's not even a bluebird anymore. So when you started seeing all the bluebird, the bluebird getting through because there's relatively way less intense censorship on Twitter. But again, as Teacher Li realized, just posting things outside of the Chinese media system is not going to do anything within China. So then as the player, you also realize that, OK, I need to send my paper crane back to the Chinese internet. That's when the paper crane got shut down again because you just cannot go sending things back. And then that's when the story introduced the white paper. It's kind of like still a paper crane, but then this time the color is white. It's a white bird inside of a green bird, and that white bird going through the automated censorship. Then it's going back to the Chinese internet.

[00:35:42.803] Kent Bye: Okay. Yeah, so I think my experience of that interaction was that, like, I really felt it. I really felt, like, the being spurned. And then when it did start going through again, it was a level of excitement. And then all of the different Teacher Lee footage of the conditions during the lockdown, you know, that all the different... really wild and crazy stuff that was happening what could be arguably human rights violations the way that so extreme and all the different people getting locked in and like fire hazards and then there's a fire and then yeah just basically like people getting locked into their homes and people starving for many days and All this on the street reporting for people who feel like they're not being taken care of and gross negligence from the government and just needing to have somebody step in and make a policy change because of how much suffering was happening. And to see the tidal wave open for once it came to the white paper protests and just to see how much it was accelerating. And it's not easy to... represent a mass movement or a feeling like something's hit an inflection point and it goes viral. But I think the way that you had illustrated this white paper protest going viral, and then the popular uprising that was happening, it just felt really powerful again, to see all that footage. And I just found it so emotionally moving because it was these people who were putting their lives on the line to do what was right. And, you know, it just, Brought tears to my eyes and I just found it like, and again, it's like a documentary that if I'm watching it in 2D, I don't know if I would have had the same level of like emotional reaction had I not been engaged with my body within the story in a way that made me a little bit more committed to what was happening in it. It spatialized something that was very flat and abstract in a way that gave it a level of dimensionality and weight by doing that. So yeah, I just thought it was so powerful and made my list of some of my favorite experiences from the festival.

[00:37:46.320] Zhuzmo: Thank you. I really appreciate it. But that's actually one thing. So I'm generally really interested in making projects about internet culture because unfortunately, I just grew up with an iPhone. I'm like really addicted to all the social medias and I'm just like terminally online all the time. But that's the thing that frustrates me, that there is so much activism happening on social media that's not really captured properly, either just by news media or just then like the 2D medium. I'm not sure, maybe just like, again, the online protest is not getting enough attention as like real life protests or maybe just like the constraint of 2D media. I'm not sure. But yeah, that's why when there's a chance, when the subject matter is about internet culture, I was trying to make it in VR first, just to see if it works. Because that's going to another point, which is towards the end of the experience, as more people sending Teacher Lee their protest videos or protest photos, the whole environment transitioned into AR. So you can actually see where you are in real life because it's going to AR pass through. That's another thing that felt really powerful in terms of conveying the sense of changing perspective and changing the scale, because the whole experience is on Quest 3. And for the performance reason, there's always like papers flying everywhere, but they're always about like 1,000 things combined, like paper planes or paper cranes or whatever. they're all combined like a thousand things flying around you. They didn't really feel that overwhelming in VR, but once you remove the layer, just having 1,000 paper planes flying in your room, it just gets really overwhelming. So I kind of tried to play with the scale a little bit by showing, hey, this is how much paper planes in your room. Even though like actually in reality, it was way more than 1,000 paper planes. It was way more than 1,000 posts. But just showing 1,000 things in your living room, or in this case, just like in the VR booth, hopefully that just like showing the scale. And I was trying to use that as like a comfy, here comes a turning point. Things really got overwhelming.

[00:39:46.854] Kent Bye: Yeah, that was one of the other things I really appreciated about the piece is because, yeah, there is like a torrent of information that's flooding in towards us every day on social media. And there may be whole strands or movements like this white paper protest that I totally missed, but it was like a whole... vital turning point in the history of the pandemic in china and because things are very ephemeral the thing that you're doing in this documentary is spatializing all these different disparate posts but doing it in a way that is being fed to us in a linear narrative that's really building up to a larger story that's being told and so you you are capturing these little segments and nuggets of a story and putting it into like a flow of information that is able to have a level of narration and context setting that feels like you know there could be many different types of stories like this are captured and spatialized and really reminds me of a piece by ellie elsami iranian xr maker who is based in amsterdam and had a piece at IFADOCLAB 2019. In his piece, he was taking these movements with an Instagram and taking the videos and you were like flying through the space and seeing whole cultural movements that were being represented in that. And so, yeah, just another way of capturing and archiving these moments and time in history. and putting it into a VR experience like this, just to even document and say, yeah, this is a story that happened. And like I said, it could be a 2D documentary, but having the 3D stereoscopic effects as well is, I don't know, just something really quite compelling and something that, you know, obviously those cameras weren't They were around, but they weren't as prolific as they were back when those shots were being taken. And so it's clearly like an AI post-processed thing, but it still had a really impactful effect of having a little portal of looking into the past where you can understand the conditions of what was happening. Yeah.

[00:41:39.238] Zhuzmo: I was actually wanting to talk more about that, so I think it's perfect timing. So again, going back to my original point, so showing a lot of spatialized media within this piece was actually not the original plan. Just because, again, like the whole thing about Teacher Li that he received firsthand information, firsthand news footage from people within China all the time. And most of the time, that footage is really low res because they're just the nature of experiencing something firsthand. They don't really have fancy equipment. Most people just have their phones. But I feel like that's just going to be the case. It's already the case for all the conflicts around the world recently. Like all the firsthand footage. Like sadly, we're just like living through the whole like chaotic world right now. There are wars everywhere. And a lot of, like, mainstream media are just, like, just big news and all that. They couldn't go into the conflict zones on time. Or sometimes they were not even permitted, like in the case of this white paper protest. Like, most media and foreign media were not even allowed in China because of COVID lockdown. You just couldn't go in. So it was all just, like, people on site taking pictures, taking videos with their phones, and sending it to Teacher Li so that he could post it. And I just feel really frustrated that those things, they're really, really important archival-wise, but their quality is just not there. You just cannot show it on the big screen. You're going to see all the pixelation and stuff. And again, like going back to the use of AI technology, I just don't feel that you could AI upscale them But I just don't feel that that is going to represent what's actually happening, actually. And somehow converting it to 3D stereoscopic media, it actually passed my judgment of representing what's actually happening there. Because it's not, well, you could argue it's adding extra information by adding another eye. Like, because you are showing images from left eye for left eyes. Yeah, you're also showing images for the right eyes. But it's not adding things that were not there. So that passed my judgment. And once I see that news footage shot on their iPhone, they're probably not using iPhone. Like sometimes. Because again, they also go through so many layers of social media compression. So all the footage you saw in VR with no stereoscopic just feels really blurry. But somehow adding the spatialization to it makes it less blurry, at least to me. Because again, like you're watching it on Quest, like the whole Quest screen is more than 2K, and each media is mostly, if you download the video on Twitter, it's like 520p or something. So it was really low res, but again, spatializing, it doesn't make it feel that low quality anymore. And I was really just touched by that point that, okay, this is a good way to represent firsthand information shot in the conflict zones. And that just maybe could be something that could be used like in a broader way, not just within the white paper protest.

[00:44:32.938] Kent Bye: Right, right. Yeah. And so I see what you're saying that you don't want to like take an image and then add a lot of resolution. It may disrupt some of the integrity of the footage because it's subtly adding stuff that wasn't there originally and it is modifying it in a way that goes beyond what most editors would want. But it is adding some additional information with the stereoscopic. But what sounds like what you're saying is that it can still be a little bit low res, but just even adding the stereoscopic effects, your brain doesn't see that as an issue because it still gives you that illusion that increases the quality level despite the actual resolution.

[00:45:06.331] Zhuzmo: Yeah, again, like I'm not a professional journalist. There's definitely going to be like a lot of journalistic argument within this, like whether you could actually spatialize archival footage, making it stereoscopic and use that as actual historic footage. But at least it passed my standard because I was initially, I didn't want to show a lot of Teacher Lise's Twitter posts because they just look really pixelated in VR. But adding this spatialization layer actually makes me feel like, oh, this is the right format for me.

[00:45:35.515] Kent Bye: Okay. So I think overall, I just had this really emotional reaction to the piece where it's a simple story essentially, but it's really effectively told, you know, and it's always hard to say that because it's always like way more difficult than anybody's ever like really given credit for, but it also just works really well. And I know it takes a lot of work to get it to that point. And so... Yeah, it just feels like kind of a poetic interpretation that is using a lot of these spatial metaphors and embodied interactions that I think just it just overall gave it a lot more weight. So I guess that's kind of like my final take and why it was like one of my favorite experiences from Venice.

[00:46:11.838] Zhuzmo: Ah, thank you. Actually, again, all I know about Teacher Lee is a very indie production made only by two people. And actually, a friend told me this, and it kinda became my guideline when making indie pieces like this. You only need two, and you only have to make one thing, and make one thing really well. It doesn't, of course, necessarily apply to everyone, but I feel like it really applies to my creative process. So whenever I'm making a narrative piece that's within a very limited budget, whether it's VR or just flat screen or installations, I'm always just trying to do one thing, tell a very simple story using one simple interaction and making this interaction work really well. So I'm glad you like it. I really look up to all your interviews. I'm a huge fan of Kent. I really look up to him and I listen to like hundreds of his interviews. I have to say, I probably wouldn't be doing this immersive storytelling thing without him. Like, for sure. Thank you. Aw, thanks. Subscribe to his Patreon.

[00:47:09.359] Kent Bye: Yeah, thank you. Yeah, I guess as we start to wrap up, I'd love to hear what you think the ultimate potential of virtual reality and immersive storytelling might be and what it might be able to enable.

[00:47:21.067] Zhuzmo: I would think just immersive storytelling in general is going to be the ultimate form of entertainment and art. Because one could argue that it literally encapsulates everything together in VR. You could watch a movie in VR. You could play a 2D video game in VR. And you could also do way much more. And we are getting there slowly. The headsets are getting lighter and lighter, so keep doing what you are doing to all the immersive folks who are going to get there.

[00:47:46.128] Kent Bye: Awesome. And is there anything else that's left unsaid that you'd like to say to the broader immersive community? Any final thoughts?

[00:47:52.549] Zhuzmo: Subscribe to Kent's Patreon. It's only $5 a month, right? Is it $5?

[00:47:56.030] Kent Bye: Yeah, it's about, you know, it's donation for what people want to give. And yeah, it's actually, you know, I'm in the process of needing to figure out how to get more subscribers to really make what I'm doing sustainable. So I can continue to do it. But whenever I come to festivals like this at Venice Immersive, it kind of like reinvigorates me because it's like, the people who are the closest listeners to my work who are in the realm of immersive storytelling and making these really amazing experiences. So yeah, it's a real honor to be able to come here and see all these pieces and to talk to people like yourself and yeah, just really appreciated being able to take the time and with all things considered and I didn't know if I was going to be able to actually sit down and have this conversation with you. And I'm just really glad that we were going to find a way to make it happen and just be able to have your voice heard through the magic of AI. So thanks again for joining me here for the podcast.

[00:48:56.309] Zhuzmo: I'm from the indie game community. There have been a lot of experiments from the indie game community. Same thing, same experiment has been happening within the research community, within the film community. And I'm just really glad that Ken has been capturing voices from all different communities and bringing them together because I wouldn't know what happened from the university research side of VR. And that are also really valuable because the hardware is changing in the same time as the medium itself. I'm just really glad that Kent's podcast exists as an archive that captures all those innovations together from all different fields.

[00:49:27.364] Kent Bye: Thanks. Yeah. Yeah. I have a pretty boundless curiosity about a lot of things. And as I think about it more and more, I'm very driven philosophically in terms of like really seeing the roots of this technology as something that is going to catalyze some real paradigm shifts. That's what I hold out hope for, at least, is that it can break us out of the stasis of an existing paradigm and to allow us to become more embodied, more interactive, more participatory, more engaged communally, socially. And yeah, just also connect us more deeply to our emotions and our feelings. And I feel like in a piece like this, it kind of ties it all together. So yeah, thanks again for joining me. Thank you. Thanks again for listening to these episodes from Venice Immersive 2024. And yeah, I am a crowdfunded independent journalist. And so if you enjoy this coverage and find it valuable, then please do consider joining my Patreon at patreon.com slash voices of VR. Thanks for listening.

More from this show

Menu

#1452: Immersive Documentary about Chinese Censorship “All I Know About Teacher Li” Tops Venice Immersive Audience Favorites Survey