I interviewed Free UR Head director Tung yen Chou at Venice Immersive 2024. See more context in the rough transcript below.
Here’s their artist’s statement:
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.458] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So continuing my series of different immersive stories from Venice Immersive 2024, today's piece is with a project called Free Your Head. So this is like a performance art piece that is involved with the audience coming in and 30 or more different people in virtual reality headsets. And the whole idea is that they're trying to orchestrate synchronized movement amongst all these different people as they're turning your head and kind of following a ball within the context of VR. There's also a conductor character who, when you're watching from the outside, is also doing this performance dance. And there was some things that was difficult for me to track, but there were, you know, sometimes it was his left hand, sometimes it was right hand, sometimes it was his head that then was kind of driving the collective movement that was synchronized. So he was kind of doing his own performance dance. And then there was like the group performance. So I actually watched this from the outside first, just to see what the performance was like from watching it as a synchronized spectacle of all these people with VR headsets on. And then also then went in to have like more of a first person perspective of this piece as well. So we're covering all that and more on today's episode of the Voices of VR podcast. So this interview with the director behind Free Your Head happened on Saturday, August 31st, 2024. So with that, let's go ahead and dive right in.
[00:01:42.301] Tung-yen Chou: Hello, my name is Yen. I'm from Taiwan and I have been working with VR since maybe 2000. My first proper VR film is like 2020 and but I explored this panoramic video back in 2012. So this year I present Free Your Head project in Venice Film Festival. Yes.
[00:02:02.531] Kent Bye: And yeah, maybe you could give a bit more context as to your background and your journey into the space.
[00:02:07.228] Tung-yen Chou: So I have a theater background. I major in theater directing. And as a student, I... As a theater student, I have already playing with TV, camera, and using video on stage in rehearsal room in Black Box. So mixing what's live and what's pre-recorded has been... Or creating a... Creating some kind of dream or memory has been a thing of my creation. So somehow VR, XR, this medium, really like everything falls into place and then it's really a medium that almost like dreams come true. Yeah, yeah.
[00:02:42.699] Kent Bye: I know we've had a couple of previous conversations around Into the Mist and Traversing the Mist, and you also had another one in the middle there that I didn't have a chance to see, but maybe you could just give a bit more context as your kind of entry point into using immersive media, virtual reality, mixed reality as a part of your artistic practice of exploring different stories and also different ways of moving your body through space.
[00:03:05.382] Tung-yen Chou: So a little bit about the trilogy of The Mist which in the beginning it was only in The Mist which is a 360 video and was invited by Kaohsiung Film Archive so it was around then they invited a few Taiwanese director from different background and they asked me if I would like to make something with VR and I was wondering what kind of thing do I want to tell and what kind of potential and then I started to collect like what interests me and what has touched me in the past and I started to see a lot of other VR world. I went to Kaohsiung Film Festival to see works and then I proposed then two things. I actually proposed In The Mist and Free Your Head at the same time. At the same time I didn't know the technology a lot, so in my mind 3.0 had was like a four people experience and then I think it can somehow work you know all in one headset was there but you know not really like a remote controllable and then I was thinking you know everybody just clicked the start at some point So in a way I start with the video first and so it's the in the mist and I'm really into this it's so real yet it's virtual this like ambiguity of it this almost like so when when people were so fascinated about how real it is or how like you are in that space I'm like I'm kind of interested in, but I'm not there. But it's something quite interesting to me. It's a little bit like half dreaming. And I think that's the beautiful part. Like sometimes I think people have this experience. You're kind of awake. You can hear already the sound around your house, but you are also still in a dream zone. I think VR can bring that feeling. And then that's the thing I want to explore. Yeah.
[00:04:59.798] Kent Bye: Nice. And so it sounds like that these two projects started around the same time and that you went off on this path of doing this trilogy and then have been continually like working on for your head. I know last year at the Venice Production Bridge, I had a chance to see you pitch the project and I was like really inspired by, you know, this idea, this concept that you had. And so maybe you could talk a little bit more around like what were the other parts from your background that you're pulling in to try to like fuse in this other sense of embodiment and dance and performance in the context of people being in VR, seeing VR, but also becoming a performer within the context of watching your piece.
[00:05:36.835] Tung-yen Chou: So I mean as a theatre student you spend a lot of time reading and understand text, understand what the playwright or what's underneath everything but at the same time I was also exploring what visuals can do and what movement can mean or can hint. So I have been always fascinated with I wouldn't say abstract, but things that are not necessarily super narrative, directly narrative. And then going to dance performance has been my passion. And so non-verbal performance is one, actually the recent work they are not so, not really verbal. And so Free Your Head actually started with a rather, do you say whimsical? Whimsical? Whimsical. Whimsical, right? Right, whimsical. It's a little bit crazy. I mean, you know, Mathieu actually say crazily insane or something project that it's really because we observe people using VR. I mean, in the first few conference that, you know, the photos, like everybody in their suits wearing VR in a high-tech conference, And we're like, what are they watching? I mean, everybody must be asking that. And then sometimes they have moved at the same time. So I start to have this idea that that's choreography idea, that that's exactly something you can do. They can synchronize, they can be in different patterns. So last year when I presented, it was, we were already drawing inspiration from like pop culture. So there was Madonna, there's Pep Shop Boys, Go West. But there's also something that we think, oh, it's a little bit military-like. So later then we also draw our inspiration from like military parade or Olympic, showing muscles of a country or showing humans precise will, like how can we conquer the world, blah, blah, blah. I mean, so we actually did a local premiere last year in Kaohsiung Film Festival in a really big format with 72 people wearing headset together. And in that version, it was a mixtape of what I say, just like Western and Taiwanese pop inserted with some techno and Tchaikovsky's classical music. And this year in Venice is totally another approach.
[00:07:59.355] Kent Bye: Nice. It sounds like there's this, how do you choreograph this synchronized movement? I think it's the synchrony that is really captivating for seeing people all moving at the same time. And so you have either 30 to 36 or how many ever slots that you have here at Venice. there's also somebody who's sort of like the conductor but he's also like the solo performer he's kind of like doing these dual roles and at first i thought he was just the conductor but then i wasn't detecting any sort of like oh he's moving his hand here and then oh i'm seeing the corresponding like head movement so he kind of was going off on his own and then coming back and then so yeah maybe you could talk about like this relationship between the audience and then this conductor performer
[00:08:45.768] Tung-yen Chou: I mean, I would say that obviously, because we are performing in the evening time in Venice on Little Island, it's still daylight, so it's very demanding for the headset. And also with lighting situation, it's hard to really put the dancer on the spot. But to answer the question first, he actually has the ability to use both his headset and his controller to give signals. So in a way, it becomes not very clear that is he using his head or controller? But it's only one device each time. So he toggles between headset, left hand and right hand. So if he has done it so smoothly, so it's hard to see. And also there are parts that is pre-recorded. So the audience are divided in four groups. So even with the same movement, they are in different patterns. Yeah, so that's a little bit behind the scene. But I think with proper lighting on him, it's more readable. And also, that's why we are also making, I mean, we are continuing making this. And if we are in an indoor space, let's say a lobby in a shopping mall or in the museum, in the gallery, when people are, you know, with cool air around them, I mean, a lot of things can be played so people can really learn step by step. Oh, now it's the hand. Now it's the head. And now, I mean, we can also record movement on spot and play the movement. So I record something, I play, they do it. I record the second group, they do it. And this is what's going to be in our theater version next year. First, it's going to be in Taichung. And then we hope to bring this theater version around also.
[00:10:33.975] Kent Bye: As an audience member, the first time that I saw it, it was hard for me to discern, like, how is he doing this? Is it all pre-recorded? And you mentioned that it was a mix and I had just assumed it was all pre-recorded because it was hard for me to see any kind of synchrony from one of the, I wasn't looking at like, oh, now it's the left, now it's the right, now it's the head. So as an audience member, it becomes a little bit more difficult. But that was one of the things that I was wanting to see, like, here's an action and here's a response, you know, that connection between those two. And then when I actually did it, being inside of the headset, the experience was way different because I'm in a big group of, like, over 30 different people, but yet I can't see any of them, and I'm just looking at a black screen with a dot that's moving around. And so when I'm moving my head around, also, like, if my attention fades away for a second, then the dot's over here, and then now I'm out of sync, and then I can see myself in the video, like, oh, well, now I got out of sync. And so then as an audience member the first time, I was like, okay, why isn't everybody in perfect sync all the time?
[00:11:33.235] Tung-yen Chou: It's really interesting in one of the maybe the day before yesterday there somehow the whole VR gang arrived first so they are all on the left hand side and on the right hand side is all the you know people who pass by and then got in so then we have a huge professional like olympic team and then the other is amateur team and then that's crazy and then but we have been testing like how much distraction is too much how much because when it's only one point i mean you it doesn't stimulate your brain enough to keep on going To be honest, like last year in Gaozhong, it was really just this one bowl in a quite beautiful space, but one bowl in the space for 12 minutes. Then the space has a certain change, but it's just light and different little thingy. But this year, I really tried to start with a bowl to the nature, to go in our brain, to be a little bit like, you know, technology coming up and money and then a cave of history and... many many things really and then really because free your head start with you know funny idea that let's be free let's relax our neck but then as soon as i start the prototype it's not free at all it's i mean we become the control freak and then we want people to turn left right and then I mean, in the beginning, it's not a controller. It's really the touchpad on the laptop. It's just a prototype, right? So when you, like, little left and right, people would, if they follow the rule, they would . And you are really, like, hurting their neck. And it's like, oh, you must be very careful. So very soon we realized it's also talk about power rather than free. And then that's the contrast that makes it so interesting. And so I started to realize how multidimensional this work can be. And the other thing that I really want to share before I forget is when, especially when the audience, they are not, you know, these VR people, they put on the headset and they come in the group. I mean, some of them are, I don't even think they are a couple or lover. they will just touch the other's lap to calm each other down or something as if they are going to see a horror movie or something but then it doesn't happen just once so it's like okay we are in the headset but i'm connected with you we are together like i feel that's like striking something to me like Yeah, so one thing will be three years ago or more, I experienced great VR work, but I find it very hard to share with people. Even if we go to the same VR work, sometimes when we talk about it, it feels like we are not in the same work. And there's so little people who's hosting it. So that, I mean, the other part about the scale, about making it accessible, is also the reason why I want to make it in a public space and why Michelle and Liz make such an effort to book this place on Little Island for us. Yeah.
[00:14:40.222] Kent Bye: Yeah, I think that was one of the things that I had a better appreciation for after going through it, because there's what it looks like from the outside as an audience, and then there's what it feels like from the inside. And some of the stuff that's actually the most interesting to see from the outside is when I'm whipping my head around quickly but yet, you know, it's sometimes hard to keep up or there's not a lot of feedback or gamification to say like, oh, hey, I'm doing okay or I'm green or this is an average of what everyone else is doing. So you get this, because it felt very isolating and then the dot was there and then it, it felt a little bit of like losing motivation for, okay, what's the risk of like not, you know, being straight on target. And there's not a feedback for me to know, actually, if I lose my attention or don't do that, then it's going to look like I'm breaking the synchrony from the outside. And so there's the experiential component of being a performer was that the nameless free your head. And then I was like, oh, I was expecting kind of like a cathartic dance movement type of like, you know, like I'm going to a rave or something that it's going to actually feel like I'm at an ecstatic dance or something like that. But like you said, it's very much more of the rigid movement. So the experience of it wasn't as satisfying as what was like appearing from the outside. I think...
[00:15:56.172] Tung-yen Chou: From the very beginning, I want to be sure that I'm not making a game. Because in a game, you need to practice or you need to be skillful enough. And I want this to be able to... I need it to be accessible. So as much as I want to stimulate people's excitement to continue it, I also don't want to put on a score. I also don't want to make it too interactive because as soon as it gives you feedback, our brand actively want to search for something else. Like we become too, how do you say that? Let's say as soon as you want to look around, you are distracted. but then how much distraction do we allow or how much organic let's say organicness that we can play with for example there is a chunk of it where you do see the pass-through a very short one which is like 40 seconds but still that's the part that we imagine hoping that the audience will just look around and then like be a little bit messy and and things like that so To be honest, because if it's very game-like, very rewarding, what would it be different from the musical game? What would it be from Beat Saber, let's say, using your head to cut off things rather than your hand controller? So I would say that in the previous version, when it's very simple, very minimalistic, things like the pattern is very clear. But then since I have the ambition to share more about our story, about I would like to use for your head to reflect on our time, which is so uncertain. And then war is happening everywhere. And then we have been repeating history. but there will be a baby and then this baby is connected to you. That's what I want to say. So when you want to put such a narrative in a choreography work that people need to move their head around, you're basically putting the stone on your foot. So I do see the ambivalence. We struggle with it every day, trying to fine-tune the balance, definitely.
[00:18:15.669] Kent Bye: Yeah, there's a lot of different component parts there in terms of what you're going to optimize for. Because you could optimize for having everyone in synchrony and moving very rigidly, and then that's not going to be a great embodied experience for them necessarily. So, like, what's the experiential component? The people who are in the experience, they're seeing a narrative, which the people outside are not seeing the narrative. And then there's kind of like abstractions with the narrative where it's very poetic. And if you were to have me explain what the narrative was, it would be difficult to be able to identify what the beats were for how it's unfolding. And so I feel like it's kind of like a dance performance is very embodied. And then... the narrative component and then sort of interactivity agency. And there's a social dimension too, which part of the thing that I was also wondering when I did the experience was like, how social of an experience would this feel while I was in it? And it actually felt not social at all. And plus it was to the mixed reality part where I saw the conductor and at least other people, but at least having some sort of like indication of how this is like a group experience that we're all having together. was something that I was wondering if I would feel that. And it's a hard problem because how do you give a sense of being a part of everybody unless you're tracking everybody's head and then broadcasting everybody's head back into the head so that you get like, here you are, now you feel like a flock. And I felt the lack of feeling like a flock made it so my attention was drifting a little bit at a certain point. And so talking about what were the ways to kind of like keep the attention so it's engaging in a way that they can keep the task at hand, which is like their movements being synchronized and timed.
[00:19:52.924] Tung-yen Chou: I think the challenge in here that, I mean, I would still say that one version in our early meeting this year, we were thinking about maybe we should publish it so people can practice at home. But then soon I'm like, but that's, I'm doing a game. I'm just, you know, people can prepare and then people who got whatever score can join. And then that's not really it at the moment. And what else? Something. And I also think because normally, I mean, the audience need to see, kind of need to get a hint of what's going to happen, but you already know what's going to happen. You know the work from, at least it's
[00:20:39.619] Kent Bye: Yeah, I saw it from the outside first, which is what most people won't have. Yeah. Yeah. Yeah, because at the end, you do have an opportunity. When I was at the performance, I heard a number of people saying, oh, I can't wait to see what it looks like. And so I was there. I also wanted to see what it looked like and share that so that they could see what it, for them to be able to compare their experiences with what they saw, but also for me to have a reference before I went in just to see what it looks like from the outside and to understand how those two were playing together.
[00:21:09.780] Tung-yen Chou: I actually want to reply to this expectation, let's say like some kind of party situation, like you can really be free and let go. And I feel that for VR, the other day I was talking to Staya and then his recent sharing was like, you know, making shoes that fits everyone. And I'm exactly doing, I mean, we are all somehow from certain level doing similar approach. I want to make this accessible to people who is above, maybe 12, to people who is in 60 or more. Because in Taiwan, I mean, also people here, people in different age group join it. And then, I mean, one would cliche, I mean, you know, put on a label thinking that, oh, elder people would not move their head. But it's not true. It's actually true. they are very engaging and so the other thing is actually the music because the music this time we didn't really choose the beats that is very easy to follow like not that we wanted to make it difficult but then the composer that we work with was really working with the visual So that, let's say, like with all the other mixtapes, you can expect something already with the music. It's a lot more party-like when it's music that you have heard of. So that part is also very different. I mean, the whole thing that is inside a gigantic festival, makes it harder to bring this social aspect. Yeah, because when the performance is longer and then there are more several part of onboarding, because now the onboarding is more about putting it up, make collaboration, but then the warmup should be longer and then the offboarding shouldn't be directly about, but then that's really the time that we have. and then we are really under this amazing sun in Venice and everybody is, I mean, our console is right beside the performance booth. So it's, I mean, a lot of things could be better, but this is the version in Venice. This is the best version that we come up with this time. But already, I mean, in our prototype, also this year, not prototype, in our test out, we have been doing like, you know, 12 or 20 laptops. And then it's not all in one, it's all streamed. And then so I can definitely see all these people's heads, but that requires 20 laptops. I mean, also maybe all in one you can do it, but then there'll be... delay and then and then what do you do with it when you have 20 people walking around or moving around like do you really feel they are live yeah so that's also that's also what i've been thinking so i i think your feedback on these several points are really accurate about really about what's the what can make it better really like the sense of being in a group i mean i really can show you my notebook that that's like it happened so many times on my notebook yeah yeah
[00:24:28.197] Kent Bye: Yeah, I think it was wanting to feel like I was a part of a collective and somehow participating. And I did get that at the very end after we're done and I'm looking at the TV and I could see how I was in the complex of everybody else and flowing or not flowing throughout the course of the performance. And if there's a way to translate what I got out of watching that video at the end, that little clip, while I was in the performance, I think it would amplify that sense of that social presence of being with other people because it's I remember being at Sundance 2016, and they were doing the Sundance New Frontier, and they were showing different 360 videos. And there was a piece by Lynette Walworth that they said, OK, we're going to have 150 people in the, I forget the name of the theater, but it was the big theater on the Main Street. And we're all going to watch this together. And I'm like, I wonder how this is going to be. I've seen it by myself. And then I watched it with a theater of 100 people, and it was actually the exact same experience that I had from watching it by myself, because there was no indication. The time that it was different from watching in a group was when it was a comedy, when people were laughing, and I felt like, oh, there's this moment where I have this shared collective experience with a bunch of people, and I feel like I'm... having a moment collectively but when it's like a really super serious documentary looking at like the impact of nuclear weapons in Australia or something it's just like it's not something that was a social collective experience and so I feel like like you can get a bunch of people in a theater but unless they're somehow tethered in some fashion then it can feel like a very isolating experience. And I felt like it's a hard problem to solve. How do you give that sense of social cohesion? The thing that I wanted to see, and it kind of gets to that gamification, is like, okay, where is everybody's center of their gaze? I mean, it's hard to know exactly where their eyes are looking at, but if you just take an average of where their head position is relative to the dot, then if I saw everybody's dot to see where it was, then I would at least be like, well, if everybody's like, slightly offset then and i'm here and i can see where i'm at in the flock then it yeah i'm just kind of imagining like that would give me a sense of like we're all doing this together and then the payoff would be at the end to see it what it looks like from the outside so i feel like there's this kind of like participate in the moment and then oh the payoff is like here's how you did as a group
[00:26:44.802] Tung-yen Chou: We have also the, well, apart from the see-through, because now the see-through is very heavily like also going to a train situation. But we are experimenting this use of like already wearing the VR and then you can see that your performance directly streaming to your headset. So that's part of the thing we have been trying. But the bandwidth and the internet stability is not there yet for the outdoor performance. So it's going to be more on the theater version.
[00:27:19.306] Kent Bye: I know just being able to pull all this stuff together technologically, I'm sure was a feat. And I know when I was there for the first performance, there was, or it might've been the second, there was like interference from other things that were like messing with the wifi or whatnot. So maybe you'd talk about like some of the technical challenges or hurdles that you had to overcome just even to put this all together. Yeah.
[00:27:41.383] Tung-yen Chou: So let's say back in Kaohsiung we have all the video documentation team and then they were flying a drone and they were asking if they can so we say why don't you come during our break and then we can test and then so they came with one drone and then it went fine our signal was totally fine so the first performance with their drone flying. It was just nice. And then so they said, can we do it again? I said, of course. And then we didn't know. What we didn't know is they changed a bigger drone. And then that drone completely blocked all the signals. And so we are very aware that certain drones can have like very strong impact on our connection and this year so we have been contacting to see if venice film festival is flying drones and they i mean from what we know they are not no one is allowed but then the first few days there are several drones on the sky i mean especially i mean the first day that you are outside it's a press day so no drums there but then as soon as the record be the very first day yeah there are drums so it really affects until they turn off the drum and then we start that's what you saw i think that's the only thing that we i can more directly make an example because it's like if you know what kind of drone or what type what model of it you can probably avoid the specific channel or frequency but then it's just it requires a lot of communication because at first they thought it's the at first we were told it's the the drone from the police and then then no not really it's actually in the video in the live broadcast so but yesterday there weren't drones so hopefully it's not coming back yeah yeah that's the first I heard of that there hasn't been too many other experiences that have like run into that problem that I've even heard of that of being a thing When you need to send signal every single millisecond, so that's gonna interfere because the other signal will be more like Once they start it starts maybe streaming maybe streaming will be also interfered. Yeah. Yeah
[00:29:49.574] Kent Bye: And so maybe you could talk a bit about the technological architecture to even make this happen, because you have a whole team of people sitting behind computers. You have this performer who's dancing. And then you have everyone with headsets. And it's all starting at the same time. But you're also potentially live mixing in the different hands. And so what's the pipeline for where the data is coming in and out?
[00:30:10.921] Tung-yen Chou: We have at least three major pillars of the interaction team. One that Yu Jie comes up with different prototypes that how do we actually give signal to the audience using existing technology or so in the beginning it was really nine dots so it's like nine direction but then within these nine dots we can make like a score and then it has bpm so that it's very musical based but then everybody's sense of tempo is very very different so very soon we transfer it to uh like the touchpad and then ipad and then and then controller directly and then so he comes up with these basic interaction system. And then FukuFish, this company, they designed a platform that monitors all the hundreds of headsets. I mean, we tested with 100 headsets and during our performance in Kaohsiung, for 72 people with all the crew on the side and with all the standby headsets, I think we, every time at once, more than 100 headsets is working and is receiving signals. so these platforms synchronize with Appleton Live which is a music software that the reason that we want to use it is that it's a live musical program that has track so actually all the movement if we want to record it or send live signal are actually translated onto the sound system so that it's live time
[00:31:52.418] Kent Bye: Like as a midi single?
[00:31:53.920] Tung-yen Chou: I think it's a midi one, but I'm not 100% sure because there are so many signals possible. I think it's midi, yeah. So then we use live to group them and to record them. And also even with this quantization, like you can smooth your tones so some signal is too sharp can be smoothed. And then can be, yeah, I think this is the basic structure. And then we have a video effect, video designer, no, visual designer team, and call interaction and control as interaction. So they are all here and then they need to synchronize their timecode, their language, and then all the tools together. It's a big task, actually. Because we really work until the last minute really. We have been constantly doing iterations of the visuals.
[00:32:57.907] Kent Bye: And so in terms of the conductor performer who's up at the, what would normally be the conductor's little podium, are you like recording the movements that he's doing through one of the either hands or the head and then like translating that into like, okay, it's this dot and this vector and this speed or like, cause there's a little bit of a dot with a blurring or at least it's not just like a solid dot. It's got like kind of a motion tracked. so that you can feel like you're at least in the tail of a comet with your gaze because you have a representation of where I'm looking. But how do you have to do a smoothing or translation for if you're taking input from the conductor and then being able to use that as a real-time input?
[00:33:39.335] Tung-yen Chou: and it also depends on how concentrated the audience are because now this time we really took off some very difficult part because there are some times it can be very challenging that it can be very quick and it can have this kind of harder punch so it's like smooth and quick but this time everything's more averaged because as I say we are trying to make sure that people don't give up too much too soon yeah and was this your question yeah so it sounds like it's not like a direct input from the hands but you are making it a little bit more average so it's not like always a one-to-one translation for the movements yeah At first, I mean, even now, it can be very direct. But still, with certain help, it's not super direct. Because we don't know how much our hand, I mean, if it moved like one centimeter or less than one centimeter here, but then it translates to a big space, it's one meter for the audience's hand. So it's really shaky. So that needs to be calculated and then use algorithms and things to polish. I've been trying to make a good balance of it.
[00:34:57.186] Kent Bye: One of the other things I wanted to ask about is this balance between the audience that's performing and then the dancer who is performing because it seemed like sometimes what the conductor was doing was like doing some dances that were much more dynamic or moving or it was almost like he was upstaging what the audience was doing and so it felt like a little bit of a conflict of like who am I paying attention to and like usually a conductor is facilitating the movement of the groups but then sometimes he would like seemingly abandon that role and go off and do kind of a solo dance so it felt like a little confusing as to like who I was supposed to be paying attention to or but also like what his direction or role was and that was from the outside and from the inside I saw him just a blip a couple of times through mixed reality but I didn't see much representation of him while I was in the in the headset so it felt like a lot of what the conductor was doing was maybe more geared towards the outside audience. But just curious to hear a little bit more around the direction that you have and the balance between him being a facilitator versus being kind of a performer. Yeah.
[00:35:57.569] Tung-yen Chou: For me, his role is also multi-dimensional, and in different sequences, he can be in different roles. And obviously, in a 10-11 minute performance, if he changes the front too many times, it's not going to be very easy for everybody to pinpoint. And actually, using the turn conductor, now I realize maybe it also gives too much of a hint that he's constantly conducting. So I think the role of the dancer is there are many roles that I put on him or he's trying to because I actually because he actually has a so his dance training is not really like you know dance school kind of training. It's more like street dance, b-boy dance and but then it evolved a lot of imagination so I feel the easiest way will be sometimes you are a dictator, but in a snap you can also be manipulated by people. So sometimes he, the performer on the stage, also feels like being affected by the mass public's opinion so there is a switch of that because we really don't have a proper lighting on him so that's to me quite unclear but that can be solved really yeah
[00:37:27.376] Kent Bye: I think it kind of gets to like this dialectic between like the freeform dance, like this more ecstatic dance that I think he sometimes slips into versus the more rigid, structured, synchronized, everybody together moving. And so like there's kind of a conflict between like what I wanted to see was he is helping to facilitate the collective orchestration of everybody moving and synchronizing together. Sometimes he would kind of go off on his own and then the audience would still be tracking it. But yeah, so then it'd be like, okay, what is happening here? So I think that was at least from my perspective of watching it, like, I mean, that could have just been my expectations for what the piece was going to be, you know, in terms of like, okay, here's what I imagine what a conductor conducting a people, like what it looks like to be an orchestra. And rather than, you know, people playing instruments, he's like controlling their heads. different than what that was.
[00:38:24.992] Tung-yen Chou: Yeah, I think I get what you say. On one point, I have been saying that with all the limitation of outdoor performance, blah, blah, blah. I have been saying my reason that things cannot be more clear. But on the other hand, it's really also not making Like not purposely making it unclear but also they are all kind of blend together and to be honest like the inside experience, the inside visual was not shown to the public was also a decision. But I feel that it was a good decision in Kaohsiung, but maybe this time since the visual is so complex, maybe we do need to give the outside audience some hint and what they are looking at. That's like an ongoing approach, ongoing research really.
[00:39:27.370] Kent Bye: Yeah, I know. For myself, the outside and the inside experience were so distinctly different that I'm glad I did both, but it was hard to get access to either one of them at the same time. And I think there's an inherent occlusion that happens where, as someone in the headset, it's really impossible to know exactly what they're saying But yeah, just to kind of see how the inner and the outer world were connected. And I think that's an interesting conceptual idea. How can you have what's happening on the outside, but also correlate or synchronize with what's happening on the inside? And as of now, those two worlds felt pretty distinct of being on the outside, knowing what they're seeing, and then being in the inside and not knowing the collective aspect of it. It's not an easy problem to solve. So you've kind of entered into a very almost intractable design space problem. But I think it then becomes like, well, what are you really optimizing for? What the experience is going to be for each of the different stakeholders here, both the performers, the people who are in VR, and then both the audience who are watching it.
[00:40:27.711] Tung-yen Chou: I wonder if it's also because with this technology with this interactivity that puts on as a structure as an important structure like we want to know what is doing what very clearly because when we go to a party we wouldn't say why is he moving that who is leading this dance And then, I mean, because I guess it's more natural that you just move along. There's nothing right and wrong. Or in a rehearsal space, you can rehearse, you can have time. But then, so these several, I guess, maybe three major difficulty or major things that we are facing or solving is that inside and the performer what's the story inside, what's the story the performer is portraying and then how the audience from outside not only get what the performance role is but also get what is he conducting or what is he doing but then all that condensed in this very very very very short time is quite challenging but at the same time making it much longer isn't also gonna work yeah but the other day Mattia does say that it can be a little bit longer but I don't know if it works for everybody like how tolerant you are with your headset and moving your head and then there are parts that you don't move your head and I think today is maybe the third day that we are performing it or fourth day. I mean, apart from the press day. There will be something that we will find. I think like every VR work, you really need to see how audience react. They react to things very differently. And then... Just like what I was trying to say, that it should be interesting enough for people to follow, but it should also make the audience focus enough. And how do you make that autonomous? That's really difficult. Yeah.
[00:42:41.673] Kent Bye: Yeah. Great. So what's next with this project?
[00:42:46.317] Tung-yen Chou: As I said, I'm making a theater version, and in the theater version, they are not just dancing with their heads. I mean, there will be, in the theater, there will be maybe 30 audience wearing the VR headset, and then maybe 100 audience sit around to observe what's being created. So the audience member will actively participate, actively join the creation. So let's say there will be six chapters. So in chapter one you move, in chapter two you make certain sound, in chapter three you make beats. It's a bit like creating layers of music. but then only in the final 10 minutes you put everything together and then it's all co-created by the audience whether you are in VR or off VR and I think in that scenario the problem that we talk about can be somehow solved like being in the community being in a group being a collective experience like co-creating something like not just following a ball or something And so that's my next project, really.
[00:43:58.689] Kent Bye: Yeah. Awesome. And finally, what do you think the ultimate potential of this type of immersive media and immersive art might be and what it might be able to enable?
[00:44:12.248] Tung-yen Chou: To be honest, I think major companies are closing down and tech giants are leaving narrative VR. But there must be... I think I see what touched me in this medium is not going to be... It's a little bit like, I kind of sense a lot of pessimistic situation going on. But I also see, know its potential. I mean, being in VR as a VRXR creator, I mean, we are in the... I'm almost repeating what the other people are saying. But we are at the beginning of this medium. And then a lot of people still don't have a clue. But when I see a lot of people in different age want to try when they stop by and then they want to go inside and try and then share this experience and talk about it. or you know saying that oh the dancer is using his hand it must be his left hand like you know making story up or like trying to understand the work i think that's already a that's my approach of continuing making it but then the potential of this medium yeah it's the all-time difficult question um Yeah, I tried to answer this several times but this time I'm specifically more puzzled and hesitated actually. Maybe in the previous few years I'm like having hope and then I feel that But I feel like there are films that are made with so little budget. I mean, there are VR works that are made with almost no budget and they're so great. And then we have been saying it's such a difficult medium and you need to work with code, Unity, script, everything, signal, strong router, everything's expensive. But I guess I just want to say that Even with the tech giant pulling out, there will be people trying to continue to use this medium to share stories and share experience. And I think that's the only thing we can hope, really.
[00:46:51.189] Kent Bye: Yeah, I feel like there's a part of a trend of having more and more installations and group projects that are here at Venice Immersive, yours included, where it's finding new ways to come together and be together. And so I feel like there's a... There's a thread of the beginnings of something, but it's still like, it's still a long ways of where it's going to be to be fully formed. So there's a lot of, a lot of stuff to kind of still be figured out in order to really fully facilitate that. Those new modes of coming together and being together. So I feel like your project's kind of like capturing the spirit of that. Thank you. Awesome. Is there anything else that's left inside? Do you like to say the broader immersive community?
[00:47:27.481] Tung-yen Chou: No, no, no, no. I think I've pretty much covered everything, really. If I come up with something, I'll let you know. But, I mean, we'll meet somewhere else. I mean, this is organic work. And this is the first time that we bring this version to a Western society. I mean, there's these Instagram reels about Asian versus Western society wearing masks because Aristotle's The Greek philosopher asked us to ask why, and then Confucius asked us to listen. So when we were asked to put on facial masks, we would be like, yes, let's put on facial masks. And then people would be protesting on the street like, no, you are interfering my freedom. Why should I wear a facial mask? And then maybe this is like echoing how people react to Free Your Head. Yes and no, like people wouldn't mind doing a little 10-minute TikTok dance together, why not? But also, I mean, I'm making this thinking out loud longer than it is, but... I mean we are definitely bringing this work to next stops and then I hope you also people and also the listener of this podcast also get to see different version of this because we are looking forward to work with local dancers and choreographer because with very different dance movement and dance training Can you imagine a ballerina holding this goggle and then use their movement to conduct? And then, you know, when the music already suggests a lot, a lot, a lot. And then, and also different type of movement. And then what if people can always go around the space or something? I mean, there's a lot of potential that I want to try. So, yeah, like, what if we play this Free Your Head in India? Because actually, last year, during the production bridge, I was talking to an Indian producer, and he was like, we have the technology, we have the music, we have the people. Come, do it in India. And can you imagine, it can be a very... Can you imagine this in the Bollywood music scenario? It can be very direct, right? It can be very visible and understandable. And that would be totally fun to do. But this time, this time I think I want to make something that is not just, that is a bit more than dance, a bit more than free your head, that is actually questioning freedom. And I guess that's the part that makes everything a little bit obscure. because I don't want to give freedom or free or conducting a definite answer so the dictator is sometimes went very crazy because technically he can put a button and then hold the audience gaze so that the thing doesn't move but only in a very short section that he does that So I also wonder, because I was thinking, should we have signals signify which part of the headset is giving signals? But then we didn't want to make it too technological, focus on, oh yes, it's the head, it's the hand. So yeah, that's my conclusion.
[00:50:59.614] Kent Bye: Awesome. Well, it's certainly, you know, starting with the head, I'm sure you could expand out to the body and other ways of having, you know, ways of using the technology to create this group synchrony or group experience. And so, yeah, I think it's the beginnings of something where I'm sure we'll see it continue to develop over time. So, but yeah, Yen, thanks for joining me to help break it all down and to explain a little bit more about your process of creating for your head. So thank you.
[00:51:21.541] Tung-yen Chou: Thank you. Thank you.
[00:51:23.683] Kent Bye: Thanks again for listening to these episodes from Venice Immersive 2024. And yeah, I am a crowdfunded independent journalist. And so if you enjoy this coverage and find it valuable, then please do consider joining my Patreon at patreon.com slash voices of VR. Thanks for listening.
