I interviewed director Steye Hallema about Ancestors that showed at IDFA DocLab 2024. See the transcript down below for more context on our conversation.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.458] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the structures and forms of immersive storytelling in the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So continuing my series of looking at different experiences that were being featured at IvaDocLab 2024, today's episode is with a piece called Ancestors. produced by the Smartphone Orchestra. So this is like a social dynamics piece where you're using the phone to have these different interactions and encounters with other people. And you're essentially having these virtual children that you're producing. So you take a snapshot of your face, and then they're able to pair you up with other people. You're creating these different virtual children who those virtual children are connecting with other of the virtual children. So you end up having these interesting conversations around ancestors, your family, and these unique social dynamics that are happening in the context of this interactive and immersive experience. So we're covering all that and more on today's episode of the Voices of VR podcast. And I should note that my voice continued to give out, so apologies as my voice gets a lot more crackly. So this interview with Staya happened on Tuesday, November 19th, 2024 at IFA Doc Lab in Amsterdam, Netherlands. So with that, let's go ahead and dive right in.
[00:01:26.112] Steye Hallema: Sure, I am Stije Hallema. I am the creative director of the Smartphone Orchestra. Next to that I also create my own VR projects, like the imaginary friend that you might know. And with the Smartphone Orchestra, which we are here at IDFA, the Smartphone Orchestra is basically a methodology and a technology to use the phones of an audience to create stories in which the audience themselves play the leading roles. So that's really what I do, I guess, as a maker, thinker, director. I like to tell stories that are about the person that's undergoing the story. It's my niche.
[00:02:04.249] Kent Bye: Great. Maybe you could give a bit more context as to your background and your journey into the space.
[00:02:09.045] Steye Hallema: Well, I always like to say, because it gives me a certain flair, that my father is a magician. And that's actually true. He's a magician. He was a world champion in some form of close-up magic one day. And my mother was an art teacher, so I've always been doing stuff like that, came naturally. Then I went to a very interesting study, which was a combination of the conservatory and art school. And in that school, we always had to make something that would appeal to more than two senses. So I couldn't just make a movie, you know, be sound and image. If I would make a movie, I would have to like incorporate smell or touch or whatever. So that's basically my educational background. And then basically during my adolescence, I became part of a friends group. And we kind of became this sort of arts collective at a certain point. We were structurally funded by the Dutch government to create kind of multimedia theater. I think we were really pioneers in putting the first computer on stage and using it to make live stuff on stage. So, for example, our first success was Wortel Combat. Wortel means carrot in Dutch. And so this was a joke from Mortal Kombat. And it was Duke Nukem, which is a first-person shooter, would fight Pac-Man, who was basically like an old Rotterdam hardcore dance, bold guys. I don't know if anyone has a reference for that. And they would fight for the hand of Lara Croft and would make computer games on stage that then the audience could play. And this is back in 2005. So that's, I think, where it started for me to dawn that it was really interesting to use the audience as your canvas. What helped with that is that I am still a musician, but I have been writing a lot of songs. I've been performing a lot with bands and my own band and at a certain point I felt so institutionalized being a singer that it's so like the institution is so much that it's about you and you have to make yourself interesting and at a certain point it made me puke so that was I think also yeah some sort of motivation to be even more interested in what can I do with the audience instead of expressing myself
[00:04:27.605] Kent Bye: Yeah, and I know we've had a chance to have other conversations about some of your previous work where we dive into the origins of the smartphone orchestra. And because this is another smartphone orchestra piece that's showing here at IFA Doc Lab, I'm wondering if you could give a brief recap of your journey into doing this type of interactive media with the phones. musical elements, but in this case more of like the social dynamics that you're using to create these kind of encounters with group dynamics that start in pairs and then get more and more as a unity by the end. But maybe you could take me back to the beginning of where this idea for the smartphone orchestra really began.
[00:05:06.147] Steye Hallema: Sure. So as a musician, as a singer, you know, you have to attract attention. So because I was part of this multimedia group, I always would do like crazy digital stuff. And so I would always try to come up with ruses to attract attention. And by chance, actually, a friend of mine made a set of 40 helium balloons and he put a speaker underneath each balloon, which is a beautiful sight. But he wasn't the musician or sound guy. So he had one line of audio going to the 40 speakers. And I saw it and was like, dude, I'm going to make music for this. So I put this 40 speakers in my studio and I experimented with creating music for all these 40 speakers. I put 10 different tracks to them and the speciality of that sound was so cool to be playing with. I guess because our brains are unknowingly always calculating space, we orient a lot more than we think with our ears. And when all of a sudden that sound is organized sound, it's really like, it makes your brain go like, ah. And while I was doing that and being inspired, I sat in the train, this is 10 years ago, 2014. And that's sort of when the zombie apocalypse started. So I saw that everyone started staring at their phones. And I had, I guess, a classical Eureka. All of a sudden I realized, wait a second, there's a speaker in these phones. And through the internet, we must be able to synchronize all these phones. So that became the first iteration of the smartphone orcs, really a musical idea that we kind of like developed, which is really cool. But at a certain point, while doing this, a bigger idea started to emerge from it. And it was that the smartphone orcs is a tool to engage the audience, to reach out to the audience personally. And I realized, wait a second, the big idea is not the music. It's really like making group experiences in which everyone is involved. And I think the potential of this is huge. I really feel like I am like one eye, you know, in the land of the blind and I see it. I tried to desperately open my other eye to see what we all can do with all of this. And so thus I have been starting to explore what kind of stories I could tell with this. One of the first ones was a really nice collaboration with Enneagram. And in this it was called Work. And we explored our dependency on smartphones. So basically the smartphone gives you assignments. And it's this really smooth voice that tells you, ooh, do you think I'm sexy? You know, it's a bit, you know, it's Enneagram funny. It's really lovely. And then at the end, when you've learned stuff as a group, you have to put your phone away and you have to figure it out yourself as a group. And you feel that fragility that as humans, analog, we have to figure it out. And you realize, oh, wow, this is the plot, which I thought was really cool. And then in 2016, I think same year, like when the Facebook scandal, the Cambridge Analytica scandal happened. When I learned about this scandal, I realized this is a story I can really tell. So we created the social sorting experiment in which the audience comes into the venue, they create a profile on our system and then they are asked to stand on a grid and they have to rate each other. So who has more beautiful ears? Who is more attractive? Who will live longer? Who would I rather save from a burning building? Who will I let babysit my two-year-old daughter? like different questions which are funny and serious and weird and and then the next phase we order the audience based on their own ratings which is a super provocative but strong illustration of the power that big tech companies acquire over us while we're just like liking cat videos and it was a really big success which is great we've toured it all over the world and then corona happened and then well no one wanted to be together with smartphones But we had time to come up with new stuff. So another one was Emoji, which is basically questioning the premises. Emoji might be the language that most people in the world speak with each other, but it's made up by a foundation that is owned by eight companies. So for the first time in the world, we have a language that's made up by companies. And I personally love emojis. I think emojis are great. But I think it's, you know, you have to question this kind of thing. So the gameplay is that people are coupled in pairs. And then, for example, you, Kent, would get one emoji and I would get four. And you with your face have to mimic that emoji. And I have to guess which one it is. So in no time, the whole audience is making funny faces to each other, which not only makes it an interesting piece to question media technology, but it's also like a really nice icebreaker and Kickstarter. and a way for people to get to know each other. When you've been making funny faces to each other, then standing in the elevator, you know, it's much easier to start a conversation.
[00:09:53.468] Kent Bye: Also opportunities to pick your favorite emojis, which is also an expression of your identity and lots of discussions around which emojis that you're using and not using. And yeah, that was another dynamic that you're able to share parts of yourself with either a group of strangers or people you know.
[00:10:09.720] Steye Hallema: Yeah, yeah, exactly.
[00:10:11.434] Kent Bye: So yeah, I guess as we come up to you now, Ancestors, where did this project really begin for you?
[00:10:16.926] Steye Hallema: Well, for me, I'm always trying to find ways to involve the player. So actually Hanna Haslati, a Finnish artist who did Captured, she was busy with face merging with an algorithm that could, with an AI that could do that, or I think it was still an algorithm. And she asked me to help her with the concept. And like straight away, I felt like, okay, this is cool because this can involve the player and then we can create children, but these children can create children and these children can create children. And this would be a way to connect the whole audience. Yeah, that was just, I guess, also one of those, because you go like, wait a second, this is it. And because as the creative director for the smartphone orchestra, I'm always looking for how to, no, let's go back a second. I think how I like to work with new media is really Marshall McLuhan. The media is the message. I like to look at what can you tell with the medium and then find the stories from there as opposed to what we're used to with film. Okay, I have a story, let's tell it with film. I don't think that works yet with most of the immersive media. So if I find a way to involve you in a natural way and to connect you, which I think is one of the main themes of the smartphone orchestra, because it's always about a group. It's always about the group dynamics, how we relate to each other. I felt I must make a smartphone orchestra piece with this. Yeah.
[00:11:36.201] Kent Bye: So you outline the overall structure of the piece where you're merging the faces of two people and having these virtual children, but then by the end, have everybody be the ancestors to this one virtual child that's being produced. And so I guess as you start to think about this journey and arc, like first of all, onboarding people, you know, choosing someone to have a virtual child with is a lot of pressure and a social situation. And so there's kind of a a gamified way that it's less stakes. It's not like high stakes. It's a way that you are making a choice for people where they don't have to feel like they're having the agency to have to make some of those choices. So you're kind of pairing up people in a way on the back end, some sort of algorithmic randomness of maybe people they had previous. What's that?
[00:12:20.429] Steye Hallema: That's one of the statements we're making. We're coupling people completely random. So men can have kids with men, women can have kids with women. I think that's really important because if we would start to order that, we'd get so in the woods. And I like the statements, you know, it's a hypothetical family. And the statement we're making at the end that we're all connected, like there's one ultimate ancestors, which is that Luca, I believe, like this last universal common ancestor or something. So we are connected. I think that it's a hippie message, but I think it's more urgent than ever.
[00:12:50.752] Kent Bye: Part of my experience of going through this piece was that the less number of people there were, the more agency and expression I felt that I could have. And that the bigger the group, that felt more distanced in terms of like, you know, there's a book called The Art of Gathering by Priya Parker, where she lays out different optimal group sizes of people. And that six to eight people ends up being a sweet spot for dinner parties. And I felt like once it went beyond eight people, then it starts to get more difficult to manage the dynamics of a group experience because it ends up being people who are active and people who are passive. It's just a different dynamic from a one-on-one to four people, and then once you start to get beyond eight people. So there's this inherent tension in the piece where how do you maintain that connective tissue between this group experience while at the same time getting larger and larger, creating that more group identity and then trying to maintain that sense of being able to participate. So it feels like that's the core challenge of a piece like this is how do you actually do that? And I feel like you've taken a first cut and I'd love to hear some of your thoughts in terms of like how you started thinking around this overall journey that you want to take people on.
[00:14:02.563] Steye Hallema: Yeah, well, you kind of start out with the question, can I do this? Will it feel like you're connected to your great, great, great, great grandchild? Because there's an urgency to make those longer lines. There's a really interesting book by Roman Kronorski, The Good Ancestor, that's advocating that we should think in longer lines. And I was really captivated by this book after I had the idea. When we did tests, at a certain point it was interesting in the feedback, the audience would basically take it on us that they didn't feel connected with that grandchild. And at a certain point I felt like, wait a second, it is completely natural that you don't feel so connected from basically your great grandchild onwards. That is the problem. So Marshall McLuhan felt like I have to use that. Like the grandparents round, I really love it. Like when they are asked to share something like a beautiful gift, a special moment, something of value they got from their grandparents, you just feel It becomes sort of tender and soft in the room. Then hardly anyone really knows their great-great-grandparents. I have one memory of my great-grandmother and that's it. And I have a memory that my grandmother once shared with me about her father. That's all I have. So I felt like, okay, let's then do a collective thing. future speculative fiction game so everyone can like come up with future visions and then to the next generation the great great grandparents you know it was really hard to create connection I so I used that I told them like now like show each other your great great great children and then I asked the audience like tell each other about your great great great and then it's just you know it's like shit I don't know anything And then I try to create a connection again by basically there's a phenomenon that I use in the piece. It's the grandmother effect where humans had a big advantage in evolution that there were also great grandparents that could share knowledge and time for the relatively long childhoods that we have as humans to help them develop. There was a bigger pool of knowledge. There were extra hands. extra safety. So I'm basically stacking different grandparents' interactions. So you have to imagine that your grandparents are on the 10th birthday of your own grandchild, and then you have to imagine what would they give. So you already did that for yourself. So it's easier to do that for your own grandchild. So you create a connection. And then at the end, with the great, great grand... It must be very confusing for the listener, by the way. Then you ask yourself, what would you give the grandchild of your grandchild? So you can create this connection, perhaps. So those were strategies that I at least tried. The fifth generation was also still the hardest one. I think I will rewrite it pretty rigidly if I find a chance. It works now. But we're also pioneering a new storytelling form. And it's really prompt art, I think. You know, like you ask the audience to do this, to do this, to do this. And I think you had a really nice name for it. You go from a mental mode to another mode. And I basically asked the audience to go back and forth three modes quite often and quite intensely. And I think we hit... the barrier of what you can ask of an audience so I think I have to make that more simple whereas for example I give one instruction which you can repeat several times something like that that would would make the piece a little bit easier also for the audience
[00:17:28.407] Kent Bye: Yeah, because there is the embodied interactions of having these encounters where you get matched up with someone that you're making a virtual child. And then it goes from 2 to 4 to 8 to 16 to 32, 64. You essentially are having these group dynamics and different phases. And I will say at the end that there was this sense of like, wow, you know, this is kind of an interesting embodied experience of like how many people it actually has taken to create me, you know, like all of my ancestors from way far back. But there was a sense of like being a part of a group and then projecting out into like this virtual child that we had all created. And then me having some judgments around that child's like, oh, really, that's the best we could create. Kind of like, it was like, I was a little, like, judgy of my great... When you were at Parents? Great, great, great, great, great channel. But it was like, it's sort of like, wow, we took all these people and it kind of, I don't know, it just looked like someone who was like a little bit of a surfer, hipster, like someone like, I don't know, I just did some judgments. So there was that kind of like judgment, but also like this reflection of... my own lineage, but also being in this speculative space, meditating on my own ancestral lineage, but also like projecting out into the future and trying to imagine the future for this virtual great, great, great, great grandchild that we had just created. And so it's a piece that there is this embodied interaction, but there's also a lot of social interaction, a lot of reading at different parts. I think, yeah, I feel like it's a piece that is at its core a speculative piece, but also a piece that is having you connect to other people as well. So there's the group dynamics, but also the ideas that you're sharing. You're kind of going back and forth between those two.
[00:19:08.933] Steye Hallema: Yeah, it really has two goals, I guess. It's really like making you think of the future. So that means also realizing that you are the future for people like six generations ago and at the same time creating that connection. And when that was clear and we started doing the first test, I also realized that I had to make it somewhat activistic, because if we go speculative fiction, what do we really want in the future? If I make people think about the future, I want to go beyond flying cars and sleek looking buildings that no one wants to live in. So at a certain point I realized, oh, this got to in some way be about climate change or social justice and stuff like that, which I realized actually I'm super idealistic. But I'm not really an activist, you know, I really like to go around that like an artist, you know, just make stuff that's inspiring. And then I really had to find a new tone in that. And I think we succeeded. By the way, I'm the lead writer, but the co-writer Shay Elmore has a really big voice in this as well. It's really finding a way to... We really like to keep it silly, and that really works in interactive works, because if you make it silly, then people don't take themselves so serious, so they give in easier. We couldn't always be silly, or we had to be... So it was really like, I think... People that try to come up with this new craft of telling stories led by your phone with groups of people. I really feel we really grew in our language as well to do that as good as possible because I don't know if we can speak about right yet. It's so new.
[00:20:52.435] Kent Bye: Yeah, when I think about this piece, it's doing a lot of things, like you said, like the social dynamics. There's like the ideas that you're talking about, the ancestors, there's the future. But there's also like, I think near the beginning, you know, there's this idea that our identities connect to a context. And then when we're in that context, then we express different parts of ourselves. And so part of the early beginnings of designing your child. What traits do you have of your child? And that's a way of thinking around what your positive or negative traits that you'd like to see carried forth, but also an expression of your own identity at that point where you're like a one-on-one expression of your identity. And then as you go out, it becomes less about identity and more around the collective ideas of both the future and ancestors. So as we're thinking about this journey from starting from like the intimate one on one, how do you connect people sharing details? At some point it doesn't become feasible to stay on that track of like sharing personal identity where you're too big of groups. And so then it switches into more of like, are we connected to our grandchildren or not? So there's connection, disconnection, dialectic, also the future. And, you know, maybe you could just kind of elaborate of how you were conceptualizing the different chapters and phases and the overall arc and journey that you wanted to take people on.
[00:22:08.985] Steye Hallema: Yeah, so doing the first test we discovered early on like the first round that you with a random stranger have a child is really good. Like people really like it because it creates a sort of connection that you wouldn't have this easily with someone. There's something it's touching upon that really makes it easier. So the feedback on that was really good and we just kept it like okay you both have a child like make the best of that. Then with the grandparents, you know, you try stuff out and then like it was really nice to go back to your own grandparents to understand what it is to be a grandparent, to bring that into the mix. Because not everyone is that, you know, so that otherwise it remained hypothetical. So if I can let you feel your own grandparents, it's easier to know how it feels like a grandparent. So that felt like a good idea. And then we did feel like, oh, but then they are not talking enough about the grandchild. So we build in stuff like, OK, let's talk about your own child, like introduce your own child to the other person. So and then from there we have this question like, what do you think that both your children would come up with the name for your grandchild? So you kind of have to imagine what your child would do. So I see it like it's yeah. Wow. That was so much work. To get there. So making all these lines that make you understand what it means. And then that kind of worked. And then when we got to the great grandparents, the first test, we thought, okay, it would be cool because then we're like 100 years in the future or more to do future visions, which is like, how do people party? How do they transport? How do they do this? People really have cliche images of the future. It's really boring often, you know, like it's all the same. And maybe not in this crowd at IDFA because people think about this stuff, but in general, like so we felt like, hmm, if we want to take this activistic, it's maybe nicer to put interesting future visions in it that might expand your own image of what would be possible of the future. And then we thought, OK, but then what's the connection with the great grandchild? So we kind of created the storytelling in such a way that it was your great grandchild's time. And this happened in that time to kind of like do both, like to make you connect in any way, at least imagine that this was happening to this person. And yeah, that was kind of like what we're doing there. At a certain point we had eight questions and then we had 12 questions and people really like it. But like the show is quite long already. So at a certain point I had to cut it. I think I will, that round, I think we have a couple of really good questions. I will make it variable depending really on the context. And a show of an hour and a half, you know, we can just make this round longer and it would be adding like a speculative thing. And then when that round is done, then we go to the great great grandparents. And I just described that it was really hard to connect with that thing. But what I liked initially, it was like you had two groups. And I felt like, oh, interesting, human history, tribes, tribalism. Actually, I wanted to see if I could make the groups fight, and then Romy and Juliet, they still get a child, because this empire and that empire. And we tried stuff. At a certain point, we had a really interesting thread, which was... That prisoner's dilemma, you know, you had to kind of like help each other, but you didn't know how to help each other to get further as a kind of like a serious game that would make the audience realize that we have to work together to figure this out. But we couldn't get it past being too leading. You know, it's like it was really hard to hide that message in such a way that you would discover it. It was probably a project on its own. So we let that go. Then I thought the idea of the circles was really strong and I made it quite deep. I went so far as like, would you want to make an apology to a future person? And I actually would really like to put that in. But it also it was still too leading. So in the last round, I took everything that could be leading. I tried to take it out so you could really have your own vibe with it and make you really make your own judgment. And then I had the idea of the generational clock, which I'm not sure if it's really working, to be honest. I'm not sure if it's wise to undermine my own work in interviews, but since you are doing it thorough, I think for the greater good it's nice to be honest with you. I'm not sure if it's really working, but at least if you're standing in two circles, in two concentric circles, so you're standing in front of another person, and one circle takes one step, so you're standing in front of another person, and you imagine that it's from another generation, you kind of like... You have sort of a dance that perhaps helps you feel like you go back in time because you do a clock-like motion, but you also see there's a real human in every generation that you can look like. So that's what I'm attempting to do, that you feel like, wait a second, in the past or in the future it will be humans just like us so to make that really real and then having like the interactions having this stack of grandmother effects felt like the way to at least try to imagine something that you would connect to and whether that worked or not i think at least taking you along is maybe already really cool
[00:27:29.585] Kent Bye: Yeah, I think there was a little bit of confusion when we were doing those last phases just because the conceit was that you form a new group and then you combine with other groups and then you do something with that group. But in that final phase, you almost skipped over that group and then you combine. So it was like, oh, wait a minute. It feels like we're supposed to do something together with this group, but now we're combining with the other group. So it felt like you skipped a generation. Yeah. So that was confusing that there was that skip. But like I said, when you're in smaller one-on-one interactions, it feels like you're able to have more of an exchange. And so once you get to those big group sizes, that's basically that design problem of how do you actually like feel like you can still have an intimate connection. And by having that internal circle and external circle and rotating, you have that opportunity to have those one-on-one connections. And then at that point, it's more of like, How do you want to like explore these different themes? You know, so I guess the question is like as you're ending What do you want people to take home from this whole experience?
[00:28:27.881] Steye Hallema: Well, super hippie. I would really like people to understand. Art can give you a certain insight or a feeling that you feel like, wait a second, we're really all connected. And we got to make sure that we figure this stuff out together. That's really what I hope the audience takes away. Actually, I would really love this piece to live outside of the art, film, festival, digital storytelling world. and be adopted by the UN or something as a reconciliation tool, like bring people together. Holland has problems, America has pretty big problems. I would love if this piece could help fix these a little bit, because it really makes you understand that you are in it together. As an artist, personally, I really feel like, again, fuck art. Let's fix the world. I was really inspired by the Ontroerend Goed piece yesterday. I think he's doing what I do, but then more effective. And he also said, first I wanted to provoke, now I want to connect. And I am totally in that school.
[00:29:38.186] Kent Bye: Yeah. I'm wondering at the end, are you going to be judging your ancestors? Do you want to forgive them? I had like a gut reaction of like, I'm judging them or condemn them for their actions. Forgive them or condemn them. Condemn, forgive or thank. I mean, my first reaction was condemn, but then it was a choice as to whether or not you're going to be disconnected or connected. And for whatever reason, I chose the disconnection. And I was like, maybe that was the wrong choice because I actually want to be more connected or forgiving. And honestly, it feels more gratitude. So I don't know why I said that, but I felt like that answer that I gave was actually more disconnecting. One of the thoughts that I had as we're talking about this is that What's really neat is that in this piece you have this group of 64 people and that theoretically you could each be producing a child that there may be something in the future that connects you now that you don't realize is connecting you because of what happens in future generations. So there's something about the unexpectedness of like how this group of strangers would actually be way more intimately connected in the distant future that they don't realize yet. Yeah. i feel like that was something that was sort of missing in terms of like the take-home message in terms of how that is unpacked with seeing that we are part of the same family in the future but right now we're strangers and how do you you think i should name that because you take it out right what i do in the end conversation when the voice comes back
[00:31:07.481] Steye Hallema: You know, first you go, she's all the way in the future. And then we go back to like your own ancestors. Then you open your eyes and see that actually it's the same size of people. And then we go all the way back. So I think, I think it's all in there. And it's up to you to at a certain point do that realization. You know, like some pieces you shouldn't say them because that's up to the people themselves to at a certain point come at. Right? I think you did that realization and that's awesome. Why should I do that? You know? And I think that's also the hard part. Like if you write a script like this, you know, the script is just a list of assumptions that I think people will feel or think or and then you do them and then you do the feedback and then you get it back. And interestingly, like what I said earlier, is that at a certain point we had people were taking it on us that they weren't connected to this grand challenge. Like, wait a second. Actually, this is just truth. you know this is the truth so it's better for me to address the truth and work from that instead of keep on trying you to connect because apparently that is really hard and then use that it's hard to connect to think about that that might be helpful in connecting
[00:32:19.813] Kent Bye: Yeah, yeah. You're right in the sense that there were a lot of other things that I was taking away of those connections. And one of the things that I did want to see was kind of like a family tree, or at least, you know, like... A family tree.
[00:32:32.108] Steye Hallema: We just put it in. If you are with the great-grandparents, you see like the family tree. But it should come back more often. You're absolutely right. And also I think your feedback with that group and then you say hi and then you go to the other groups. Thanks for saying that. I think there's something not right there. I think I can just make the groups right away, the two circles, and then you can do maybe the padding on the back again, you know, like the connect. There's also a moment that you realize, hey, this is my family when everyone has to say their name. But I don't have a slight like now remain silent and then say your name. So now that's just sheer chaos. So there's, yeah, we're still like perfecting tiny little bits to fix.
[00:33:14.009] Kent Bye: Yeah, yeah, yeah. And like with all immersive projects, you're continuing to iterate and add stuff. But yeah, that was one of the things that I just, even when the very first interaction of like, because you take photos of us, because I never saw my photo or the other photo of the other person just to see how that was like kind of playing out. Because you end up being with all these strangers, and you don't know how they get connected. And if you're just doing it on a phone, obviously it's going to get a little, like, then people end up doing the thing you're trying to avoid, which is, like, they're staring at their phones, you know, rather than actually engaging with each other.
[00:33:47.661] Steye Hallema: I have to give a disclaimer for your version. Your version was the first real audience-facing one. And the beginning game took half an hour and supposed to take seven minutes. So the AI was just completely screwing up. It didn't do that before. That's when you're pioneering. I think that's important to take in like the two other shows were really smooth and then I think it skewed the balance that you're having all this interaction with all these people that are quite silly and sometimes serious but when that takes longer there's more room for the real thing and then there's also much more rest. So I'm not saying that to save myself but just because I want you to know what it's really like.
[00:34:27.068] Kent Bye: Yeah, for sure. Yeah, and finally, I'd love to hear what you think the ultimate potential of this type of immersive and interactive media might be and what it might be able to enable.
[00:34:38.243] Steye Hallema: Well, like I said in the beginning, I think smartphone orchestra-like experience, they have so much potential. And we all have a phone. It all kind of works. We're all used to it. And now we're basically always on networks mediated by companies that want to make shitloads of money. That's basically an experiment we're doing now. And I think... there's just a really big potential to play games together, to have experiences together, led by a smartphone-like device that we can use to learn how to figure stuff out together. So they can be ancestors, but it can also be a game about choosing collectively, you know, fake news. I personally, now I'm the initiative taker of the Smartphone Orcs, I'm the creative director because we live in kind of like an art film kind of world. I come up with a project, I make it look really strong in the papers. When that looks strong, then I sometimes get money to make it. Then we make it, and that's really tiring, actually. So I want to make it bigger. I want it really to become a medium. So I want Smartphone Orcs to become a label, from which I am like the label manager. You want to be the company now. Well, exactly. But the good company. Well, for example, Elvis Presley was discovered by Sam Phillips in, I think, Memphis. He had a record studio, he had a record press. And he would record local artists and then he would press records and would sell them. As the smartphone orchestra, we are the best ones to sell smartphone orchestra pieces. So what I want to do, I want to make the system in such a way that it's not consumer phasing, but at least creator to creator facing, kind of a CMS system where other thinkers, makers, storytellers, musicians can work with. And we curate makers to work with that and they create other pieces and we help them. And I'm kind of like a chief editor. We have the system that works and then we sell these pieces in all kinds of contexts because they also work in company events. Company fans actually really like what we do because everyone can join. It's fun, it's innovative and it's mostly also in one way profound or something. It's about something at least we're all dealing with in our life. So if I could expand that on an international scale, I am quite certain that we could hold up our own pants and would not be so reliant on funding. So that's what I'm doing now. And one of our coders and technicians, Amy, she's a woman, she's queer, she's a woman of color, and she's a coder. And she has a really good idea. She had an idea like, I want to safely attach an AI you know, the attachment styles and stuff. It's like a psychological thing. And I was like, wait a second, that's a great idea for a smartphone orchestra piece. Let's have that as a ritual, like safely attach this AI coming into the world and make it love us instead of like becoming this monster. And then obviously it should come this monster at a certain point and we have to fix it. And I thought like, this is a great idea. And especially with her perspective telling that. So I would really like to create more of this kind of stuff because again, I think the potential of shared experiences is huge and I think we need more shared experiences in the here and now to know that we belong together.
[00:37:55.076] Kent Bye: Great. Is there anything else that's left unsaid that you'd like to say to the broader Emerson community?
[00:38:02.149] Steye Hallema: You know, when I was a theater maker and then I was a musician and I always felt like an ugly duck and then I came to this community and they discovered that I was a beautiful swan. No, I really like this community. It's like this bunch of warm, intelligent nerds that I really like. Again, yesterday we were doing that Ontroerend Goed with all this... smart people like you and May and Amy of Enneagram and like everyone is in the room and we're at Avinash and we're figuring this stuff like then I I really feel like there's nowhere else in the world I would rather be than here figuring this out with this bunch of awesome people I hope we will have many great great great great great grandchildren in the form of wonderful projects and ideas Awesome.
[00:38:47.338] Kent Bye: Well, Steya, always a pleasure to catch up with you. I think what you're doing with the smartphone orchestra is really deeply inspired in terms of using the technology that we already have and using it in a very immersive and interactive way. And starting with music, but now really exploring these emergent social dynamics that are able to create a way that we have these encounters and interactions but are also like telling a deeper story so yeah i think it's pretty boundless and endless for where you can take it for the future it's really cool to hear that you're thinking around institutionalizing it a little bit more and formalizing it in a way that you could start to bring in other folks to start to take this as another form of medium which is like these interconnected you know synchronized smartphones that are able to cultivate these emergent social dynamics so thanks again for joining me here on the podcast to help break it all down
[00:39:34.438] Steye Hallema: Yeah, thanks. Thanks. A big pleasure as well, Kent. And let me just put in this podcast that I think it's so wonderfully valuable that you are doing this. You're like the chronicler of this scene. And I think where it is avant-garde, that's figuring something new out. And it's so, so cool that you are like this thoroughly capturing everything. So thank you for doing this. Yeah, quite welcome. Yeah, thank you.
[00:39:57.128] Kent Bye: Thanks again for listening to the Voices of VR podcast, and I really would encourage you to consider supporting the work that I'm doing here at the Voices of VR. It's been over a decade now, and I've published over 1,500 interviews, and all of them are freely available on the VoicesofVR.com website with transcripts available. This is just a huge repository of oral history, and I'd love to continue to expand out and to continue to cover what's happening in the industry. But I've also got over a thousand interviews in my backlog as well. So lots of stuff to dig into in terms of the historical development of the medium of virtual augmented reality and these different structures and forms of immersive storytelling. So please do consider becoming a member at patreon.com slash voices of VR. Thanks for listening.
