Emojiii is an immersive social experience for 20 – 200 people by the Smartphone Orchestra where people use their phone web browsers to be guided through a number of interactive games and quizzes exploring how we make meaning and communicate with emojis. Steye Hallema is the founder of the Smartphone Orchestra, which aims to “unlock new possibilities for telling stories and creating unforgettable, large-scale experiences.” Hallema identifies as a multi-hyphenate creative director, artist, storytelling, experience designer, and musician, and I had a chance to catch up with him after experiencing Emojiii to get the origin story of the Smartphone Orchestra, how it evolved from distributed spatial musical experiments into exploring group social dynamics and ice breaker experiments that are creating new rituals for connecting.
Emojiii from Smartphone Orchestra on Vimeo.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the potentials of immersive storytelling and the future of spatial computing. You can support the podcast at patreon.com slash voices of VR. So continuing on my series of different experiences from Tribeca Immersive 2023, today's episode is about Emoji, which is from the Smartphone Orchestra. So the Smartphone Orchestra is by Steya Halima, who started to bring people together to explore different musical explorations. So using the speakers on people's smartphones and all synchronized to be able to create these musical compositions. And then at some point started to collaborate with Enneagram, supported by Kasper Sonnen of DocLab, to start to explore some of the other social dynamic potentials that you can start to do when you have a bunch of people in the same room with their phones and what kind of interactive social dynamics they can start to play around with. This piece called Emoji with Three I's, Emoji, was a piece that was using emojis as a way of communicating what your favorite emojis are, or having little charade games with people, or creating different clusters, and just a lot of different interactive ways. It's sort of like an icebreaker event. You're getting anywhere from 50 to 100 people together, all having this synchronized experience by using their phone web browsers and interacting with each other and exploring different stories and narratives around how we communicate with emojis and exploring this alternative language in the context of how do you start to create these different moments of cohesion and emergent dynamics with people. So that's what we're covering on today's episode of the Ways to Severe Our Podcast. So this interview with Steya happened on Sunday, June 11th, 2023 at Tribeca Immersive in New York City, New York. So with that, let's go ahead and dive right in.
[00:01:56.049] Steye Hallema: So I'm Stijl. I think my job is called a creative director or an artist. And I would say my niche as a creator, director, is creating stories that are about the person that is undergoing the story, which essentially is an experience. So then I would be an experience designer. But that doesn't really cover it as well. So it's all these words together.
[00:02:21.363] Kent Bye: And so maybe you could give a bit more context as to your background and your journey into doing this intersection of experiential design, creative art making, and with the Smartphone Orchestra.
[00:02:30.876] Steye Hallema: Sure, sure. So, if I start with the start, my father is a magician, my mother was an art teacher, so it's a very creative family. Doing creative stuff was always, as a kid, my identity, I think. I was a nerd in primary school, but then I had some status because I could play guitar and draw and stuff. And that kind of informs the rest of your life, in a way, I think. So eventually I went to art school and I really was into music. So I was always playing guitar and at a certain point I got into electronic music. I bought a sampler, started making beats and stuff. And then I got into art school and tried to work on stuff that I thought was inspiring. Like very broad, the art school I did was A combination of art school and the conservatory. So I studied music and art at the same time and to find ways to combine these. Not per se music and image, but like sound and image or smell and image. It was like super multidisciplinary. One of the earlier educations that really did that, at least in Holland. And at the same time I was part of a group called Pipslab. And we were basically just these boys from a village close to Amsterdam that couldn't play soccer. So we had to find other ways to impress girls. And that was being creative. And that became kind of a really weird, quirky, multimedia theatre collective. which we kind of hit on a niche. And this was like 90s, beginning 2000s. One of us was doing interaction design. We kind of were maybe one of the first theatre groups that put a computer on stage. So we had a couple of dogmas, like for example we wanted to unsheep the audience already in, I don't know, 99, 2001, 2003. like finding ways to involve the audiences in ways. And also we had a dogma that every image that you see projected on stage should be made on stage. And there were intuitions, dramaturgical intuitions to make it something that actually really happening. And because when the computer came, you know, it gave us so much freedom, but it's also quite easy to make something slick that just runs. And especially film is so engaging, you know, you just look at it and you don't look what's happening on stage anymore. So that was happening in my life and in parallel I became a musician. My beats became songs and all of a sudden people were responding to the songs. I got invited to put my record out at a label and it was put out and I did a lot of music. And trying to sell music is super difficult and it's probably even more difficult now because you compete with A lot of people that are able to make music now because you have all these tools. You compete through the growth in distribution media. You compete with all the music that's ever made in the world. So it's actually really hard. So you have to try hard to sell it. And the ruses I came up with to sell it were actually kind of mostly digital weird jokes because we were doing that with Pipslab, the theater group I was in. And at a certain point I realized, like, the digital tricks I'm doing, they are more successful, or at least I reach more people than the music, because it's so novel. For example, I made a music video, a VR video in 2014, where I do transitions with a hula hoop, which was, I think, a great find, because it was a transition in space, but also very much involving the player, the dance spectator, because there's a 360 video. And that video was like a worldwide hit, you know, while my label was struggling to get interviews. So at a certain point I realized like, oh, this is too difficult. And actually I didn't even think it, that music video was so successful that I all of a sudden was like a very successful creative director flying around the world creating VR, like riding that 2015-2016 VR wave and creating lots of bullshit, which I had fun with.
[00:06:32.303] Kent Bye: So how did the smartphone orchestra come about then?
[00:06:35.645] Steye Hallema: Smartphone Orchestra was also one of these ruses to sell my music. It is a bit bigger than that, I think. As a musician, it's actually very institutionalized, how you sell yourself. Like, you have to be this interesting character that can play music, something really well. Like, it's all about you. And trying to sell that music, at a certain point I got so fed up with that, being institutionalized in every marketing thing that it should be about me, that I got really fed up with that. I also had quite a serious disease for a very short time before I made it up and I remember sitting in the hospital and seeing all of a sudden I was like, wait a second, everyone is going to die. We're like, this is it. So that was kind of like an insight that did something to what I found important to share in life or something. And I felt like sharing life here now with everyone is maybe the only thing that really matters or something. Plus, it was a cool ruse to see if I could sell myself again as an artist. And then, a friend of mine, he's a set designer, and he made this very interesting and cool set. It was basically 40 helium balloons with speakers underneath, just as kind of a set. But he wasn't a sound guy at all, so I saw it and I said like, hey, I can make music for this. Because I just made music and sound for Haunted House, and I had this 10-speaker setup. So I put the speakers in my studio, connected them to my 10-speaker setup, and started making music for that. And it was so cool to do, because it was also freeing up for another institutionalization, the song. Like mixing with two speakers always in stereo. I had 40 speakers, I would record all kinds of stuff, distribute it in the room, walk through it, leave it on the whole night, lock my studio, come back the next morning and walk into my creation instead of having to listen to that same old speakers all the time. So that was very inspiring. And then I took the train one day from Amsterdam to Utrecht. That was 2014, so everyone's looking at their phones now. This was like a new phenomenon. And then I realized, wait a second, there's a speaker in that. And through the internet we must be able to synchronize this phone. So I could make something that I'm trying to create in my studio with phones. And that was just such an eureka. You can sleep and you really want to pursue it and do it. I got some money from an art fund in Amsterdam. I found some people that wanted to help me and we started to build the first prototype. It was entirely a musical idea and that's when we started.
[00:09:08.445] Kent Bye: Okay, so yeah, I had a chance to see your latest smartphone orchestra orchestration, or what do you call it, composition or experience? What do you usually call it?
[00:09:16.490] Steye Hallema: I call it experience now, because I think what was really important for the smartphone orchestra is that at a certain point, a much bigger idea than just having a lot of sounds coming from lots of places emerged, and that is that you can tell stories with the audience instead of to them. that we have a tool to really involve groups of people. And I think that idea is very big, and I don't think a lot of people see it yet, because we are social creatures, we want to do stuff together, and we all have phones, and we will have something like a phone for probably the rest of history, that hopefully will last long. So there's this potential to do games, to have experience, to meet each other, and that's something that we're trying now. So Emoji, I think it's an experience, it's a group experience, an interactive group experience, that's about right. How would you call it? Like, you have all this experience interviewing people.
[00:10:06.154] Kent Bye: Yeah, I definitely think that it's a social dynamic experience at the core, because you have everybody with their phones. There's an element of emoji, which is about language and communication. And so it's all the ways that people have their own association with these emojis. But you're creating these dynamic, emergent social interactions with people clustering up with each other, finding in-group, out-group dynamics based upon different meanings of emojis or favorite emojis. But then there's a lot of pairing off. So there's like a one-on-one experiences where you have a little bit more of a dialectic conversation between people that emerges. And so there's different phases of it. But at the core, there is music that happens that there are certain moments while seeing the experience that everybody's phone synchronized at the same time. And so being immersed into this course of spatialized audio from these cell phone speakers, which are not like high fidelity like you would see in a theater, but there's somewhere between 75 to 100 people that were in that room all at the same time. You get this sense of deep immersion into like this sound field. And sometimes all the sound field was the same. Sometimes it was playing different sounds. So it was just like this exploration of both the social dynamics, but also the musical element. But for me, the core of it was interacting with other people and the different emergent social dynamics for people that I either knew from the industry already or people that I didn't know. And so there is revealing of certain aspects of their character based upon what emojis they are associated with or what they were feeling in the moment, and also an opportunity to connect with complete strangers. So there's a lot of emergent dynamics, I'd say, in the piece. Yeah, I don't know. Maybe we can take a step back for a smartphone orchestra and say, what number piece is this that you've created here with Emoji? And when did you start to create some of these different social experiments with music?
[00:11:50.296] Steye Hallema: Well, I would say there's categories of pieces. So I think we created three real musical pieces, like the smartphone orchestra based on a musical idea. Then we're in the phase of creating our fifth smartphone orchestra experience with an orchestra, actually. I would really like to elaborate later about what we're doing with the score with the Metropole Orchestra. So that's five and then this is kind of like our third category. Interactive work. We made the Circle of Truth, the Truth Analytics and now this. So this is our fourth group experience piece. So that makes a total of seven plus five, twelve pieces now.
[00:12:31.735] Kent Bye: So you have 12 that you've finished, and you might be working on some other ones. But there's a number of different genres. So you have the social dynamics and music. What was the third one?
[00:12:39.261] Steye Hallema: Well, basically, there's music. Then there's us with orchestras, which is in the middle. And then there's real interactive group experiences. And that's what we're focusing mainly on now. So doing it with an orchestra is quite difficult. And we're creating a really cool new piece. But that's really a side track. We mainly focus on interactive group experiences that bond people.
[00:12:59.970] Kent Bye: So yeah, I wanted to ask, because you started with the music, and I can see the roots of being a musician, the types of experiments that you were doing with the piece. And there were some musical moments that stuck out for me as being like a unique musical experience. But primarily, the center of gravity for me was the social dynamics that were emerging. And so at what point did you realize that there was some interesting social things that you could start to play with, with the smartphone orchestra?
[00:13:22.559] Steye Hallema: So the first one we did, I think for real, was work that we created together with Enneagram. I was always talking to Casper Sonne, the head of content, I think, from ITVA, who is a very visionary guy. And I was always talking to him about the potential of the Smartphone Orchestra and to make it interactive. He was in this program of coupling up artists, so we worked together with May and Amy from Enneagram. We were together really exploring, like, okay, if we give the audience assignments, what happens then? That was hugely successful and cool. We've played that piece many times. And we basically gave people the assignment to sing, to make words. At a certain point, they all have to sing kind of in a different rhythm that combines together. And then we work towards that they learn a word. And that word is actually a sentence. And at the end, they have to put away their phones and they have to say the sentence, which is super easy. But that's basically the plot. We're so dependent on our phones that even doing something simple as that, we almost struggle to figure it out. Which I really like, because the plot is not a story plot, it's an interactive plot. It's really making you aware, like, wait a second, are we able to do this? Which is really what I'm out to discover, I think. Also with Emoji, I think we took a big step in creating a way of telling the story that it's very loose, but still, it's engaging, but still loose. That's a design challenge we were trying to solve because if I guide you by the phone, I need your attention on the phone. But then kind of like my goal is to bring you in contact with someone else because you could say the smartphone is a form of contact art. And then you are talking, so how do I bring you back to the phone? And I think that's a craft that we're slowly trying to learn to do that as so clear and forgiving that even if you miss, you can still continue and it kind of works. So yeah, I think the real beauty is in the chaos and creating organized chaos.
[00:15:22.416] Kent Bye: Yeah, I really appreciated how the fact that emojis are a form of communication that people use and sometimes there's different meanings of emojis that people have or people have their favorite emojis. And so what was the moment that was a catalyst for you to focus on emojis as the primary mode of communication and interaction in this piece called Emoji?
[00:15:44.349] Steye Hallema: Emoji came mostly, I don't start before I feel like I have like a story or an idea kind of with really perfectly fits a form. And Emoji was, it was actually just an idea we had when we're trying to make another piece. We were asked by Lauren, who was then the curator of Tribeca Immersive, to create a piece for Tribeca. But then it was 21 and we couldn't get into the country, so we had to abandon it. And one of the ideas we were working on was an emoji, actually. It was just an idea of engaging people. I have one emoji on my phone screen and you have four. And then you have to guess which emoji I'm mimicking with my face, because I'm trying to do the emoji that's on my phone. And then it couldn't happen and we were all super disappointed. And then we felt like, but we should at least make a good prototype out of this idea. And we did and it was fun. And then I kind of had to find what we really wanted to say with it later. Which is not how I work normally. Normally I really have that form and structure very tied together before I start. So then you dive into emoji and you realize that there's eight companies coming up with what emojis we're gonna talk with. Which is also weird and I think something we should be aware of that there's companies basically creating the language that we are communicating in. So I thought that's a strong point. So I think most of our works that we do at Smartphone Orchestra are kind of like technology critical but then in a silly way because we have to be silly because otherwise we don't engage people. So that's, I think, a layer that was important for us to do. It's also the exploration, like, how can we get people together? How can we make them meet in a way that's meaningful if we have the chance? Because we were discovering during all these Smartphone Orchestra pieces that people actually really appreciate this. that being this icebreaker that connects all these people out of socially accepted frames that we all are functioning in. We can break them out and you can connect in a different way. I think one of the motivations for the piece is also that I as an artist at this moment feel that Stuff we really need right now is stuff that makes us feel that we belong together as humans as much as possible. Whereas in the last century we were really defining what art was. I think that is luckily long gone. Then art often is like people really having this drive to express themselves for many motivations. And I think good art could also be bringing people together right now, finding rituals to connect. Especially in the technological realm, where now it's so clear that technology is so able to drive us apart. We need to counter that.
[00:18:25.590] Kent Bye: You mentioned the icebreak component of this piece, where you have people choose your favorite emoji, or you provide a question, and then people are answering in an emoji, and then they're showing it to people around them. And it's a way of seeing how you have these emergent connections with people. So I'd love to hear how you think about the journey that you're taking people on with connecting people, and then having these games, and then what the overall arc is.
[00:18:51.249] Steye Hallema: Good question. So it started with the game, but then we felt like people needed to... This was very much an intuitive process, I realize now. So at first it's kind of a quiz, you know, like what is appropriate, which is the right emoji, what would you choose in this, and you do that very collectively. That always feels like a very strong start, you know, that it's not individual straight away, that you feel like, hey, wait a second, we are a group. That was important. And then you do the story thing, so you form a group and you have to, like, fantasize a little bit. it felt needed to put a little bit of creativity and like your own personality into it as a way to easily open up because otherwise the game would be too much a game and there's not enough human connection yet I guess. Because that's really what we're exploring as well. How can we really make people connect in a good way? Which is also hard. There's this design challenge, how do you make a shoe that fits everyone? So I can be this hippie that's super extrovert and wants to meet everyone, but some people are not. So you have to tailor to that as well. So the arc of the piece is the quiz, the collective quiz. Then it's the story where you become a group which is nice because it becomes a little bit more like hey we're together here. It creates also like you can talk about like your relationship with emojis which also gets you more invested. and then you do the game, which is kind of time pressured, and it's fun, and people make funny faces to each other. And then we try to, in between, also create moments of connecting, but we also try to keep that real, but also really emoji-connected, which was a challenge, because I think we came up with the term while working with Anagram, and it's a term we call talk to the moment. It's like if you do something interactive, for example in VR, you are still someone with a headset on standing or sitting in a room somewhere. That's always like the base relationship. I think stuff is good when you take that really much into account and you don't let you be too much indoctrinated in like how film can really transport you somewhere else and you completely forget yourself. because you're always present. Acknowledging your presence in the realist way is very important. So, a very simple example, most smartphone orchestra pieces start with something in the vein of, here you are, staring at your smartphone again, or staring at your phone again, because you realise, actually that's what I'm doing, and that's kind of keeping it real. So we're also with questions that we want to make people connect we try to keep it close to what you're actually doing and that's that sort of Very severe and strict towards each other to keep form and function like as close as possible Because otherwise it doesn't make sense and there's other stuff you can tell stories with like film or books or podcasts
[00:21:39.410] Kent Bye: Yeah, I really appreciated the way that it felt like at the end of the experience I felt like I was able to feel more connected to the group but also to get into this altered state of consciousness by jumping around and there's this embodied interactions that you have people do that are kind of silly like jump up and down and say a word and it's near the end where you have this rapid cycling of what ended up being a bit of a cacophony. There's other people that were not fully synchronized, but there was pockets of people that were stepping through these series of actions that created this chaos in some ways. But, you know, overall, the experience that I had at least was that by using the phones, you were able to have us have it as a mode of communicating with other people, but also to connect to other people through these games and this whole arc, and that by the end of it, it just felt like a really fun, light, joyful experience that did make me feel more connected, and it was a great way to start the festival. So, I feel like this is a type of experience that would be a great kickoff to the beginning of some group of people that are coming together.
[00:22:41.088] Steye Hallema: Yeah, I think that's also why we have kind of like a business model potential. We're struggling getting it off the ground now at this moment because it's actually a lot of work to do all the bookings and make sure that goes well. But this is like we're playing a lot at corporate events and stuff that people go to because people come together. And again, that proves my vision, if you will, that this is a huge thing. People want to come together and do stuff. And with devices we have, we can guide them in completely new ways. It's huge.
[00:23:09.600] Kent Bye: Just a technical question. It's shown on the web browser. How do you synchronize everything? Because it seemed to be pretty synced across the music and everything. And I know that can't always be an easy thing to do. So how do you keep track of time on these browsers?
[00:23:22.825] Steye Hallema: This is not so hard, this one. This is not synchronized, actually. It's just the prompts are sent out at the same time. That's it. When we did the musical pieces, we went to great lengths of creating an algorithm that would measure everyone's phone's moment at that time and then go back and forth and adjust the time and then synchronize them beautifully. Which was also a design challenge because at a certain point we realized, oh wait a second, all the iPhones, they are tight because they have the same hardware and software. So you can really trust them with what they will do. And then the Androids, because there's different manufacturers of Androids, they're all over the place. So we started to compose music in a way like the iPhones would do the percussion and the stuff that needed to be tight, and the Androids do all like the pads and every stuff that doesn't matter if it starts phasing. So that was actually fun because that gave you a really like, what am I doing? I can only do this, so it's always easier to create like that.
[00:24:16.475] Kent Bye: So what's next for you for what type of things you want to explore, what kind of themes or stories that you want to tell with a smartphone orchestra or other art that you want to create in the future?
[00:24:25.395] Steye Hallema: Well, a piece that I'm super enthusiastic about, we just made the first prototype, in the face of we just applied for some money, we'll apply for more soon, is a piece called Ancestors. And Ancestors is a Smartphone Orchestra piece, so you will get into the venue, you'll be asked through your phone to make a selfie, either we make a selfie or you're still figuring it out, and then you stand in a circle, you're in a group of like roughly 32 people, might be more, might be a little bit less and then you hold up your phone and a voice comes from all the phones and that's a female voice and she's from the future and she has something very important to tell you and then a photo appears on your phone and this is for example if I'm there and you're there I see you on my phone and you see me and we have to find each other and then we have this short moment after finding each other of sort of dance, which is actually mating ritual, but we don't know that yet. And then all of a sudden, ping, a new photo appears on our phone and it's our child. Because with the algorithm we created a mix of our faces. And this is actually like, I don't know, how old are you? I'm 46. Okay, so this will be like a roughly 46 year old man, and we have to decide a name for this man. And then we have to think about what will his job be in the future. So we think about the future, but we also get to know each other through these questions. And after like five minutes and a couple of questions or so, another photo appears on our phones. And we have to find two other people with the same picture on their phones. Like for example, these two people standing there. And we find them and they have the same picture. So there's four phones with the same picture and this is our grandchild. So we're going to have a conversation. What do we want for a grandchild? And so we gradually create one big family out of all the audience members, which is actually the family of the descendant that was talking to us in the beginning. And we tried this out and it was really wonderful how people also, again, through these faces and this hypothetical idea of having a child together, you connect in a really different way, in an original way. And also creates this feeling like, wait a second, we are sharing life here and now and this is it. And actually we're also, everything we do relates to the future. Like, it's actually quite moralistic. But yeah, we need that now. And if we find silly and fun and original ways to make people aware that we've got to fix this, then I think I'm not wasting my time.
[00:26:46.560] Kent Bye: Sounds really fun and amazing. I can't wait to try that out as well. Yeah, I guess as we wrap up, I'm curious what you think the ultimate potential of these different types of immersive media might be and what they might be able to enable.
[00:26:59.400] Steye Hallema: Well, every technology is always a double-edged sword, so I don't think that smartphone architects, it's hard to use it in a bad way. So, for example, one thing I think it can enable, and that's basically what you do, you bring a group of people together in unity of space and time. And then you put this digital network between them and then actually because you put this digital network between you can start seeing actually what happens really between us where we are mediated by a party that might have different stakes than us, which happens on every digital platform. Like we are mediated by people, by companies with different stakes than ourselves. That's the experiment what we're doing with society at this moment, which is huge. And I think the smartphone also can be a tool to figure this out in a better way, because a group of people together is the reference network, we add the digital network and we can start testing what happens here for real. So I think it can be a tool like that. Then I think it can just be a tool to have fun, you know, this could be... Like, I wouldn't mind if at a certain Mattel would pay a billion to what we created and create the next hit games in the future. It could happen. I do really believe in this potential and it's kind of fun that hardly anyone sees it. I hope I will be as far that I could make this exit before people start stealing the idea.
[00:28:24.458] Kent Bye: Awesome. Is there anything else that's left unsaid that you'd like to say to the broader Immersive community?
[00:28:30.285] Steye Hallema: To the brother XR community, well, I just really love our community. I've been a musician and a musician is kind of a scene kind of thing too. And when I got into this scene a couple of years ago, it felt like There's so many brilliant and fun, loving and warm and intelligent and creative people that I just... I love us. And we should grow. I don't know, it's like, yeah, this would be my last thing. See, I just try to create people together and have fun. That's what I do.
[00:29:04.673] Kent Bye: Cool, well Steja, thanks so much for creating the Smartphone Orchestra and all these different social dynamic experiments with music. It's a great way to start the Tribeca experience, I think, with this one of the first pieces that I saw, and just a great way to bring people together, and yeah, just had a great time seeing it all.
[00:29:18.657] Steye Hallema: I'd love to talk with you more, Kent. Not now, because I have to prepare a performance. But I think one of the reasons I love doing this so much is the dramaturgy of it. Figuring out the dramaturgy, how do we make this stuff work? That's why I love VR, AR and Smartphone Orchestra, which is also XR in my sense, because it's immersive, it's involving you. I would really like to have a deeper conversation about dramaturgy, because I think you should have a lot of knowledge.
[00:29:44.721] Kent Bye: I think there are certain moments, like at the very beginning, there's a moment when you have, like, where are you at? And you have, like, a heartbreak, and you have people dancing. And so it's interesting to think about what happens to the people who are in a place of heartbreak. Do they get a different experience? It didn't seem like there was any tuning to that, but that'd be an interesting thing.
[00:30:03.478] Steye Hallema: Totally. We are one eye in a world of blind people. We're trying to get that one eye open. There's so much potential. We create this and again you have this idea. And I think it relates to an idea which I don't think is a real idea yet. I call it mood conductor at this moment. Imagine we could go into a venue and everyone brings his mood in a way. So you have a set of questions that's measuring your mood. and then that mood kind of controls the music that what the orchestra is playing or the DJ is playing or the AI is playing and then that will change us and then we have this feedback loop between how our mood was and then the music is changing us until we maybe have some moment of that like wait a second we are kind of feeling this some sort of thing so it's an idea it's a thought it's not an idea yet I would say but but it's in the same vein like sharing what we bring I hear a lot of co-creators talk about rituals, about shamanism, about healthcare, and I think that is a natural evolution of what we are doing, because if we create work that is about the person that's undergoing or experiencing the work, we all ask ourselves, what can we add to this person? And then you kind of get into shamanism and helping someone, and I think that's very interesting. It could become really pretentious as well, Yeah, I don't know. There's a lot of interesting ground to explore there as well.
[00:31:27.853] Kent Bye: Yeah, I think the main thing I'd say is what are the emergent potentialities to create the perfect experience for people for wherever they're at, which is difficult when you have a big group. Because right now it feels like Emoji has got a certain tone and that the tone is pretty consistent. regardless of whatever mood you're coming in with. And so you lose agency as a creator for what story is being told, but are there ways to modulate people's experiences in a way that creates emergent dynamics between them that is novel and unique and perfect for that moment, and it's different every time? So that's sort of my thoughts of where you're at now and where it could go in the future. Yeah. We can certainly have a chat more about what that means later. But I know you have a performance to go get ready for. And I've got some other interviews I'm going to go run off to. But I just really appreciate that. Yeah. So.
[00:32:13.384] Steye Hallema: I would really love that. Yeah. Let's do that. Yeah.
[00:32:16.929] Kent Bye: Awesome. Well, thanks so much for joining me here on the podcast. So thank you.
[00:32:19.312] Steye Hallema: You're very welcome. And thanks for having me.
[00:32:21.797] Kent Bye: So that was Steya Halima. He's the creative director and artist, and he's creating stories about people who are undergoing the story, which is experiences. So he's an experienced designer and storyteller and a musician and the founder of the Smartphone Orchestra. So I've had a number of takeaways about this interview is that first of all, So this experience was a lot of fun exploring the kind of emergent dynamics with emojis as a form of language and allowing people to connect to each other and having little games where people are picking their favorite emojis. So you'd pick emoji and then you would show your screen to the people around you to show what your selection was and then you would connect people with each other by having this mechanic where You have a certain emoji on the screen and you have to find someone else with that emoji on the screen and there's sound effects They're happening all the same time You have charades where you have one emoji on one side and then you have an option of four emojis where people have to communicate with each other and then just starting to explore different kind of group dynamics and synchrony at the end and and Overall, it just felt like an experience that was a lot of fun. It was light-hearted. Also allowed to create these emergent in-group, out-group. Almost like using emojis as a way of expressing your character or your mood at the moment or your temperament as a person. And yeah, just the way that we use emojis generally to be able to communicate with each other. There was this really interesting moment at the beginning where it was asking you like, what kind of mood are you in? Are you in a dance emoji mood? Are you in a heartbreak type of mood? And it felt like the tone of the experience didn't really afford for people who may have not been in the lighthearted party favor icebreaker type of mood. I think it would be interesting to see how you start to look at different ways that people maybe feel a bit more. extroverted or introverted, or maybe they feel sad or happy, you know, if there's different ways of clustering people together, or to take people down a customized journey to pair people up who maybe are in the same mood or different moods, just a lot of opportunities to customize the experience to a lot of different types of experience that you could do within a social dynamic. So I feel like there's a lot of potential there to see how you can have these type of emergent dynamics with people because there's a lot of chance operations that are happening here, but also people may already know each other or may not know anybody in the room. And so that's kind of an activity that allows people to both connect to each other and express themselves, but also to play around with other folks. And so this icebreaker framing of an experience like this, I think is very well suited, but also has a lot of potential, I think, to go in a lot of different types of directions for how to explore these types of emergent social dynamics. So yeah, and there's a whole other aspect of the music, which I feel like this piece, like Steya said, there wasn't any synchronized music that's happening. It's all just stuff that's being pushed out from the server. There's also a lot of stuff where you have to emergently connect to people in order to connect to them. Then you have to turn on your phone and then shoot a QR code, and then it matches you up in a dyad exchange that's happening in a larger group context of hundreds of people. So being able to create these clusters of four people or a couple of people and big group experiences. So yeah, I feel like there's a lot of potentials for where this could go in the future. And also just the sound dimension of it, the spatial audio that happens with these sound effects that are happening all around you. So like I said this piece was much less focused on that spatialized audio experience from people's phones but much more into the emergent social dynamics. So that's all I have for today and I just wanted to thank you for listening to the Voices of VR podcast and if you enjoyed the podcast, and please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listeners-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voicestvr. Thanks for listening.
