I interviewed Impulse (Preview) co-directors May Abdalla & Barry Gene Murphy remotely after the SXSW XR Experience 2024. See more context in the rough transcript below.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So continuing on my series of looking at different immersive stories that are being featured at South by Southwest 2024, today's episode is with a piece called Impulse, which is a continuation of the Playing with Reality series by Anagram and directed by May Abdalla and Barry Jean Murphy. So while the previous piece of Goliath was focusing on the mental health condition of schizophrenia, Impulse is focusing on ADHD. So Attention Deficit Hyperactivity Disorder. And so this is actually a preview and so it's not the full experience yet, but you're thrown in the middle of this piece where you're exploring these four different stories of folks who are dealing with ADHD. and in a mixed reality context and so it's a tabletop scale and you're seeing these different characters move around these different spatial contexts but it's kind of a mixed reality context and so you also see your environment around you. It's being shown there at South by Southwest but since I wasn't able to attend this year I was seeing it at home and so it's shown with the context of my living room. And so we talk a little bit more about their plans for where they're taking this in the future, but also starting to explore some of the different affordances of mixed reality in the context of telling the story of ADHD. So that's what we're covering on today's episode of the WSIS VR podcast. So this interview with May and Barry happened on Thursday, March 21st, 2024. So with that, let's go ahead and dive right in.
[00:01:42.073] Barry Gene Murphy: Hi, I'm Barry and I'm a director with Anagram. Together with May, we make VR experiences to kind of focus around social issues about, I guess, mental health and how the brain works.
[00:01:59.164] May Abdalla: And I'm May, and yeah, I started a studio called Anagram ten years ago to explore storytelling and immersive mediums, particularly looking at how interactivity can make storytelling more meaningful. And yes, together with Barry, we worked on Impulse.
[00:02:18.615] Kent Bye: Nice. And each of you could give a bit more context as to your background and your journey into working with VR.
[00:02:24.629] May Abdalla: So yeah, it's interesting. I guess initially I was really interested in documentaries and I think always like really fascinated by like the long form non-fiction genres and worked in television and film for about 10 years. And in a way I think I was just a bit frustrated, you know, always trying to get kind of through the screen, trying to make things feel more real. And it's not like I think VR makes things feel real, but I'm not one of those people. But I guess I got excited at the time I was working at the BBC. There was a lot of experimentation around interactivity and storytelling. It was a really fertile era because they had lots of money to invest in experiments. And then actually they closed all the experiments down and decided to launch iPlayer as the kind of thing that was actually what new media film was going to be. But on the journey there, there was loads of stuff around games and documentary. I kind of like the idea of ARGs and what happens when it feels real to you without necessarily being trite. How do you make it the kind of authenticity of the moment, also something that the user would feel as well as the story? Because there's just so much around the philosophy of documentary and how we tell other people's stories and whether that's actually, in a way, the politics of viewing other people. Especially working at the BBC, where I was working a lot of international affairs and thinking around Who's the person that's sitting on the sofa consuming the story of the other person? Is it something that we use to learn about ourselves ultimately, or is it something that we use to distance ourselves? It was back in 2013 that me and my friend Amy decided to bury ourselves in a little hole in Brighton and mess around with some ideas. 10 years on, there's still things I don't know how to do and still things that feel curious enough to kind of keep trying, I think.
[00:04:33.030] Barry Gene Murphy: I guess I started out in animation, really, but before that I was into engineering. So I kind of had this interest in technology, like computer, like CG and all the new stuff in that arena. But also, I had this love of animated documentary and One of my favorite animated films are animated documentaries like A is for Autism or 15th of February and all of these kind of, you know, I had an experimental animation teaching, you know, so I was really into a lot of that kind of stuff. And I guess in 2014, a friend of mine got like, uh, Oculus DK1 and That was my gateway into seeing what you could do there. I always really liked tinkering with the new tech and just trying to break it or just find its edges. I kind of got involved with May through helping her out on her Collider piece, I think. Then we made Make Noise and then we kind of just started doing Goliath and now we're onto Impulse. So yeah, it's been that kind of very organic route. I guess myself and May are both really aligned on the kind of documentary angle, like, you know, telling real stories. So yeah, I guess we're just kind of aligned on that mission to kind of tell real stories and real experiences. So that's why we're here.
[00:05:51.696] Kent Bye: Nice. And the last time we had a chance to chat was on Goliath kicking off this playing with reality series that was showing at Venice, like during the pandemic. And then also I saw you present some of the latest ideas around where you want to take this playing with reality series at Venice production bridge back in 2023. So let me get a little bit of an update for how Goliath went out there and then how it's leading to this much broader series that you're continuing on to explore different topics of mental health.
[00:06:20.339] Barry Gene Murphy: Yeah, like I remember speaking at that time and, you know, when we brought Collide to Venice, we were just, you know, worst case scenario going like we didn't realise that it was going to have like an impact or people were even going to like it. You know, we are so deeply entrenched in all of the things that we didn't get to do. And I think that after that, we were really surprised about the audience reaction and the really positive feedback and encouragement that we got. And so, you know, it's been a long road. We've been still engaged with Goliath in the sense of still touring and there's still always keen interest about it. There's a medical route now as well as a teaching mechanism to bring people into what it feels like. for doctors and stuff and tackling compassion fatigue and all of these kind of things. So Goliath still is alive, you know, and we were kind of like invited to consider doing a follow on and we wanted to just focus around like how the brain works really, you know, and we've been developing this new piece, Impulse, based around that. And that's kind of like about Attention Deficit Hyperactivity Disorder or ADHD. We followed a similar route of finding people, researching. We held 50, 60 interviews with people and tried to find a balance because apparently it's an issue that we were told very early on. We listened to the experts and they told us you cannot tell just one person's story. You have to tell it from different perspectives. So we did that. And so, yeah, we're at the phase now where we're in our downward slalom to the end, I suppose, and trying to bring it all together and make an experience that people can empathise with. I suppose that's the bottom line, is that we're trying to not be too scientific, but just to put people in a position where they can consider other people's states of mind.
[00:08:23.084] May Abdalla: Just to add to that, I guess, you know, the wider thing, just like I said, you know, we didn't have as much faith in Goliath as we discovered we could have had. And you're always kind of wondering like, what is good about this tech really? You know, like often we're just so inside like the echo chamber of immersive VR and MR that you kind of want to have as much of a critical eye being like, well, why are we still doing this? Like, where's it useful? And, you know, I don't want to be too glib about it, but it's this thing about like, I think there's something in using immersive stories to tell the experience of people whose experience you can't see. Like, I know that seems a bit trite and I know that there's lots of bad ways to do it as well. And there's just something about the sensations, the combination of you know, fantasy and reality and terror that's in psychosis that made a lot of sense in VR. And I think, you know, for various reasons, we were just talking about ADHD a lot and the idea of it's a divisive topic. It's, you know, on one hand, you've got like teachers who are irritated with more kids needing to have special treatment and it raises a lot of reactions in terms of like, is it a trend? Isn't this really generic? And at the same time seems to kind of affect people in such a huge way. And it's so kind of mind altering that it was like, it's not obviously the same thing as psychosis, but is it something again, that if you could kind of feel the non real fantastical, invisible experiences of others, would it have an impact? So I guess it was kind of after Goliath and thinking about ADHD, and then there's a few more of these up our sleeves. They were like, well, you know, let's not bail on this team. Like the team are really great that we're working with and shout out to all of the like artists and everybody. Like you go through a lot when you're trying to make something that's so intangible. Like imagine trying to explain some of this stuff in like, in team meetings, you know, it's really great when people believe in it being possible. So it felt like, yeah, it really brought in the people who were interested in this medium, you know, younger, people who play games and it just had such a broad audience, Goliath, which I think that's what surprised us because on so many levels, it's an extremely niche art project, but on one level, you know, it's gone into like six figures in terms of plays and stuff. So what does that mean? And are we building kind of like an interesting audience also for the medium and let's do more of that because what else are we going to use this headset for?
[00:11:06.395] Kent Bye: Yeah, I know that from doing a brief little audit of the different experiences that are on the quest, it's around 68 to 70% of all the experiences that are available in the quest are games, and so the rest of the 30% are split between apps and entertainment experiences, which. I think Goliath would be considered entertainment. A lot of the immersive stories end up being tagged with entertainment. So you were able to get on the store to be able to show it and get much broader audience. And then also, I know you've had a number of different showings and screenings and opportunities to show like a whole impact campaign to bring it out into the world. So I'm wondering if you could maybe recap some of the highlights from the different types of contexts you were able to take Goliath.
[00:11:47.002] May Abdalla: Yeah, so I think the biggest thing, as Barry was saying, was that it's great to win awards at film festivals, but with this stuff, when it touches people with whom a part of the community, it's really, really inspiring. And so pretty quickly after it launched, we had interest from doctors and people teaching nursing or people working in ambulance and emergency services, and they were really looking to use it as a teaching tool to encourage other people to... Even cops. Yeah, cops as well. Like, how could we use this kind of within the police? Yeah, amazing. Many, many people wrote several paragraphs around the issues, which we knew about because it was part of the reason why we made it, which was yeah, you're more likely to be shot dead in America if you have a mental health problem, like 16 times more likely. And, you know, there's people within a movement around changing culture and healthcare towards people with mental health problems. So that stuff and kind of the ongoing work we're doing with the NHS and University of Western England in Bristol, and yeah, like working to kind of launch that more formally at the end of the year. Like for me, that's like whatever it means to be an artist or whatever that is, like, That is just really good to know, given that at the very beginning of this project, you're sometimes you're talking to like really vulnerable people whose lives are like, there are many moments of like extreme bitterness and isolation. And so, yeah, raising that profile feels really good. And then on the other side, you know, the fun sides of it got adapted into a 360 immersive interactive projection for a venue in Montreal.
[00:13:30.548] Barry Gene Murphy: Oasis, yeah.
[00:13:31.715] May Abdalla: Yeah. Wow. We're so into the next project. How can you even remember?
[00:13:36.659] Kent Bye: I don't know if you had like a David Chalmers or something that you were trying.
[00:13:40.823] May Abdalla: Oh, yeah, of course. Yeah. There's a version including David Chalmers, which was on the Science Gallery, Bengaluru Indian Museum back in 2022. It's also out on Mandarin and receiving people's comments about mental health in China, especially given everything there. Yeah, it's connected to kind of something that's very alive.
[00:14:06.502] Kent Bye: Yeah. And so as you move forward with Impulse, then, you know, this, what you're showing here at South by Southwest is just a preview. It's not the full experience. And so you're kind of throwing us into the middle of the piece. So whenever I see preview projects, I'm always hesitant to dive too deeply because I know that it's going to be likely some changes or it's going to continue to evolve, but maybe you could set a little bit of the context for the beginning of the piece, because we're skipping over some of that prologue to dive into the middle of some of these different stories. And so. I know with Goliath that you were showing some of the metaphors for how to use the medium to explain some of the basics of schizophrenia in that case, but here we're exploring ADHD. And so I'd love to hear a little bit more elaboration on where you're planning on beginning with impulse to set this broader context for ADHD.
[00:14:54.301] Barry Gene Murphy: Yeah, I guess you saw the kind of the story side characters, testimony and stuff. And prior to that, we're kind of getting the user to focus. Impulse is like not focusing at all about ADHD, because ADHD has so many angles and aspects that you could go into. in loads of different ways. So we kind of just decided to focus down on the area about impulse. What drives the person to find themselves in risky situations, or what they're striving for to feel normal, almost. So we introduced the person to their brain, essentially. I'm trying to get a very embodied way. We have the user absorb information and consider how you sort information. We introduce the amygdala and the prefrontal cortex and the systems of the brain that interact with reasoning and impulse. It's a game, really. In a sense, Whereas Goliath had the game at the end in a way, because we wanted people to become gamers by the end of the experience. This very much starts as a game, you know, as it put the user under stress. And we're trying to get this reaction of like overload, you know, and how you react in that situation when you realise you don't have time, which is kind of like a persistent feature throughout the narratives that we interviewed, you know. So we kind of like put the user straight into that kind of system.
[00:16:13.354] May Abdalla: Yeah, it just felt really, you know, handy, right? Because in a game, there are all these mechanics which kind of exaggerate usefully the sensations of dopamine hits and adrenaline and the need for fulfillment and kind of crushing sense of loss and chasing the buzz and all of that stuff. It's kind of accessible in this like really neat little package of kind of Games do that. Yeah. And that's why there's 68% of games on the store. Because you want to kind of establish with the user like some language that they can then carry on, which is just like, what is that feeling when the thing that is bringing you pleasure is taken away, or you don't feel the validation. And so much in the design of games is about bringing that validation. So once we've kind of established that we've all felt this together, that's the material in which the stories of the characters can spring from.
[00:17:10.407] Kent Bye: And so when the experience starts here, the first preview that you were showing at South by Southwest, you're going into these mixed reality scenes with these four different main protagonists. And so maybe you could first talk about the choice to go with mixed reality. And, you know, I was watching it in the context of my living room. And so it was overlaid on top of what is happening in my home, but at South by Southwest, were there any other like installation components that you were trying to bring about? Was it just like a big empty space or how are you starting to see like the use of mixed reality in a way that you could start to blend and blur the physical reality with the types of kind of tabletop scale ways that you can put these different scenes and these characters in the context of a space?
[00:17:53.562] Barry Gene Murphy: Yeah, I guess that was the real challenge, really. I think that the mixed reality You know, we engaged the technology at the time and I think that it was rolling through, like, you know, his past through kind of layer. And I think that the way I was kind of conceiving it was trying to just think of things as like collage, you know. And I think that when you make a composition on 2D and stuff like that, stuff doesn't really sit like the rules of like, mixing 3D and real and stuff. They haven't really been written in that way. So there's a lot to kind of test and try out and see what works and, and give like, I guess, you know, holding one space and having another at the same time is an interesting kind of idea. And we did try to do that with Goliath with the kind of like really expansive sets in the hospital scene and stuff. But I think that we tried quite a lot. We iterated quite a lot on how to kind of bring these characters' worlds in. And it's still in flow. Like, you know, I think we're still bridging this sharing, an understanding of architectural spaces and always holding in the idea that, like, that there's another world out there, you know, that there's other spaces and other lives out there. And that's where we're aiming for that kind of concept. I don't know if there's a word for it.
[00:19:10.677] May Abdalla: It's like interstellar, that kind of thing, you know, the tesseract, there's kind of the adjacent lives. And I think initially, like for me, it did make sense. You know, we spent about three months just interviewing people. And like, as you know, from interviewing people, you know, lots of things stay with you. And people have their own ways of describing things. And After about 50 interviews, you started to just have a feeling for what the feeling was, and it felt important. I don't know if we've achieved it, but that kind of fractal thinking. With psychosis, it's quite almost simple that there's a virtual reality of things that are not there, et cetera, or the sense of that. there's kind of like everything in their space triggering more thoughts.
[00:19:59.587] Barry Gene Murphy: And the way in which kind of normal life can become overwhelming, the kind of supersonic hearing, you know, one of the brain fog or, you know, like just, yeah, I guess just the layering of, you know, I think that there is, you know, metaphorically fits with what some people describe from the condition. And
[00:20:19.907] May Abdalla: So yeah, so that's kind of like a bigger picture and then I think like what we're talking about are these scenes within your space and the juxtaposition of your... I guess I wanted to try and get people to engage with their space anyway, like the thing about the question about the South by installation, I think, weirdly, it might be more interesting to do this piece at home. Because it is about like the proximity of your clutter or lack of clutter or your, you know, what you're talking about, kind of lives that people have that you don't see. And there's something almost disturbing. I mean, at the beginning, you just kind of mixed reality is quite irritating, because you don't have any control over it doesn't look good to see your mess adjacent to some nice piece of animation. And so like, you know, Bayeux was kind of talking about collage, like how can it, and I think, yeah, like you said, you know, we just kind of struggled in many ways to get to, but we're like, we could get excited about something feeling good without it necessarily having to be designed and embracing in a way that ADHD, you know, you get whatever thought comes to you. and you know we just really want to build on that stuff as much as possible and as well as play with the medium and not think of it as
[00:21:40.113] Barry Gene Murphy: And I guess I don't know if it landed in South by Southwest either, but the kind of mechanism of you kind of like absorbing these people's kind of experiences where the release version will have like, you know, you are like essentially like building clutter, you know, the marks and scratches of these people's lives come along with you. And, you know, you are in a sense kind of like layering up your space as well. So a kind of crucial point about, I guess, one of the core kind of ideas in this is about that moment when there's a diagnosis and then there's a kind of a reckoning. Almost all of the people that we interviewed had this kind of point where the diagnosis really changed stuff for them. That was a big moment where they are allowed to forgive themselves or they saw in a kind of an externalised way. That's kind of like a really important stage for somebody with this diagnosis. They go to a kind of sense of mourning, you know, the later stage they are, like, you know, the more terrific that kind of feeling is of sense of loss, you know, of like, you know, why things could have been different and how things could have been different if they had this kind of hindsight and knowledge about themselves. So, you know, I think that we are kind of writing to that as well. We're trying to kind of summon that kind of feeling. And we use the space in the way it's kind of almost like an art installation at the end of the story to kind of like bring that home. So, you know, we didn't want to do too many portals or roofs falling open and stuff like that.
[00:23:06.902] May Abdalla: But there's always time to answer.
[00:23:08.262] Kent Bye: Yeah, we can always do that. Yeah. I feel like that, you know, there's a number of different types of conceits within VR and AR. AR tends to be a lot more tabletop scale, smaller scale, not usually one-to-one that allows you to have a little bit more expansiveness of space because you are just working within your physical limitations of a room and then I also noticed that in this piece, you were actually moving the environment around relative to the external environment, which is something I haven't seen a lot. And when I first saw it, I was like, Oh, is this making me motion sick? Because I'm not used to seeing like a mixed reality scene kind of move through a scene because usually you make it smaller so that you can have this all self contained scene, but you're actually moving us through different spaces in a way that I haven't quite seen before with a mixed reality. I don't think I actually did get sick, but I just noticed that
[00:23:57.805] Barry Gene Murphy: There is definitely one that's too fast, but we are working on that. Yeah, like, yeah, we're at our comfort level. Spidey Sense hasn't like really kind of done a pass on that yet, but I do understand what you're saying there for sure.
[00:24:09.695] Kent Bye: But there was at least the conceit to move through a location like a mixed reality context that is tabletop scale and have characters move through the space in a way that I haven't seen as much before. So love to hear any elaboration on pushing forward what we've seen so far with the affordances of how to tell stories with these techniques. And it seems like you're still trying to push forward what's even possible.
[00:24:34.261] Barry Gene Murphy: Yeah, always in that way. It's kind of uncomfortable sticking with what you know, I think, even though it's probably sensible.
[00:24:42.024] May Abdalla: I think only certain kinds of people find it uncomfortable to stick with what they know, let's be honest. It's a personality trait. I think what probably drives us, and maybe to some degree, yeah, is there's like a disappointment I think we have with mixed reality. and I think it'd be great to release all the experiments because you're right, there is a language already that's settled for this really new medium and that seems a bit frustrating. Why is it tabletops? Why didn't we do it one-to-one? Why all the characters one-to-one? Now you're making me kind of want to kind of just experiment with some of these other things.
[00:25:26.677] Barry Gene Murphy: There is a one-to-one, like ledges one-to-one.
[00:25:29.179] May Abdalla: Yeah, but that's not very next reality. But I think actually what kind of started to become interesting is thinking around the room mesh, the user's room mesh. some of the kind of moments when the effects that's happening kind of in that tabletop actually are happening in your room as well at the kind of scale of your room that I think started to feel like it would be interesting you know using kind of the potential for the filmic background of your life and the kind of yeah maybe like since Who Framed Roger Rabbit there hasn't been a lot of very good animation meets real cinema But in a way, you can kind of see why Hooper and Roger Rabbit, people stop there when you try and make interesting mixed reality.
[00:26:18.260] Barry Gene Murphy: I mean, it's like, it's a super challenge, isn't it? Like, you know, I think it's so easy to turn off the lights and just control the space entirely, you know, and mixed reality is a challenge. And I can see why now, you know, why it is a kind of difficult aesthetic problem and also use case as well. But apart from obviously information and learning and tutorials and stuff, you can see how that overlay is like, so going to be part of the fabric of how people like communicate, I think.
[00:26:46.785] May Abdalla: But there's this airline that, you know, you can look at your bag in like, it's an AR layer. So you can look at your bag to make sure it'll fit in the cabin. Like that's got so many more practical use cases. So kind of pushing that experiential one is interesting.
[00:27:04.118] Kent Bye: Yeah, I think there's a lot of emerging genres of say, like, site specific location based augmented reality, where you're going to locations, and you're able to overlay specific information on those locations. And then, you know, in this case, you're more than likely going to be at home. because even though people do take their headset out and about in the world, more than likely when they're watching experience like this, they are going to be at their home. And so then the question becomes, well, how do you start to use the context of your home to start to play you in a narrative context for something that could actually be a context that's not even your home at all, or their homes are off doing all these different types of adventures. So that's how at least I think about it is the mixing and mashing of these different contexts. And now as we continue to have new technology affordances with augmented reality that has us go out into these different locations, this kind of omnipresent, always on reality, then that may be different ways that you're really leveraging some of the true affordances of context with AR that goes beyond what you can get out of your home.
[00:28:01.823] Barry Gene Murphy: Yeah, yeah. And I guess it's also a technological thing as well, because there was libraries and dev kits that we perhaps would have incorporated if we started this project six months later. The meshing facility arrived halfway through and we segued towards that very quickly to absorb that because we really wanted it. But prior to that, we had designed a whole scene where you had to mesh your own room, you know. by just tracing it with your hand. We had to get that room in. If we had another six months, we probably would be able to incorporate object recognition or whatever. I don't know what will be coming along at that time, but it'd be another nice thing to have. We're at that point, because it is literally just trickling in on a case-by-case basis, these functionalities, and it's great. But I think the real power is coming as far as being able to leverage that for storytelling.
[00:28:57.835] Kent Bye: This piece is going to be having the introduction part. Is the introduction part going to be more VR based?
[00:29:05.778] May Abdalla: It's AOR. It starts AOR.
[00:29:11.510] Barry Gene Murphy: Or do we kind of mix it, mix it up, you know? Yeah. Yeah. So room mesh is always there.
[00:29:16.952] May Abdalla: Room mesh is always there. In the brain you are in your room, but it doesn't look like your room.
[00:29:22.435] Barry Gene Murphy: Okay. Similar to the set at 7.7 West.
[00:29:27.257] May Abdalla: Okay. But it is because your brain is your room, you know, like have you ever thought that your room and your brain are probably quite similar?
[00:29:34.080] Kent Bye: Yeah. You can think of it as a depth psychological projection of your inner life. If you have a lot of clutter. So you end up with in the very last section, at least as in virtual reality. So you're starting with mixed reality and the first couple of sections, and then you move into more abstract kind of like shader, like outlines of more of a one-to-one scale. So you see people that are more your size and you're able to kind of play around a little bit more with people reckoning with their diagnosis. And so, yeah, maybe you could explain a little bit how. you were choosing the visual aesthetic in the last piece to also recreate some of the fragmented nature of the experience.
[00:30:08.816] Barry Gene Murphy: Yeah, it's kind of like an internalised experience of the characters. So in a way, it's kind of aesthetically, we're looking for a negative space or a kind of understanding of negative space. So like seeing that there's a lot more there, but being aware that there's a lot more there, but not seeing it, you know, so just hinting at a vastness and a complexity. And so like the aesthetic is essentially just like rim lit. We've based it off like Caroline Leaf, you know, she's like an animator, National Film Board of Canada, who made some beautiful scratch animation films. And it's all done with like, you know, edge lighting and It's very soothing and kind of meditative. It's kind of easy on the eye, I feel, and that's what we're trying to get to anyway. And it's kind of trying to depict that internal space of Tara, Errol and Omar, the two characters that we kind of focus in on at that point. And then there is two more scenes after that. We move in and out of the mixed reality paradigm, I guess. So when we want people to consider more, we remove the stimulus.
[00:31:18.786] May Abdalla: Yeah. I think that thing about negative space is really key, right? So in a way, getting your diagnosis, trying to see your own brain is impossible, right? Not only is it impossible for you to see it because you can't turn your eyes around, but you're trying to understand something not through seeing it, but through seeing the permutations the kind of the impact of that thing so you know like for example there's there's a train in that scene but you can't see the train but you can see the sparks coming off the train and the people looking at the train and it's that sense of how can you hold on to a thing which is so elusive and it's kind of always behind you almost and And, you know, in many ways, although this is about ADHD, and we have like, lots of kind of issues with the nomenclature of these conditions, you know, it's very much around kind of having a trait and trying to understand what it is. And then invariably the mix between, like, am I really feeling like this? Or is this the condition? Or is this my laziness? Or is this because I didn't listen? And they all kind of have this confusion around a similar way to Goliath, you know, what's real and what isn't real. Because obviously, if it's a figment of your imagination, and it's your brain, it's kind of still your reality.
[00:32:41.487] Kent Bye: Yeah, I had a question around both the narrative structure and the interactivity because in the mixed reality portion of the piece where you're telling these stories of these four different characters, you often end with some quote and then that quote is displayed with text and then you use your hand to collect the text and then shoot it out of your other hand. So you're kind of like doing this cleaning up of the space, cleaning up the clutter and kind of redistributing it. And so there's the interactive components. And then when I was reading through the press release, I saw that there was some branching narrative components, but then when I played through it twice, either I was doing the exact same thing and I didn't find any of those branches or it's just like a linear implementation. So I'd love to hear if there are plans to have more of a branching or if there was branching and I just, you know, happened to do the exact same thing twice.
[00:33:28.701] Barry Gene Murphy: There's branching, but not in that section. We did look into it. It was like it would have been turned into Bandersnatch and we all went crazy. I think we tried that. But there is branching in a sense there is agency and there is like a repository of information to find in later scenes in the hope that has a kind of revisit value for some people.
[00:33:53.572] Kent Bye: Okay. So the version that you're showing us out by didn't have that yet.
[00:33:56.655] May Abdalla: No, sorry. Sorry about that. I'm glad that you tried it a few times.
[00:34:04.712] Kent Bye: Usually see it once and then just get a sense and I if I'm doing remote interviews I usually have a chance to watch it again just to kind of refresh my memory because the first time I'm watching and I'm really paying attention to What's happening with the medium? What's new what's different and the second time? I'm really trying to trace the narrative arc of what's happening with the story because VR is very difficult sometimes to take everything in all at once. Yeah, I sound design, the interactivity, and if there's branching narratives, then, you know, there could have been whole other areas that I completely missed because of how I played through it. But this piece, when I was watching through it, without having the intro, I think is difficult because it's not setting that broader context because I feel like I'm thrown into these four different stories and what I was trying to find was, okay, what is the through line between these four different characters? What's the similarity? And there's the risk of trying to represent ADHD as a monolith, like an individual is going to have the exact same experience. But there seemed to be this risk taking impulse control dimension that each of these four protagonists that you're focusing on, they're doing a lot of really high risk things, you know, sometimes like putting their life in danger. And so I'm wondering if you could maybe elaborate on the name of impulse of what you're calling this piece, but also if this is more of an extreme expression of some of these different ways of trying to put themselves in these high risk situations in order to flood their emotions, in order to feel the sense of normalcy as an antidote to ADHD, or just the process of trying to winnow down these compositive 50 different interviews down to these four different characters that are trying to get to the essence of what this condition is.
[00:35:38.863] May Abdalla: Yeah. So I think that the key feeling, let's say, through all of these characters, and I mean, the real people that we interviewed, and we kind of go into this more in the beginning, is that effectively, there's a dopamine deficiency in the brain and looking kind of into what the experience of that is. And people talk a lot about boys with ADHD versus girls with ADHD being very different. Like you look at often a girl with ADHD, you know, you don't see the kind of high risk getting into fights, but you might see... Masking, self-harm. But actually there is a through line through all of that stuff, even though it's kind of externally different, it's internally it's this kind of extreme dissatisfaction, the sense that something's wrong, the sense that something needs to happen, that something big needs to happen, and that can be, you know, like getting into crime or like enjoying being at that precipice, at that edge, where there's that tension-seeking clarity. But it can also be internalized into this kind of real sense of self-hate. And what's interesting, so there are three characters who, let's say, maybe have that more typical external kind of brazenness. But they have it for different reasons. And then the fourth character, her self-criticism really stems from the sense that she's too much, that if she really let herself be the person that she was, she will break everything around her because she's kind of sitting on a volcano. But this sense of kind of like, it's not just restlessness, you know, it's just rest is not an option. Like it was in all of those interviews, if you could kind of in a synesthetic way, if you could kind of touch that core. And, you know, we've been speaking with, you know, some great Yeah, like, you know, Dr. Edmund Senaga-Burke at KCL, you know, one of the global experts on ADHD and has ADHD himself. And, you know, we talk a lot about this, and he's really interested in our approach to some degree, because, you know, not as scientists, because we're trying to discuss experience. So sometimes what we're looking at cuts across loads of the things that may be there'd be a research paper on one aspect or the other aspect. And it's interesting to see what... this approach has to offer. Because we're already living in quite a kind of medicalized perspective of experience. And all of these papers, as important as they are, they'll always just silo one aspect. But as an individual, you're always going to have the mishmash of the whole gamut. And really careful listening to, sure, they were all people based in the UK. I'm not saying we could do a global comprehensive study. But there's just something that felt like, yes, it's this is either oppressed or not and kind of comes out into so many different ways.
[00:38:39.533] Barry Gene Murphy: Yeah, I guess. And that was something that we could write to and use to cut the technology for. And the focus around the impulse was true, like just seeing a pattern true, like those 50 or 60 interviews, but also our need to kind of like make the technology meaningful as well.
[00:39:00.648] May Abdalla: And for example, one of the characters, he talks about how when he's high up on a ledge, perhaps because he's going to break into somewhere because he's in the middle of committing a crime, he talks about that being a moment where he can finally think, where there's only one thing to think about, and so it focuses his thoughts. Whereas for someone else, it's like, well, I feel like I'm impressive. I'm worthwhile. And they're very, in a way, part of the concept was that all the characters, you do meet them at a kind of high risk place for four different reasons. It's interesting because I feel like you do need a through line, but at the same time, you don't want to say that everybody does the same thing, but otherwise you end up talking a little bit about a chaotic life or just And so yeah, I'd say also specifically the characters were chosen on the basis that their experience of ADHD was dramatic. I'm sure we all know people who have had a diagnosis whereby they might not externally seem like they're having a dramatic life. Although I think once you start talking to people about what they feel, There's a lot of richness there. There's a lot of awareness that people with this condition have of emotions around them. And it tends to spark a lot of things. So, yeah. You know, one of the characters, she talks about being a volcano. She talks about being set on fire. She talks about being a firework. You know, all of those metaphors just came around over and over again.
[00:40:33.247] Kent Bye: So I watched through the experience once and then I took it off when it said, okay, take off the headset. And then when I put it on the next time to watch the second time, it was actually playing through the credits. And then there was some scene that I saw, it was almost like an end credit scene that was kind of like a series of texts that was all jumbled up and it was describing what the project was. but it was very difficult to read because obviously it was kind of like real jumbled. And so it just was really striking to me, the ways that you can start to use the medium of virtual reality and mixed reality to start to communicate some of the visceral direct embodied experiences of ADHD. So I'd love to have you maybe elaborate on that specific scene.
[00:41:10.470] Barry Gene Murphy: That was probably a bug. So like, I'm kind of interested in what that was actually. Cause like, obviously you, you would be a great user tester. I have to say, uh, like what, what did you, what did you see?
[00:41:22.782] Kent Bye: So it was a text that said with ADHD, it was a bunch of texts describing what ADHD was. It was after the sequence. And then like, when you try to read it, then it's like really, the letters are all tumbling around.
[00:41:35.654] Barry Gene Murphy: Yeah, so that's actually a descope scene. So you found something that you might not see in the end experience. But I think that was a chapter called Text Context. And I was trying to get the idea that it was about the text itself, not playing ball. So we're working with agitated text. We're experimenting with the gaze base. But unfortunately, I think that the quest tree doesn't have eye tracking, for instance. So that scene probably would have made it had we leveraged that kind of tech more effectively. So we just kind of moved on because we're in a very fast iterative cycle of ideas and with a kind of like a timer at the end. And we can make decisions quickly to kind of like kill our darlings sometimes. But yeah, so that was essentially like trying to get the text itself to embody why ADHD, some traits of ADHD, essentially.
[00:42:33.783] Kent Bye: Okay. Yeah. It felt like an ending credit scene or something or something.
[00:42:37.326] Barry Gene Murphy: It might come back in the credits. Cause I was looking at it today and I was like, God, I don't want to lose that. Like, you know, but.
[00:42:45.119] Kent Bye: Yeah. Okay. And finally, I'd love to hear what each of you think the ultimate potential of virtuality, mixed reality, and the combination with immersive storytelling might be and what it might be able to enable.
[00:42:58.155] Barry Gene Murphy: Me, I think that like, I feel like we're still to go there, but like, you know, the kind of the social aspect of bringing, you know, more people into the room, co-located kind of experiences, you know, with your friends all around the world, that kind of thing. I think there is like a real exciting kind of like. world of possibilities for that, I think, you know, sharing the spaces, but really in the room, you know, rather than going into like the virtual rec rooms and what have you, you know, like still having your real body and stuff like that, but still being able to kind of share a lair. of communication. There's a lot there. I feel, I feel like theatre, like, you know, there's so many applications, like the Tender Claws do really good stuff with like live theatre, but kids storytelling, like there's so much live performance that can happen now because of this layer, you know, like Wayang Culi, shadow puppetry. I can just see quite a lot of like, you know, really nice experiences to come that allows you to kind of not completely get that sense of like, a burn from being in a VR, probably virtual world, you know, I think that like the AOR there gives you comfort, you know, offers comfort. And so in that way, there's kind of like a lot to experience and experiment with. So that's what I think about me.
[00:44:17.161] May Abdalla: Yeah, I guess also, yeah, that's a long list, isn't it? And I think also this kind of is a bit around what the next project is actually. I feel like, you know, we talk a lot about embodied experiences and I think kind of like messing with our sense of body a bit more concretely will be really interesting in terms of, you know, obviously you can kind of see it in places like VRChat and, you know, it's a comfortable space for a lot of people who might have gender dysphoria or all the variations of what that is that doesn't have a name given by the potential of having a very creative relationship with your body. I personally think there's something quite thrilling when a game properly gets you to kind of move so much so that once you've left, you have that afterglow of that sensation in your skin. And, you know, I used to kind of think a lot about that when you kind of leave the cinema and for a moment, you know, depending on the kind of film you're in that like after experience of that film. And I think you can have that ripple kind of in your body and like, what it means to feel kind of powerful or delicate or all of that things and give people an opportunity to kind of like change how you actually feel in your skin.
[00:45:38.460] Kent Bye: Nice. Yeah. Certainly lots to explore there for sure. And, uh, yeah. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:45:49.497] Barry Gene Murphy: I haven't left the house for a long time.
[00:45:54.199] May Abdalla: It's hard to think about. Come round. Come visit. That's what I'd like to say. Just pop over. Show you some.
[00:46:05.229] Barry Gene Murphy: I think I acknowledge that it's a really difficult medium to create in, and I think that everyone kind of beats themselves up on the results and how everyone forgets how difficult it is. You're not alone.
[00:46:19.917] May Abdalla: I'd say come around, actually. I always think it's helpful to share the stuff that hits the cutting room floor. because that's actually why it's nice to make this stuff. It's just so thrilling to try something and then get diverted and feel what's alive.
[00:46:39.795] Kent Bye: Awesome. Well, Barry and May, thanks so much for joining me here today to help break down Impulse, the preview that you're showing at South by Southwest. There's always lots of different innovations and experimentations that you're doing with how to tell stories in this medium. And certainly this is a topic that impacts a lot of people in the world. And, you know, just having a first glimmers of understanding some of the deeper dynamics of this condition of ADHD and how it impacts some of the people that I know and love. And so, really quite interesting to see the very early beginnings of it and very excited to see where you take this in the future. So thanks again for joining me today to help break it all down.
[00:47:13.459] May Abdalla: Thank you. Thanks, Kent.
[00:47:16.782] Barry Gene Murphy: Thanks for being a prolific VR custodian.
[00:47:21.687] Kent Bye: Sure. Yeah. Been a lot of fun to be on this journey with everyone. So thanks again for listening to this interview. This is usually where I would share some additional takeaways, but I've started to do a little bit more real-time takeaways at the end of my conversations with folks to give some of my impressions. And I think as time goes on, I'm going to figure out how to use XR technologies within the context of the VoicesofVR.com website itself too. do these type of spatial visualizations. So I'm putting a lot of my energy on thinking about that a lot more right now. But if you do want a little bit more in-depth conversations around some of these different ideas around immersive storytelling, I highly recommend a talk that I gave on YouTube. You can search for StoryCon Keynote, Kent Bye. I did a whole primer on presence, immersive storytelling, and experiential design. So, that's all that I have for today, and I just want to thank you all for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you could become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
