I interviewed Anouschka creator Tamara Shogaolu at IDFA DocLab 2023. See more context in the rough transcript below.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye and welcome to the Voices of VR podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. You can support the podcast at patreon.com slash Voices of VR. So this is episode nine of 19 of looking at different digital storytelling and immersive nonfiction pieces at IFA DocLab 2023. Today's episode is with Anushka, which is a AR interactive narrative game by Tamara Shigalu. So this actually won the IFADOCLAB award for digital storytelling, and it's an augmented reality piece about Anushka, who is living in an Amsterdam district called Belmemeer. So she's a black teenager who recently lost her grandmother. And so she's dealing with the grief of the loss of her grandmother. And she's recalling all these different stories of how her grandmother connected to music or food and connections to activism. So this is like an interactive narrative game using Niantic Lightship to overlay like a virtual world on top of this outside section within the context of the venue of IFADOC Lab. And you're able to have these kind of interactive game components where there's a gameplay loop of story, you go out and you explore around, you find some of these objects, you solve some puzzles, then you go into the next chapter of the story of these cutscenes. So it's a really beautifully designed piece and actually is being developed across a cross-platform context. So there's like the game components that are going on different platforms, but also there's an AR component. There's a single player aspect and a multiplayer aspect. And so there's a lot of things that are going on in Anoushka, but at the end of it, it's just a really well-told story and exploring the relationship between Anoushka and her grandmother, trying to understand who she is by looking into her past of where she's come from. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Tamara happened on Tuesday, November 14th, 2023 at Ifadak Lab in Amsterdam, Netherlands. So with that, let's go ahead and dive right in.
[00:02:10.627] Tamara Shogaolu: Hi, my name is Tamara Shogalu, and I'm an artist and creative director of a studio called Autoroto Pictures. And I work across interactive media from VR to AR and mixed reality of all sorts. I'm really passionate about storytelling and utilizing technology to tell stories.
[00:02:29.971] Kent Bye: Great. Maybe you could give a bit more context as to your background and your journey into the space.
[00:02:34.565] Tamara Shogaolu: Yeah, sure. I guess I've been working in the field of XR for almost 10 years now. And one of our first big projects was called Another Dream, which premiered at Tribeca like almost six years ago now. And most recently, I did a project called Unresolved, which won an Emmy for immersive storytelling last year and was done in partnership with Frontline about racially motivated unresolved cases. And that was an augmented reality experience. And since then, we've continued exploring more in the realm of augmented reality.
[00:03:06.680] Kent Bye: Great. And so we're here at IFA DocLab 2023, and you have another augmented reality storytelling experience. This time, it has a lot more game-like elements, I'd say. And so maybe you could give a bit more context for how this project came about.
[00:03:20.468] Tamara Shogaolu: Yeah, so this project started in 2020. It started basically like when I lost my grandmother in the height of the pandemic. She died at the age of 102 for natural causes, but I wasn't able to go to her funeral because of travel restrictions. And I spent a lot of time sort of like reflecting on how she impacted me as a person. and then at the same time there was the protest surrounding George Floyd's death and I was working on Unresolved and looking at these cycles of violence throughout history but then also looking at it from the perspective of resistance and like how multiple generations of people from different groups have been resisting and speaking up for justice. So that inspired the story and so the story is about a young girl who's in high school. It's kind of designed for teenagers who's dealing with the loss of her grandmother but she's able to navigate through the memories and kind of like have these fun moments of realizing that her grandmother was like a fighter and so is she in a way and find her own voice through the memories of her grandmother. And it was originally designed to be a location-based experience when I first started working on it. And then, of course, the pandemic happened, and that wasn't the best scenario. And simultaneously, we were working with Niantic, who did Pokemon Go, or does Pokemon Go, and utilizing their lightship technology for a different project. And I was really excited about all the different possibilities that were possible in AR now. in terms of like really allowing you to immerse and create totally virtual environments that you could explore through AR. And I thought that it would be a really cool experience to adapt it. And the more we started working on that, the more it started becoming a game and this sort of foray, I guess, into really like fictional storytelling that was grounded in sort of archival and nonfiction inspiration of materials in a way.
[00:05:15.722] Kent Bye: Yeah, so I can definitely see all the different strands that are coming in into this piece called Anushka is featuring this fictionalized character I guess her name is Anushka and then she plays the violin and I think one of the things that was really striking about this piece is the gameplay loop I'd say where you have introducing this world with cutscenes with really nice animation of these avatar representation of or I guess I think of my avatar but I guess it's just like animation characters because Otherwise throughout the course of the experience you don't really have any Embodiments of any other characters that you're really seeing you go into these cutscenes to see the characters So then you're having your phone horizontal to watch these cutscenes and then it flips into vertical to be able to use your phone with a scanning an area around 20 feet by 20 feet or so to have enough space to walk around what ends up being a translation into like this virtual world that's overlaid on top of this physical reality and so then you're going around and essentially using a mechanic of finding these glowing objects and so you end up having to explore the space to find the objects and then once you find the objects then there's often some sort of gameplay mechanic and that then you eventually go back into, once you finish that, get into another cut scene. So it becomes this whole loop of between the animation and the story, to the exploration of the space, and to then the little puzzle mechanics and the game mechanics that you have. So I'd love to have you explain a little bit for how you begin to structure this as an experience, if you started with the script, or if you started with the different gameplay mechanics, and how you created this fusion between the interactive components and the narrative components.
[00:06:54.859] Tamara Shogaolu: Yeah, that's a really good question. I don't know, the script, honestly, the script was really hard because I think it was three writers, three of us who wrote it together. And at the beginning, it ended up being like 20 pages. And we knew that we wanted it to have sort of like a poetic, rhythmic flow to it. So it was, it had a lot of spoken word elements in it as we were playing around with it. And then when I was working on the UX design of it, I realized that I think it was a lot of going back and forth, which is often the case. But I think with my strictly documentary projects, the story is kind of locked in a way. And then from the story, I'm able to create the UX. But this, I had more control in terms of how I went back and forth. And I knew that I wanted it to be a narrative and story-driven project. And we knew we had an audience that mostly played games. So it couldn't be super easy in a way of just allowing very simple interaction to go forward. like with some of the other projects that I've done, where I think that sometimes we try to make sure that the interactivity wouldn't prevent people from going forward. So having it be a game in that sense, we knew we had to make it more challenging. And then setting up a game loop, basically, that allowed it to incrementally get harder, I think, was also challenging in the process. I don't know if I'm answering your question. But yeah, I think for me, Yeah, we would do a draft, then I would go into the UX design. We knew what our game loop was in terms of like that there were three main quests on each level and that we wanted to keep the base sort of similar so that people would be able to like build up to it so that it got easier for them as they went on but also was still challenging enough to keep people who were playing it for the gaming aspect interested. But I think that that was one of the most challenging parts is like finding that balance and even now like it's still a work in progress and we're working on it and we're realizing now from testing it here at the festival that there are certain things that we want to remove and polish in terms of like the UI But I think finding that happy balance between keeping it challenging for the gaming purposes and then enough so that people are still invested in the story was the most challenging part, I think, for this.
[00:09:10.092] Kent Bye: Yeah, maybe it's worth describing both the spatial context of this place, because it seems like we're in a place called The Bims here in Amsterdam, but also the character of Anoushka. So maybe you could give a bit more context to both Anoushka and also the spatial context of where she's at here in this experience.
[00:09:27.797] Tamara Shogaolu: Yeah, so basically the Bilmer, which is like the southeast part of the neighborhood in the southeast part of Amsterdam, was designed to be the city of the future after World War II. And, you know, so it had these very kind of like brutalistic, large, massive tower buildings. But it wasn't very conducive for typical Dutch living, which people have their supermarket around the corner and all aspects of their life in a small radius. And this was trying to be bigger in terms of what a big city would be and having carports and all these things. So a lot of people ended up not living there and you had all of these empty buildings. And basically what happened was as Dutch colonialism was coming to an end, a lot of people from the former colony started coming to the Netherlands. And because of discrimination, they couldn't find housing in the city center. And they ended up having to squat and they squatted these buildings in this like abandoned city of the future, which has now become the most diverse part of the Netherlands. There's over 250 nationalities that are there. And I think it's like a really exciting and culturally thriving part of the Netherlands. And it's interesting to me what I thought was beautiful that even though it was designed for like a different set of people, people kind of made it their own. And now it has become in a way a city of the future. Oh, wow.
[00:10:46.226] Kent Bye: It really comes full circle in that sense. So yeah, maybe you could also describe a little bit more about Anoushka as a character.
[00:10:52.718] Tamara Shogaolu: Yeah, so Anoushka is of Surinamese heritage, and the story kind of delves into the memories of her grandmother and her great-grandmother. And her great-grandmother was from Java and was brought over to Suriname as an indentured servant, and then her grandmother emigrated to the Netherlands, and that's how Anoushka was born here. She's a very talented violinist from the Belmer, which today in present-day Dutch society, a lot of people from the Belmer still face discrimination and it's kind of looked down upon in many aspects. It's something that I think is changing, but it's still kind of segregated in a way, sadly. She's from there and she has to travel to the city center to get her violin lessons and she starts learning at a young age that there's a difference between her community and the rest of the city of Amsterdam. And through that she finds her voice and finds pride in who she is when a teacher challenges and tells her that she should just fit in instead of embracing and accepting her differences. So as a player you go through these memories of Anoushka, her grandmother and her great-grandmother, kind of like really learn who she is as she's learning that to embrace her heritage as well as like embrace who she is to make her craft unique and yeah and allow her to like pursue her dreams and stuff like that.
[00:12:12.062] Kent Bye: Yeah, so I had a chance to play it on an afternoon here at Ifadak Lab, and it actually is in this courtyard that's outside that is in this big space that you're moving around. So it allowed you to overlay what felt like a whole spatial environment that had some pass-through elements, so it wasn't like a complete virtual environment, but it still had some elements of the physical space. and as I am walking around and basically you're trying to find different objects and then those objects trigger more gameplay elements that then become the quest and I'd actually gotten stuck on one part where there's some music notes and I figured out eventually that you have to wait until the music notes go on top of the tracks and then that's what activated it but it was like a very much a timing thing and it took me a really long time to figure it out but there was another kid who is probably somewhere between 10 to 13 or 14 maybe Who had started after I had started and finished before I had finished It was sort of like one of these things where the fluency of this is a technology It seems like the youth have just take to it like water They just understand it in a way that took me a little bit longer and sometimes when I'm in a festival context it's like I want to see the story and And then if it's got too many puzzle or game-like components that are difficult to figure it out, then it can be a little bit frustrating. So I think I was in a little bit of a loop there of there are some aspects of this experience that do require you to puzzle through things or at least find things. And then there were options to skip if you are done with trying to figure out some of the puzzle that you can skip to the narrative components. But for people who are really into those game-like components, there was a lot of different ways that you were trying to push forward what I'd say is the typical type of gameplay mechanics that I see in an augmented reality experience. So I'd love to hear you elaborate on the design that you had of, not only you have the narrative arc that you have, but you also have an arc of the gameplay loops that are trying to increasingly become more difficult. But there's also a way that you're tying the different mechanics and the objects of these different games back into the narrative. So there's a narrative drive that is connected to it that then in the next phase of the narrative that you're unfolding, then it ties back into diving deeper into either food or music or statues and what's exalted in the culture and the protest. And so, yeah, I'd love to hear you expand upon how you're both thematically picking different aspects from the narrative, but also translating those into these different gameplay mechanics.
[00:14:43.825] Tamara Shogaolu: Yeah, I think, yeah, the thing about kits is true. In the testing sessions, we noticed that and even with like really rough prototypes earlier on, like they totally got it. And I was like, wow, how are you getting this? There's like no UI. So it's really funny that that happened. But yeah, I think when I was designing the gameplay, I knew that it was a story about music and how music ties different generations in a way. So that's why music is a big element in it. And in that particular quest that you're talking about, like the tracks are, the idea is that they're staff lines. So you're like writing music, helping Anushka sort of write a song as you're tapping on the musical notes and having to like place them on the staff lines. and trying to connect that in a way I wanted to create it so that basically you have to move around so that you really explore the virtual space that we've created that is connected to Anoushka's memories and also like get a better sense of her neighborhood and like the mix of cultures and stuff like that that exists or in the market. The idea was that you would be picking all these different dishes and then understand how they're culturally significant and related to her and then having to make the recipes as well as hearing stories about her family and her grandmother in the process of cooking them, which I think is something that often happens. I guess if you cook with your grandmother and stuff like that, there's usually stories that go along with it. So I wanted to create that feeling. And then similarly, I think for some of the other, like I went to the physical market that this is inspired by and started just paying attention to what is in this market and what games could be connected to that. But I think that in all the levels, Each one has a theme in terms of how I tried to design the interactive gameplay elements, where the first level touches a lot on food and musing and culture and setting up the community, and then the second level is more about this further legacy of the first immigrants who came to the Netherlands, in terms of touching on Anton de Kom, I learned about recently who was born into slavery and ended up dying in a concentration camp and emigrated to the Netherlands and helped to fight the Nazis here. And this part of black history is something that I had no idea about. There's also another level where she finds her own sense of agency and voice and how her grandmother inspired her to do that so that's more about like voice and speaking up and then the final level for me was more about coming to terms with grief and like what are our connections from like our ancestors in the present and how do we continue like move forward with that so that one touches a lot on like the musical sort of legacy that she inherits from her grandmother and from her great-grandmother and how many of the musical influences that she has stem all the way to Java and Indonesia and through storytelling. So I wanted to tie the music to the family in that final level ultimately. So that was the objective between how I designed like each of the game elements for each level.
[00:17:52.022] Kent Bye: Yeah, and coming back to the narrative for a moment, you said that this piece was created on the onset of losing your own grandmother. And in the piece, I have recollections of seeing some cut scenes when Anoushka is with her grandmother, but I don't remember at the end, does her grandmother pass away in the course of this piece as well? Is there a moment where she dies in the piece or is she already dead?
[00:18:14.145] Tamara Shogaolu: Yeah, so basically the idea is that on the final level, you kind of realize that the grandmother has passed away. And hopefully, I don't know, my intention was that people would kind of realize that part of the reason she wanted to give up music was because music was so connected to her relationship with her grandmother that she felt so alone playing music without her grandmother. And then her finding the strength to continue to play music by exploring sort of these memories that she had with her grandmother.
[00:18:43.035] Kent Bye: So in terms of the timeline, had her grandmother already passed at the beginning and all the cut scenes are memories or does she pass away in the middle of the experience?
[00:18:50.703] Tamara Shogaolu: Well, her grandmother had already passed at the beginning. So the idea is that like it's kind of her grandmother spirit basically that is with her at the beginning when she's having those moments where she's sad. Yeah.
[00:19:02.895] Kent Bye: Okay. Okay. Yeah, I think part of the challenge sometimes of these experiences is that sometimes there's the narrative component and then I'm like having to solve the puzzle and then I'm outside and then there's like light, you know, if it's just a film, it's something that I think the affordances of the film, I can just sort of like absorb and read and decode it. But sometimes when it's through the different cut scenes and then like was super frustrated of like not being able to figure out a puzzle and then eventually I get there and then I'm still in this like state of, not being in the most receptive being able to receive it. So I did actually play it through a second time because there was actually a technical glitch where it had crashed during one of the cutscenes so I didn't see the end. But I did see this overall arc of letting go of the music and coming back to it and also the nostalgia of the pictures and the memories and the looking back but also reflecting on the food and the music and I guess the making of the recipes is reflecting of the music and the gameplay component, but then there's the first time that I did the musical component, I had a really hard time identifying the common genres because there's this kind of matching game that you have where you're supposed to find the similar genres that go together. It's a memory game where you flip over a card two at a time and you're supposed to find the pairs. And then there's one where there's 12 different total albums and you have to come up with four different pairs of three matching the three genres. And I had the hardest time trying to like identify from music and I was only looking at from the music perspective. And eventually I realized, oh, there's also like a color component that that was like easier to decode. And I'd probably have to play it again just to like authentically like match the different music. But I found like stuff like that where there's a little audio cue that plays and then you're supposed to say, what are the common genres of this music? It was something that was unique and fun, but also it's near the end. So it's a little bit more of the challenging one. So you have this progression of these tasks a little bit harder and harder, but Yeah, I'd love to hear you expand a little bit on this process of turning these specific music genres into a gameplay mechanic.
[00:21:07.283] Tamara Shogaolu: Yeah, I mean it's exactly what you just said, where like in the first level there's a similar quest but it's based purely on visuals for the memory game and then so in this level we decided to add a sonic quality to it and yeah everyone kind of has the same experience so it's really funny to watch people because I'm like If you just see that they're the same color, it becomes much easier. But I think it's also fun because then people can explore these genres that maybe they're not familiar with or that could be like lumped into being one thing. And, you know, like one of the genres is music from Indonesia called Keroncong. And then once you unlock it, you hear why like that type of music was important and that her grandmother played it on Sundays. and I'm from like a very mixed cultural background as well and like there's certain types of music that were played in my household as a child like on a specific day of the week and like music really is connected to culture and memories so I wanted to translate that into like it's not just music in some cultures it's like music is really part of the sensory experience of a memory and moments with family or like in growing up and childhood and stuff like that. So that was my goal, to really try and get people to really listen to what these different genres mean and how they're connected.
[00:22:31.449] Kent Bye: Yeah, and there's another musical puzzle where you could play and it would play through the staff and it would play certain notes and then there'd be a missing notes where you'd have to drag and drop a little like glowing dot and to fill it. And I also found that to be difficult to the point where I ended up just randomly doing it until I got the right ones without really trying to puzzle through it. So I feel like that's another one where I don't know if there's a way to hear it like at the top just to hear what it sounds like and then hear what's missing and hear the contrast because it is a little bit difficult because they sounded so similar it was difficult to know what the overall arc of the tune was. There's a lot of repeating notes as well and so yeah and once you do it you get to hear what it sounds like but I did find myself kind of spamming it in a way where it was like I wasn't really puzzling through it was just more of like Randomly doing the embodied action without really thinking about it, which was kind of like a brute force approach I guess of solving the problem rather than trying to puzzle through so that was my experience of that one
[00:23:27.942] Tamara Shogaolu: Yeah, no I think that one is challenging and we've been talking kind of internally on like the tweaks and aren't totally sure if we want to keep it because we're also adopting the experience now for like console and PC and stuff like that and trying to figure out like how is it gonna play on all these other different platforms. And I think for us, sometimes it was more of this fun idea of being able to grab stars and make your own beats. But yeah, I think it needs to be fleshed out more, totally.
[00:24:01.703] Kent Bye: Yeah, and there's also, in the course of the story, there's some of Anouska's ancestors that are coming from other countries, like Suriname. Maybe you could talk about why you chose some of the countries, or just to give a little bit more broader context for the different countries that are coming into here in the Netherlands.
[00:24:20.086] Tamara Shogaolu: Yeah, so Anoushka is Dutch and it's based here in Amsterdam and there's a significant portion of the population that is of Surinamese descent, so this type of cultural mix is something that's very common here in terms of the story stemming all the way back to Indonesia where Dutch colonialists took Indonesians from Java and moved them to Suriname and then that mixed with like also the enslaved people who were brought from Africa to create this very unique mix of culture where it's a mix of like Dutch and African, West African and Indonesian and there's also some workers that were brought from India as well that are part of that mix and then once the colonial rule in Suriname ended a lot of those people came and here in the Netherlands you can find like all of that sort of culture is like very alive and present so It just felt really natural to go into that path for creating Anoushka and the community that she's from and her background and stuff like that.
[00:25:22.095] Kent Bye: Yeah, and going to an earlier part of the experience, you mentioned that there was the George Floyd protests that had happened with Black Lives Matter and that there's a whole section in here that is around this aspect of racial justice that was happening at the time that is also reflected in this piece. And so you have a couple of things where there's a statue, but also you have like this ability to kind of write this sign says like, no one is free if anyone is oppressed. And maybe you could talk about both the sign and the statue that's in this piece and how that ties into the narrative.
[00:25:56.162] Tamara Shogaolu: Yeah. I mean, I don't know. I, I think a lot of people may be like outside of the Netherlands inside of Europe, like aren't aware that there's like a pretty sizable Dutch population of African descent. And I think that throughout the African diaspora, like a lot of those stories are marginalized and undertold. And I really wanted to center and highlight that and, most of the world saw how George Floyd's murder kind of ricocheted throughout the world. And a lot of people were able to connect to that and see their own personal struggle or fight for freedom and against oppression in that. And I was in the Netherlands when it happened, even though I'm a black American, and there were protests that went on, and I saw how interconnected the black Dutch experience and the black experience throughout the diaspora is connected. So it seemed very natural that that was something that Anoushka as a character would experience and go through. And it's something that I see in this younger generation or like Gen Z where they're very actively connected to like politics and outspoken and protests and movements when it comes to climate change. and other forms of oppression, so felt like if she's a 16-year-old, this is what a 16-year-old would be doing. And it's something that, I don't know, at least with my personal relationship with my grandmother, like I do a lot of my work is usually connected to social justice issues, and that was very much inspired by my grandmother in many ways, who always taught me to speak up and when I'm in a position of privilege like to help others and speak up for others. So to me it just felt something not fit with the story and yeah I was inspired by my own lived experience but then also the world and what seemed to fit with the character.
[00:27:43.457] Kent Bye: Yeah. And so as you've been developing this interactive narrative game for AR, there's obviously a lot of iterative processes and they're likely going to take some of the feedback you get from this festival and continue to iterate on it. And so I'm curious what's next for this project. You mentioned that there might be translating it to other platforms. So yeah, I'd love to hear what's next for Anushka.
[00:28:05.043] Tamara Shogaolu: Yeah, we're currently in conversation with different partners and are fundraising for the next phase. And our goal is to release like the on the App Store and on Google Play next year. And then we're also working on adapting it for console and PC as well. So hopefully, fingers crossed, we'll be out next year. And then we'll continue. There's a possibility that we're going to expand on it as well and like add to more layers of it for gameplay. And the main idea was that I think something that I should add is that because we're working on this or started working on it during the pandemic, like we really wanted to get people to go outside and kind of like move around and be physically active. So that's why with the AR, it was designed to be this experience that you have. outside and there's also like when you play it on your own mobile it remembers where you stop because it is quite a long experience so I think in the festival setting I realize that people kind of feel forced to go through the whole thing but it's meant to be like digestible at your own pace and not all of the quests for example need to happen in the physical space so sometimes you can activate it in the physical space and then sit down and do it and we really wanted to offer like this sort of flexibility So what we're working on now is that it's gonna happen, the experience can also happen cross-platform so you can be playing it on your phone and then you can get on your PlayStation and continue playing it at home and vice versa. So that's sort of like what we're working on right now.
[00:29:32.939] Kent Bye: Yeah, I noticed that there is a lot of features in terms of continuing but also like recentering is something that I ended up using a little bit more than I expected just because maybe I was sort of exploring around in areas that I didn't actually need to go into. But when I was doing it in a little bit more of a confined space inside because I wanted to go through it the second time just to be able to see the ending because it had crashed at the end with some sort of glitch or bug. I was able to go through and play through it but I was in a constrained space but I noticed that when I was really close to a white wall and just tracking it didn't have enough features to be able to keep the tracking it would lose the tracking and so I would have to kind of reset it and so it is possible to play inside but it's much better as an experience a lot more stable when I was outside. And sometimes I was using the recentering just because it is like a pretty broad space. And there were moments where I was like asked to kind of walk into what was a little bit more of a garden space, or I was running into a wall and needed to go back to the extreme and recenter it and then walk back. So there was a little bit of just with the constrained space outside even, but doing it inside, it worked, but it was like really quite glitchy in terms of the technology wasn't nearly as solid as doing it outside.
[00:30:43.247] Tamara Shogaolu: Yeah, I mean, it works indoors, but it has a minimum space requirement. And I think that white walls and flashing lights are AR's worst enemy. I don't know anyone who's been able to overcome that. So I'm not surprised that there was loss of tracking with the white wall. Yeah, we designed it to be something sort of outside. And the idea was also that it allows for up to four players, which we weren't able to finish the multiplayer function in time for this festival. But the idea is that you will be able to work collectively on the quest. So it was something that I was imagining, like friends at a park playing outside with lots of space and then being able to explore this world together. And then for the other versions that were working for the other platforms, Yeah, I would allow you to play it indoors as well when it's cold or like the weather isn't nice and stuff like that.
[00:31:34.853] Kent Bye: Yeah, there's definitely moments when there's other people doing it at the same time I was. And I'd gotten all the ingredients, but I didn't know what to do next. I was like spamming, tapping just to kind of figure it out. And then I asked someone else and it's like, oh, you have to find the pot, you know? So there was this level of not having a lot of interface clues. Sometimes there was a little light bulb and the light bulb would maybe give a vague clue, but the clue may have been related to either the previous or the next, or sometimes there wasn't a clue. Yeah, having a clue system that is robust for people to get unstuck. But also, if you are doing it in a group, then were you planning on having one person solve the puzzle? Or would it be more of each person has to do their bit to be able to then collectively move forward? And if somebody hasn't done it yet, then everyone gets stuck.
[00:32:21.209] Tamara Shogaolu: Yeah there's so right now the way it is is that on each level there's one collective one so on the first level for example with the different players you have your own grocery list that you have to gather but then you have an ingredient and somebody else has an ingredient you have to come together to like cook the dishes in order to move on collectively from the level but then there's still sort of individualized gameplay that happens and yeah exactly what you were saying is what we were hoping so that it would be some moments where it feels like a personal experience, but then also have these moments of play with other people and interactivity when exploring the world as well, so that it has like a social element to it as well.
[00:33:00.339] Kent Bye: And I wonder if you could elaborate a bit on the whole spatial context of this environment that's overlaid on top of the physical reality. So just the design inspiration for those virtual spaces that were being transported into.
[00:33:13.373] Tamara Shogaolu: Yeah, I really wanted the world to feel like it was made of paper in a way so that like it's kind of a storybook coming to life. I work a lot in animation and I worked with the stop-motion artists that I collaborate with a lot and we created like hand textures and then scanned them and then applied them to models. to give it that sort of like paper quality and then created the environments. Yeah and then the scanning allows us to kind of like allow sky segmentation and for it to know where the ground is essentially and where the sky is and for us to be able to like set it so it matches up with the real world and feels like you're actually in that virtual environment.
[00:33:56.091] Kent Bye: And so in using the Lightship APIs, I'm wondering if you could expand a little bit of some of the new features in AR that you're able to explore in the context of this project.
[00:34:07.240] Tamara Shogaolu: Yeah, I mean, the sky segmentation, also like the real physics of objects and stuff like that. So when you drop things, they actually fall on the floor. You're able to pick them up. Or what you saw with the sculpture, where you're able to move around the 3D pieces in order to place them. And it also allows you to actually walk in through the space as well. And I think that one of the things that I really liked about Lightship was the way that they're tackling occlusion, which allows you to move through these virtual spaces with AR.
[00:34:39.666] Kent Bye: Great. And finally, what do you think the ultimate potential of augmented reality and immersive storytelling might be, and what it might be able to enable?
[00:34:48.956] Tamara Shogaolu: Oh, I think there's tons of potential. And I'm really excited. I don't know. Personally, I'm really excited about the cross-platform aspects of it and being able to have a story that has an element that is AR and an element that isn't. That's something that I'm exploring. And I just think that there's so many applications for AR. We already have our phones in our hands, which I think is something that automatically connects us to an audience. And there's so many. I mean, there's tons of applications that are being used in fashion right now. And I think it's going to become something that is way, way more part of our daily lives, for sure.
[00:35:26.013] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:35:30.682] Tamara Shogaolu: Yeah, keep an eye out, like follow us on social media. You can find us at auto auto pictures on Instagram and Twitter and LinkedIn and everything. And we'll keep posting updates if anybody has feedback and has a chance to play it to reach out. It's a work in progress and we look forward to continuing to share updates for sure.
[00:35:50.935] Kent Bye: Awesome. And yeah, I really enjoyed this piece and I felt like it's a real nice blend of the narrative components and the gameplay components and also the AR. And I feel like, you know, this kind of puzzling aspect. And certainly as I watched the youth that I saw play it, they were able to pick up things that I wasn't able to pick up. And so there may be some different ways in which the affordances of the medium are very natural and intuitive to the zoomers and even younger. Yeah, I just thought it was a really well told story and I look forward to seeing how you continue to explore this fusion between the interactive game-like puzzle components as well as with the story. I also really appreciated just being able to move my body through large swaths of space which doesn't always happen in other VR experiences that tend to be much more constrained when I'm able to just go outside and really immerse myself into the story. And I'm really surprised to see how much I'm able to look through the phone and kind of project myself into this virtualized immersion in the context of AR. Really quite powerful in that way. So anyway, thanks again for creating the piece and for taking the time to help break it down.
[00:36:52.563] Tamara Shogaolu: No, thank you. I think what would be really cool is for parents and kids to play together. And I hope that happens. We were trying to tell a family story and something that could be fun for kids alone. for families to be able to play together. So fingers crossed.
[00:37:08.060] Kent Bye: Yeah. The kid that was playing it was playing it with his father. And they were, I think, helping each other as they were going through it.
[00:37:13.387] Tamara Shogaolu: That's great. That makes me really happy to hear. Thank you.
[00:37:16.952] Kent Bye: Thank you. So that was Tamara Shigolu. She created an AR interactive narrative game called Anushka, which won the IFADOCLAB award for digital storytelling. So I have a number of different takeaways about this interview is that first of all, well, there's a couple of pieces that were at IFADOCLAB this year that were games. There was Despalote, which I had a chance to actually play and talk about a little bit with the curator of the Tribeca games back early in 2023, but the, Anoushka is like an AR game that is integrating storytelling with these more cinematic cut scenes, and then having you interact with this space where you're walking around a physical space, but you're using your phone as a portal to look into different dimensions of this Belmimir and other regions around Amsterdam, but also just digging into the history of Anoushka with the relationship to her grandmother and all the different cultural strands that are coming through music and food and activism. Yeah, lots of different fusion of this gameplay interactive component with the narrative components as well. So it was the winner of the award for digital storytelling. And part of the jury statement said, stepping through literal and animated portals, this piece beckons us on a journey that intertwines with the essence of home, skillfully navigating a tapestry of mixed cultural identity and the personal grief of losing a grandmother. In a realm that unfolds through exquisitely crafted animation, original music, and narration that evokes awe, as participants traverse the physical garden, they are seamlessly immersed in the augmented reality of Inushka's first-hand experience. Following the glimmer of sparkling objects and engaging in simple yet profound mini-games, we are graciously invited to partake in the participation of traditional meals, savor the melodies of old records and family home, and stand in solidarity with Black Lives Matter movement. This piece masterfully strikes the delicate balance between a unique narrative and its chosen medium, offering inspiration to future creators. The work carries an important message that we collectively bear the responsibility to nurture an understanding of each other's stories. Much like Anoushka, we are spurred to explore our own heritage with a sensitive and inquisitive spirit. So that was the jury statement for Anushka. And yeah, it was a really strong story. And I think probably the fusion of interactive game components with the narrative components, I think just worked really well. And the games were actually interesting and intriguing, sometimes challenging. I don't typically see that a lot within the context of these immersive storytelling festivals. And so it's really great to see more experimentation and fusion of both the game-like components and the narrative components. And it opens up the possibility for a piece like this to appear on a variety of different platforms. they're going to be continuing development and probably launching sometime here in 2024. So that's all I have for today and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoyed the podcast then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a VISTA-supported podcast and so I do rely upon donations from people like yourself in order to continue bringing this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
