Iago: The Green Eyed Monster is an Animated AR, Rock Operetta, exploring the villain origin story of the character of Iago from Shakespeare Othello. There are links to the iOS and Android AR app linked on their website, and I’d recommend checking out and then listening to this in-depth story and process breakdown that I did with co-creators Josh Nelson Youssef and Mary Chieffo.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. So I'm going to be starting a series of my coverage from the Tribeca Film Festival, where I'm going to be going through a number of the different experiences that I think are interesting to unpack the different structures and forms of immersive storytelling. And also just as a reminder, I am a Patreon supported project. So if you want to support the podcast, you can go to patreon.com slash Voices of VR. But the first episode that I'm gonna be diving into is Iago the green-eyed monster This is an augmented reality piece that you can actually check out you can find the link in the description But this is a short music video that's animated and it's about one of the main characters called Iago from a Shakespeare play of Othello So it's kind of a poetic take it's a animated music video But it's gender-swapping Iago and it's a piece that when you watch it this is introducing you to a character some of the character dynamics and it's taking some Shakespearean and translating it into a song. So I recommend checking it out and then getting into this breakdown where I talk to the co-creators who talk all about the evolution of this as a piece and some of the easter eggs that are hidden within the context of this piece from a Shakespearean perspective. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Josh Nelson-Yusuf and Mary Chifo happened on Friday, June 10th, 2022 at the Tribeca Film Festival in New York City. So with that, let's go ahead and dive right in.
[00:01:34.848] Josh Nelson Youssef: Hi, I'm Josh Nelson-Yusuf. I'm co-creator and director of Iago the Green-Eyed Monster, an AR, rock operetta, villain origin story, Shakespeare-infused project.
[00:01:50.072] Youssef Mary Chieffo: Indeed, and I am Mary Chifo. I am the fellow co-creator of Iago the Green-Eyed Monster. Musical AR. I've done, like, all the different ways we can describe it. Musical AR, animated experience. I'm also the performer in the piece, playing Iago. We did Xensuit Capture, so it is my movement that we've animated. And it's my voice singing the song that Josh and I wrote together.
[00:02:15.665] Kent Bye: OK, yeah, maybe you could each give a bit more context as to your background and your journey into AR storytelling.
[00:02:20.947] Josh Nelson Youssef: Absolutely, yeah. So I worked in the traditional film space for years, worked on about nine different feature films. And I worked on a project where I got to meet a wonderful actor by the name of Beth Grant, the mother of Mary Chifo. who's looking to produce an adaptation of Shakespeare's Othello, which was written by Mary. And I had a big love for VR. I'd seen VR at Sundance and had some friends who were in the space at Within. And yeah, we jumped right into it. We started producing 360 videos back in 2016, understanding the pipeline, doing manual stitching, all of that. And eventually, we worked together to set up Operation Othello with Juby Productions, Viola Davis and Julius Tennant's production company, and Oculus, and at the time, it was Here Be Dragons. And yeah, that was sort of our foray into the space. I ended up staying on with Juby Productions, heading up their immersive and interactive department. And we created several pieces. We did a piece called Songs of Infinity, which was a large 360 video adaptation of an episode of Spheres. We did Operation Othello. We did the March on Washington. It was called the March with Time Magazine, Digital Domain, Valis, Riot, and where you got to actually walk on Constitution Avenue, get to participate in the March on Washington, and get to see a fully rendered Dr. King deliver the I Have a Dream speech. It's been a really exciting space to be in and really excited to get to release the green-eyed monster into the world. Yeah, that was a piece by Alton Glass, right, that he helped write. Yeah, so Alton Glass, Mia Trams. Yeah, it was a wonderful team. It was an honor to get to work on that piece.
[00:04:04.085] Kent Bye: Yeah, I saw that it just had its opening in Chicago right at the beginning of the pandemic, so I don't know if it's had a chance to actually be shown out in the world yet.
[00:04:10.550] Josh Nelson Youssef: Yeah, I know. Unfortunately, yeah, it was poor timing. I mean, we had great receptivity for the privileged few who were able to go through the experience, but hoping we'll be able to expand more broadly in the future.
[00:04:21.851] Kent Bye: OK, maybe you can now share a little bit more of your background and journey into AR storytelling.
[00:04:26.548] Youssef Mary Chieffo: Yes, absolutely. Luckily, Josh has covered some good ground of how I've gotten into the immersive space. But just going a little bit further back for me, my training is primarily in theater. I did grow up the daughter of two working character actors, Beth Grant, my mom, and Michael Chifo, my father. And just in those creative spaces, eventually realized that I also wanted to act, much to their chagrin. But still, they encouraged me to go after my passion, which is very kind of them. That's the shortest version of the story. And then I ended up auditioning and getting into Juilliard for drama, and that's where I studied. So it's great to be back in New York right now. And my father is also an alum, as is Viola Davis of Drew B. Productions, our savior of life. So my background is primarily in theater, study a lot of Shakespeare, study a lot of classics in general. I think because of my height, I'm six feet, and my stature and all that, people have often placed me in these very intense classic roles. And when I was in school, I ended up playing King Lear as a woman and Macbeth as a woman. And then right out of school, I got asked by Trezana Beverly, who had been a director at the school, to do this all-female Othello for Harlem Shakespeare Festival that fall of 2015. And I auditioned, and she cast me as Iago, and I really fell in love with the character. I was playing it traditionally as a man, the inverse of Shakespeare's day sort of thing. Joss has heard me speak to this many times, but I really reevaluated my idea of honesty. And I realized that because I was playing Iago as this, like, kind of gritty, swashbuckling, shoot-from-the-hint, blunt guy, and how sexy that was to the audience, I was like, oh, as a woman, I don't allow myself to be that gritty. I expect my honesty to look like Desdemona, pure and honest. And I still think there's a really badass interpretation of Desdemona because she's also such a rebel, like she goes and marries, find her father's back and all that stuff. Anyway, I have a lot of opinions about Shakespeare. But what really struck me and what the through line leading us to this piece is that I was like, you know, I want to keep working on this role because it's Shakespeare and it's one of the greatest, if not the best villains to work on. But I want to see what Iago looks like as a modern woman because I felt the themes that I really embraced in working on the role originally of him being a grunt soldier that feels overlooked and jealous and angry and wants, in a very adolescent way, to have his superior officer feel the pain that he feels. So for me, as a modern woman who's ingratiated herself in this male-dominated military world, it totally tracks that Cassio getting promoted over her would spiral her into becoming the notorious villain that Shakespeare has written. So we had worked on that together, and that brought about Operation Othello. And then, actually parallel to that time, I was cast in Star Trek Discovery as the Klingon L'Rell. who is also a woman in a man's world. The Klingon society is pretty patriarchal still, unfortunately for Laurel. But it was interesting working on my adaptation of Othello, which we call color and gender conscious. So we're really looking at how changing the gender or race of any character informs how they function in the world. But I've had that parallel journey as I was immersed in the Trek world and Allurel was rising into her own power as a leader on the show. And then all of that was happening and then Josh continued to work in the immersive space with Juvie. And then about a year and a half ago, Verizon was looking for content and we ended up pitching this song. So that's kind of what led us to this moment of pitching this and then, you know, since then have refined what the song's about, all based off of Shakespeare's actual words. All the lyrics are derived from or direct quotes from the text and really discovered that the piece was about microaggressions and about how these small mosquito bites, when you get enough of them, they create a monster and how the micro becomes macro and how we all have a monster within us but also It's because of the monsters without us. So, you know, I love looking at the kind of, I was saying, the Ouroboros of this cycle of prejudice. And that's obviously something that Juvie is very enthused to point out and look at in their storytelling. how marginalized and minoritized groups need their voices to be heard. Because for me, marginalized and minoritized, I love the phrasing of that because it isn't a minority, it's because we've been pushed to the side in different ways. That's part of my origin story for sure.
[00:09:06.251] Kent Bye: That gives a lot of context to this project as well, because I feel like when I saw it, I watched through it twice now. The first time I was just getting the music, and the second time I was like, what's happening in this as a story? And there's a whole other layer that I'm not as, like I have a background in electrical engineering, and so I went the whole math and science route, not the humanities route, so my fluency in Shakespeare is not up to the point where there's a lot that's going over my head in terms of a piece like this. But what I find interesting is that there is established characters and a lot of dynamics that you're building on top of, there's a lot of immersive experiences like Sleep No More that has, you know, elements of Macbeth. There's Elsinore was based on Hamlet. It's an interactive media piece that's like taking a minor character from Shakespeare and then what would happen if she was a major character being able to have a dynamically changing storyline. And so it seems like Shakespeare is, for storytelling, a way to start at the baseline but then use the emerging technologies to see what's new. And so it feels like this is a project just starting to do that as well to start to play with both the structure and form of the story but also to just create a banger of a music video like even if you don't know any of that it's still an entertaining experience but I feel like for me at least there's elements of the story that I have to watch a number of times to kind of unpack what's actually happening. So love to hear any initial thoughts as you're taking this as a piece and trying to explore the forms of storytelling within AR but starting with the canon of Shakespeare.
[00:10:28.632] Youssef Mary Chieffo: Well, it's great to hear you say that, because it really is a huge part of our goal, is the cross-blending of the immersive space and the theater space. And for anyone who doesn't know every word of Shakespeare, or the person who does know every word to the person who is not familiar at all, or everyone in between, we want the piece to just be exciting enough. And we have a snap lens in conjunction with this. Veronica Flint, our technical director, designed this great lens where your eyes get green. We want people to be commercially excited by this, and at least enjoy the visuals, and know something's going on, or they love the song, and then that makes them go, let me watch it again, let me watch it again. And the beautiful thing with AR, and this being an app that you can download, is that you can watch it as many times. It's not creating a separation or a hierarchy of who gets to watch content. You know, it's accessible, and we want kids, adults, anyone, to just have fun with this piece. and then maybe go read the play or, you know, watch a movie version of it or be inspired to look at a different Shakespeare they had seen and switch the gender of one of the characters to fit more like them. Because for me, the more I work on this role as Iago as a woman, the more she's become like me. You know, this piece is about Iago, but it's really about Mary Chivo's experience in the world as a queer woman who has a lot of power and wants people to pay attention to what I find important, which is equality and inclusion and genuine diversity. I have a dear friend who says diversity without inclusion is tokenism. So, you know, I think that that's obviously what Juvie embodies and what Josh and I try to embody in our work is really genuinely including everyone and the diversity of insights and opinions is really what we're looking for. It's not about the facade, it's about what do different perspectives do and why it's so great to have a co-creator like Josh because we do really ebb and flow really well together and we have the concept for the piece and then he has his directorial eye and then I have my perspective on the character and it's just really great to have that symbiotic relationship that then expands outward to 3DR who we're working with as animators and just kind of all the different departments that we're working with. Q Department, who did our sound design. Koosh, our music producer, you know. Thank goodness we have so many different perspectives, because it just makes it so much more interesting.
[00:12:52.146] Kent Bye: Yeah, 3D AR, who did Paper Birds, and Gloomy Eyes, right?
[00:12:55.167] Josh Nelson Youssef: Gloomy Eyes, yeah. No, they're incredible to work with. And I think it's exciting because the medium lends itself so well to theatrical spatial storytelling. I mean, theater creators and artists have been telling stories in space for thousands of years. So rather than being, you know, trying to adapt from being limited to a rectangular frame to having this virtual space, it was great having just embodied characters performing and being able to do this real-time motion capture and lend that to 3DR and have it be this organic riff between the ancient and the bleeding edge of storytelling. And, you know, I think Theater has been so privatized and exclusivized and inaccessible, and Shakespeare I think even more so. Inaccessible in the sense of, you know, just getting to the theater, but also just in terms of the comprehension of the text and sort of this high-nosed snootiness that can be around, like, preserving, like, the Shakespeare tomes of scripture. But really looking at the text and the ways in which so much of what these plays are doing speak so directly to what we're experiencing today. And then being able to find new voices and perspectives, like getting to hear Mary's interpretation of what happens here when we have this play that deals so much with jealousy, but it's Othello's jealousy and it's, you know, we have that term, the green-eyed monster that comes from this play, really exploring and like, well, let's expand on that, let's figure that out. And being able to create a piece, you know, it's mobile AR, right? So it's a short, we had to keep it contained and it helped to have a driving song But hopefully it has enough onion layers where you can kind of go through and get more sense of what's happening and using these hologram images that Iago casts as her own memory. So she's going around, she's casting these different holograms, and these are all different things that are pulling from the play. And yeah, like Mary was saying, a reimagined way to visually depict in AR the concept of PTSD, of microaggressions. leading to her own jealousy in the manifestation of this literal green-eyed monster. But yeah, it's been a lot of fun to get to play in.
[00:14:56.964] Kent Bye: Yeah, I'd love to hear your take on setting the context for Iago and the character of Iago's relationship to Othello, and just to help set a baseline, because there's stuff that I'm not quite sure how Iago fits into that play, and then where you're starting from with this as a piece. Because there's a moment where the characters have the names above them, Iago and Othello, so you're telling the audience that these are who these characters are, if you know all the backstory.
[00:15:20.730] Youssef Mary Chieffo: Yeah. Well, yeah, I really feel, again, something that really struck me in doing a more traditional version, but looking at Shakespeare's day or how these characters would have had a hierarchy in the military, as Shakespeare said it, and how that, I think, by making Iago a woman, again, augments, pun intended, the themes of the piece, is that Iago and Othello, in my mind, I mean, there's stuff that's alluded to in the text, But to me, as I was just creatively investigating, imagining her relationship with Othello, is that they were peers and that he was a mentor to her. And so for him, and in this case, Iago being a gay woman and Othello being a black man, there's kind of that like, hey, we're supposed to be allies here. And for her, there's an extra layer of feeling spurned by him then promoting this classically handsome, white, young man, Cassio. instead of her, after she's been working much harder. He, in our mind, he just graduated from whatever high-end school, has like lived a life of privilege, whereas she, to me, in my mind, came from no privilege and has just really worked her way up, and this is her way of surviving. And our overall idea of color and gender conscious casting is it's always a balance because you want to show representation is so important, but we can show the most positive versions of things, but we also have to look at our world today, which obviously has become more and more relevant by the day, how as a society we're reflecting back on our history and how we've told it and how, to me, neutralizing or blindly casting things can make it seem like everything's fixed. And what I hope to do with this piece and just my overall concept for this Othello is show how it's so much more infectious and that everyone is affected by racism, by misogyny, by homophobia, that the world we live in, we all have these buttons that can get pressed. And so the investigation of Iago, like I said, is so personal because to me it's like she's who I could be if I didn't go to therapy, if I didn't have great friends, if I didn't have a family that I could talk to. All these different ways in which, or even my theatre training of just like evaluating myself. as a human and I've been lucky enough that I've been able to look at my demons and look at my monsters and channel them in my art and that's what I've done with this piece so to me Iago chooses the villain path because she's not given the opportunity otherwise so she is able to then vilify Othello for herself even though she loves him and I think throughout the piece even when I did the full production as a man It's a tragedy. I think Iago doesn't even realize how easily he can make Othello go mad. That he doesn't realize how cruel the society has been that Othello can so quickly turn and say, you're right, I'm sure she is cheating on me, which is tragic. And in those scenes where I would watch Othello go into what Shakespeare calls madness, and there's plenty of different terms I think in our modern world we could use, I was always like, I've gone too far, but I can't step back now. Unfortunately, I think Iago's too much of a coward. And if you look at the play, he disappears for a huge chunk of the second half of the play because the cogs have been set in motion. And some could say he sets it in motion, goes, OK, great. In my mind, he's going, what have I done? There's no way to stop it. I'm just going to kind of let this, you know, hopefully it won't get too bad. And in my mind, Iago never wanted Desdemona to die. He only wanted Othello to feel the jealousy and the rage that he feels, or now she feels. Long-winded answer to your question, but I think that, to me, it is all of that. That the relationship between Othello and Iago is so interesting, and interesting too in that So often, in the past, when it's been two men, often people have said, Iago's in love with Othello. There's all this sort of stuff. So I've still kept Iago gay, just now as a woman. And what is her love for Desdemona? And as someone who looks at herself as a panromantic person, I think that there's a certain sort of affection that she has for Othello that she can't understand or is frustrated with or doesn't know what to do with it. And again, the hope is that this is about looking at when we don't give ourselves the space to channel our feelings, how it comes out. And it's a time of peace. And these are soldiers. And I think Iago definitely doesn't know how to behave when she's not on the field. And I think Shakespeare does that really excellently with all of his pieces dealing with people who are in wartime or out of it. It's a lot of PTSD. Obviously, Macbeth that I did, I investigated that deeply when I played Macbeth as well. He really writes it well. And yeah, so I like Shakespeare. Josh, anything to add?
[00:20:10.740] Josh Nelson Youssef: No, I mean, yeah, I think you said it well. You know, just like in its simplest form, Othello is about jealousy, right? It's Iago, this classic villain who is really poisoning Othello, the main character's mind, into believing that Desdemona is not being faithful to him. drive some crazy with jealousy, but it's all this crazy concoction, obviously gross simplification of that, but it's been really exciting, like Mary's saying, to be able to, we get hints in the text as to why Iago is doing all the crazy conniving schemes that Iago's doing. So being able to have this female perspective, being able to back up to the moment before and look at this, like Mary was talking about, this promotion of this way less qualified Casio character, That's really what this piece is. It's the no good deed from Wicked, origin story, rock opera song, where we get to see the formation of this interpretation of Iago. Yeah.
[00:21:07.266] Kent Bye: Yeah, that's really helpful, because as I watch it the first time, I'm just, as an XR journalist, looking at all of the structure of the forms that you're using. It's sort of a tabletop with different layers, I'd say, like the far layer, the middle layer, and the close-up, and then also using a lot of theatrical lighting to focus attention. But then, the second time, just focusing really on the lyrics, there's subtitles at the bottom, so I could, like, say, okay, what is actually being said here? And then trying to understand what's being said versus what's happening in the show, and then Now talking to you and getting all this whole other layer of the onion in terms of this whole backstory of the Shakespeare context, I have to go back and watch it again just to see. But I feel like the phrase that's coming to mind is like a spatial poem. Like you're taking something that's a very established story, but you're taking a poetic take of trying to interpret things so that when you watch it, you maybe get a sense of what the story is about, but I wouldn't be able to articulate all the things that you're saying about what this story was just by watching the AR experience. So this is sort of like a thumbnail or a poem that's too much larger that the form of AR is able to create more of a poetic spatial take of something. And if you have that additional context or information, you can maybe dive deep. But it's still just on its own a banger of a music video that's really a great song. And it's watching a music video that's in a spatial context on your tabletop. Anyway, that's, as I was like, watching this experience and trying to unpack it, that's the different phases that I've been going through at least, so. Love to hear any reflections or thoughts on that.
[00:22:29.802] Josh Nelson Youssef: No, again, like that's exactly kind of what we're hoping for, where it can be on its own, just like a banger. It's a bop of a song. You're like, whoa, I might not have any idea what the hell's going on, but this is really cool. But then be able to open the door to a rabbit hole where you can sort of go in and understand a little bit more of what's happening contextually with all the different layers. I mean, there's some really nerdy, fun Shakespeare Easter eggs. I can let Mary talk a little bit more, even just like from the lyrics. But yeah, that's kind of exactly what we're hoping for, because there's not a whole, I mean, again, we don't want people's arms to get tired holding up an iPad for a three-hour play. So it's like, well, how do we, like, introduce some of the core concepts, some of the core characters, at least to sort of, like, whet that appetite, open the door for curiosity, because there's so many different layers of conversations that are embedded deep in the bowels of the piece.
[00:23:18.158] Youssef Mary Chieffo: Totally. And yeah, we have our website, greeneyedmonsterar.com, correct? Yep. Good. That we have a little bit more of the backstory there. And just in general, we want this to be a conversation piece. Like that's kind of always been, you know, we want to give that flavor. Like I said, we have our snap lens. That's fun. And you get your green eyes and like, so ways to get people jazzed and then investigate further, which I love that spatial poem. So great. I'm definitely going to borrow that. Because that's exactly it. Like, how can we give this flavor and then get people to look further? But yeah, I think my favorite Easter egg, lyric Easter egg, is, oh, these men, these men, these men, which is the end of the first verse that Iago sings. That's derived from Desdemona's line near the end of the play where she says, oh, these men, these men, which I just love. Like, one of my favorite parts of Shakespeare is there are these lines that feel so contemporary. Like, I mean, oh, these men, these men. Like, it's just so prescient. And so when we were putting the lyrics together, I was like, oh, if we can get that in there and then have Iago, who is now a woman, saying it, how cool is that? And then, of course, our chorus is the exact line from the text. And the bridge is also part of one of Iago's soliloquies, where she decides, OK, I'm going to be the villain. And then, yeah, we have fun Easter eggs, especially near the end as Iago is infecting the holograms with her new power, that there are moments that actually do end up happening in the play, where she gets Cassio drunk and she causes tension between Othello and Desdemona. So hopefully, visually, you go, oh, it looks like she got this guy drunk, or, oh, it looks like she made this couple upset. So if you can get that, that's great. But if you know the play, you're like, oh, my god, that's that part. So that's really the fun. And Wicked is a great example. Someone can watch Wicked and have never seen Wizard of Oz and still have a great time. I grew up with Wizard of Oz. I actually, for a period of time, watched it every day as a small child. So when I saw Wicked, it was also very exciting because it was the first time I was seeing myself on stage, really. And as we're standing here, literally in green glow, I think it all tracks real well. I've made it so that I am actually green during this whole exhibit. But I do just love that that's what this piece can be. And to hear you say that is so thrilling because that really has been the goal always is how do we just let this spark a whole conversation or just an investigation further? And then again, we love music. We love creating these songs. And it's just like so fulfilling to have things that we're passionate about creatively be exciting for other people.
[00:26:00.377] Kent Bye: I'd love to hear any other reflections on the language you used to describe the space, if you think about it in different zones or spheres, because it feels like with AR you have the tabletop, but there seems to be kind of like orchestration that's happening that feels like it's borrowing elements from theater, especially with the lighting that you have in this piece. But I didn't know if being able to have like a radial sphere out with monsters coming up from the back or things coming in and out, but how do you describe this set that you've created here in this piece?
[00:26:28.701] Josh Nelson Youssef: Yeah, well, I think one of the most exciting things about Shakespeare, or any classical text that's been performed spatially, we don't have the blueprints of how things were blocked, how it was staged, how it was lit, if it was, you know, to whatever extent. So it's really incumbent on the artist to make these extra-textual decisions of how you're portraying what's in the written text, and then obviously those decisions change sometimes, or comment on the meaning of whatever the text is. and being able to create something spatially in an augmented reality world where you're not confined to like a specific theater stage, you know, it's like totally free reign. It's like a little paralyzing but also really exciting and so for us to be able to think about our context of sci-fi futuristic stage, really trying to imagine what is this world, this sort of futurist like dystopian underground bunker military base. Heavily Star Trek, Star Wars influenced production design. We got to think about the layers and also get to play with scale. We knew that the monster was kind of like the money shot when we get like the big monster looming. So we wanted to be that a moment where it's like, whoa, okay, this is like, there's a real sense of scale here. Even just in the visual design, another thing that we're trying to think about is, how do we communicate what's happening emotionally, internally for the characters? How do we convey that in an embodied way? What's the character design that lends itself really well? We're thinking about, for instance, 3DR who did Gloomy Eyes. In VR, they have very big heads, very big eyes. You get a lot of emotion that reads really clearly through that. But they also have the benefit of being this black box where it's very clearly visually guided and directed. And that was one challenge that we found. It's like, well, we have a smaller scale that we're dealing with. We can't have large models with large eyes. So being able to have these elongated figures where, you know, really like, so movement became a huge instrument in being able to communicate what's happening emotionally. which was so fantastic to have Eve Jacobs as our choreographer and work with us in getting the timing and the beats and figuring out, you know, which index vectors are helping, from a blocking standpoint, point the viewer into the right direction, what staging makes sense for, like, holograms being in the foreground rather than the background. So we wanted the right balance of being able to guide viewers with the iPad to move enough to see what's happening, but also not have things like way too compact where it's like, well, this should have just been a flat short video or music video. Yeah, so it was a lot of iteration and we did a lot of different scale tests, a lot of different animation tests, and just a lot of iteration and a lot of play. There's a lot of play in getting to see physical relationships and you know I think that's one of the cool things is being able to see creative push the tech and the tech push the creative and getting to do live real-time mocap with the models that we were actually going to end up with and being able to see that in AR while we're filming being able to look on the monitors was so invaluable it was incredible so you know even five years ago we wouldn't have necessarily had the same pipeline with the XM suits and the real-time capture
[00:29:32.683] Youssef Mary Chieffo: Yeah, the Xen suits were incredible because we really got to work in this black box space. I didn't have to be beholden to all the orbs and dots around us. And Veronica Flint, our technical director, is just a wizard and just really set us up for success. And it was so cool to see how all of our capture, the data was so helpful to the animators. Like when it comes to being innovative and collaborative, It's just thrilling where it wasn't just for our benefit so we could be in the space. It was like by giving them the blocking we imagined so fully, they could then really take the animation to the next level. And then, you know, fun to then refine from there. One moment at the end, we hadn't envisioned the floor breaking and her rising with Othello. We had had the moment, the physical choreography, and I remember Julian at one point in a Zoom was like, we're thinking this, and we're like, yes, no, that just augments what we were already thinking. So I just loved the ebb and flow of setting the space and discovering this dystopian world. And then obviously we're in this gorgeous installation here, but what's great is that the way this stage is set, I think it really allows the viewer, wherever they are, if they're on their bed, if they're at their kitchen table, to be immersed in the world. But we had someone yesterday just saying how it's fun that it didn't bother him to see people walking behind because it's such a stage. that is like you still thought you were in it and it's kind of cool to realize someone's walking behind you and you're fully there. And really, to quote Shakespeare, all the world's a stage and the accessibility we've been talking about, that's what AR really allows us in such a cool way. And I think it's been really fun for us coming from working in VR as well, which is awesome, obviously, in and of itself. But I think we've been inspired to see how AR may be a really great fit for our aesthetic.
[00:31:25.887] Kent Bye: Yeah, I liked how your installation here, you have basically recreated the whole stage in a way that's like a little miniature. But then a lot of it at some point gets kind of replaced with the virtual context. So when people at home, they'll be able to get the sense of that with the characters walk around on a table at first. I guess one other final thought in terms of just my experience of the piece is that I feel like when I'm watching AR pieces that I'm like capturing a virtual realm but I'm like the center photographer of something that's happening and I'm like I'm the camera person and I get to see what is the best angle and I did notice in this piece how I did feel like I wanted to get really close up into the characters and then there was that moment where I had to kind of like go backwards and so the first time I think I was just trying to get a sense of like the large picture because I didn't want to miss anything. And the second time, I was like, OK, I'm going to follow my cinematographer instincts. And there was that moment when things sort of changed scale, where it's like, oh, wow. So it's like your use of scale then makes me have an embodied reaction when moving backwards that I think adds this additional element of, like, if I were actually recording something, then I would be having that cinematographer reaction. So I think there's something about using the phone and people to be embedded into what's it mean to be able to recording a moment. So this is a moment that's unfolding, but they're holding the phone. So it's sort of setting the context for that. But it's rather than just any mundane moment, it's like this magical moment of a story that I feel like is using a lot of the techniques of AR. So for me, I've felt a lot of times like VR versus AR, a lot of VR, just like LumiEyes or Paperbirds as an example. It's like the tabletop aesthetic, sometimes I just prefer to be completely immersed. So I've been a little bit more skeptical about AR storytelling in general, but I feel like starting to see some aspects of like this short form, three to five minutes of like a song based or just ways of getting a little taster. And if people want to sit down for a longer version of the story, maybe in VR, but that this is like a way to take what technology is already out there and kind of leverage people's experiences of what it means to capture moments, but using the immersive spatial technologies there.
[00:33:24.644] Youssef Mary Chieffo: I mean, yeah, that's just so great to hear. Like everything you're saying, I'm like, oh, good. That's been, I think, so much of our goal is to see, yeah, which mediums cater to which aspects. And, yeah, I mean, to do a fully VR, like, extension of this would be so exciting. And, again, with the data that we have, you know, there is that potential. So, yeah, it's exciting to feel like we're on the precipice of so much with this project.
[00:33:48.160] Josh Nelson Youssef: Yeah, I mean, I think this is all anticipation of Web 3, the spatial web, being able to have more consumer-friendly AR HMDs, right? Like, anticipating that reality, I think. Hopefully this sets at least a thesis statement that there is potential for AR spatial storytelling that can be compelling, and we can really bring great experiences into something that could be like, you know, an Apple AR headset, I think. You know, as we're still trying to figure out, and I think Oculus has done a great job proving out within their own ecosystem all the new possibilities of storytelling, but I think being able to extend that into the AR space in anticipation of that coming reality is really exciting for us. So hopefully it just spurs on more creations in this space.
[00:34:32.822] Youssef Mary Chieffo: And I'll just say too we're so grateful to partner with Verizon who is willing to take a risk on something really creative and innovative. I mean and I don't want to say it's a risk because I believe in it. But you know it really means so much that their team wants art to be a part of their own 5G, and it's just been really excellent to work with a group that's supportive of our vision, and Chris Sumas, who, she's just been such an excellent ally in recognizing, as a woman, in a powerful position, you know, it was so great to be able to speak with her about that, and Chase, who's been our awesome advocate and power source in just seeing, oh, this is an interesting story to tell, how can we collaborate on it and that's just been a really awesome partnership and I hope that more brands do that. That's how the art survives is by those who support us and so it's exciting that we can really not only cross over immersive and theater but commercial and that it can all have integrity is really really thrilling.
[00:35:37.965] Kent Bye: Maybe you could give a bit more context with Juvie Productions. I know Viola Davis is an Oscar-winning, Tony Award-winning, Emmy Award-winning actress. So maybe you could give a bit more context of Juvie Productions and what other projects or intentions you have within the XR space.
[00:35:52.363] Josh Nelson Youssef: Yeah, absolutely. So, GP Productions, Viola Davis, Julius Tenen, it's their production company. We have a first look deal with Amazon and there's three departments. There's film, TV, and immersive and interactive. So, done a few pieces, The March, Operation Othello, Songs of Infinity. But yeah, the mission statement of the company is to provide a voice for the voiceless and really, in a meaningful way, bring voices that historically haven't had a seat at the table at the fore. And so in the immersive interactive space, yeah, it's a lot of really exciting video games in development, interactive VR, AR. A lot of it stems really just from the fact that Viola is a huge VR lover and also just the crossover of seeing the potential from theater into the immersive space and just seeing the connections in the spatial storytelling has been really exciting. But yeah, she had her Gear VR back on the set of How to Get Away with Murder and that was like her escape in her trailer. And same with Julius as well. It's been really exciting to see their enthusiasm to really innovate in a meaningful way in the space.
[00:36:52.806] Kent Bye: Oh, that's really cool. I didn't realize that she was into VR back in the day, so I'll have to catch up with her at some point.
[00:36:58.050] Josh Nelson Youssef: Yeah, it's been a minute, but it's been really exciting. Yeah, a lot of really cool stuff in the works. Really excited to get to work with Juvie.
[00:37:04.376] Kent Bye: Awesome. And finally, what do you think the ultimate potential of immersive storytelling and augmented reality might be, and what it might be able to enable?
[00:37:13.343] Josh Nelson Youssef: Yeah, I think that's a really good question. I have a few hot takes in this regard. So personally, love my meta quest. I still use it. I love a lot of the made for VR experiences, but I also watch a lot of 3D movies on my quest on the plane. And I can imagine, you know, an environment where if we move to AR HMDs as our primary method of computing. There's still, I imagine, a massive diet of 2D fly imaging that we'd watch in those headsets, but I also think that there will be great potential for spatial experiences, whether those be live. I can imagine being able to see productions that are live mocap that are interactive. or just really beautifully crafted, curated pieces with high production value that immerse different viewers in the space where they can kind of watch something simultaneously, whether that's tabletop or projected onto a larger screen. But it's exciting, you know, it feels like in some ways we've got so much amazing tech that's available to us right now with ARKit, a lot of great VR HMDs. But I don't know, I wonder if we'll look at this time and compare it to sort of like the Palm Pilot days, where we're like, oh, we're, you know, still kind of waiting for the 2007. This is sort of the iPhone that totally translates us from Web 1 to Web 2. But I think, you know, there's exciting potential to dream about what spatial storytelling will look like in the future. Yeah, really I think I have great hopes for the revival of theater and being able to make it more accessible as devices become more accessible and bringing embodied stories to more consumers.
[00:38:47.255] Youssef Mary Chieffo: Yeah, I mean, seconding everything Josh said and just adding as a performer, you know, I worked a lot in prosthetics on Trek, but being able to be in the suit, the XM suit, and get that motion capture, we got to work with actually another Juilliard classmate, Justin Lawrence Barnes, and then my dear friend from elementary school, Eve, who's a great physical actor as well, Caitlin Hunt, our VP as Othello, and then Eve Jacobs, our choreographer, I mean, these are all theater-trained people, and everyone was so excited and inspired to be able to get in these suits and really move their bodies. And as Josh had spoken to earlier, with knowing the animation was such elongated figures, we knew that the communication had to be physical. And again, doing the whole first minute and a half, two minutes in real time, that took theater training. And I think it's whatever theater training means, people who know how to use their bodies. But to me, it's just inspiring. And if I, as an actor, get to keep working in these spaces, using my physical self, my vocal self, creating these really exciting, maybe shorter pieces, these poems, That's so thrilling. Like, I think a lot of actors are very eager to perform, and it's just a great new way to, I hope, employ a lot of actors who can really lend that side of themselves. Yeah, so that's just really thrilling, too, that on the performance side, I think it's really just on the precipice of being extremely fruitful.
[00:40:13.665] Kent Bye: Awesome. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:40:19.025] Josh Nelson Youssef: I'm just so grateful to get to be a part of this project. There's so many people that made this possible. It would be impossible to thank every single person. But yeah, like Mary said, we're so grateful to the Verizon team, to Aaron McPherson, to Kristen Sumas, to Chase, to everyone who believed in this, to CAA for helping set things up for Julius and for Viola who have just been the biggest champions of this space and of this project. The 3DR team who absolutely crushed it. Hermann and Heller who jumped right in and has just been a fantastic partner with us. Camila and Julian who were the co-directors on the 3DR side who just like took what we had crafted and just brought it to this insane next level. Steven Paul, Andrew Mertens, our wonderful producer, who was just like the lifeblood of the entire project. Madison Belcher, our associate producer. It seriously would take an hour for me to thank everybody, but just, you know, it really does take a village. And yeah, we're just excited to share this with the XR community and beyond. It's been such a blast to get to work on and a privilege to be here at Tribeca.
[00:41:16.733] Youssef Mary Chieffo: I also realized I need to thank my girlfriend Maddie Goff not just because she's awesome but she was my stand-in because as you know there are holograms that Iago pulls up so that was very fun and she also taught me a thing or two about punching bags because she has like much more background in punching things. So I feel like ending on that note sounds really good. But truly, as Josh said, Takes a Village and everyone who also, Maddy, who has also listened to the song ad infinitum. And everyone loved it. Oh yeah, Khrushchev, our music producer. I mean, it is amazing how many people put their heart and soul into this and care about it. Q-Department, who, like, when they... What's so cool is all these different aspects, when people hear the concept and the idea, they're so excited about the story. And as a storyteller, it's so fun, because I come from the acting background mostly, and then, you know, have my eye on larger creation. But to see all the different ways someone can be inspired by a piece, and then do their strength, you know, is just very exciting. And I really have been so grateful to be on the producing side of this project, outside of creating it and being able to help shape a piece. you know, it's been empowering to me and I'm excited to do more of that and hopefully in the immersive space.
[00:42:28.650] Josh Nelson Youssef: We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got to talk about the sound. I'm so sorry. We've got
[00:42:48.086] Kent Bye: So it does have an ambisonic mix, or a stereo, or spatial mix?
[00:42:51.588] Josh Nelson Youssef: There's a spatial mix, yeah. So the song is non-diegetic, so that's like, you've got a really good stereo mix. But all the different holograms, all the different movements, you know, the footsteps, like, there is a spatial mix. But one of the really fun things that we got to play with, that we were curious about at the beginning, was we wanted it to feel like a musical, like a theatre performance. So we had this question of can we directionally have Iago singing where like as she's moving around and if you pay attention like if she's out of frame there's a decrease and there's a spatialization. So you have all of these different elements that are rhythmic and I mean they're in many ways collaborators in the song because they took a very musical approach to the sound design. so it all layers in rhythmically, but isn't overwhelming. You're still able to hear what's going on. And to be able to use it as a directional tool if you're too close into one object, being able to hear when you have headphones on, like, oh, there's something going on the left. So yeah, it was a really fun challenge, and we're just so thrilled with what Q Department had built. Definitely want to make sure we get to it. Yeah, they crushed it. And Kush Modi, as Mary had mentioned, our music producer, who's just Absolutely phenomenal. Yeah, absolutely crushed it with just incredible layering and production.
[00:44:02.521] Kent Bye: It's showing here at the Tribeca Immersive. There's a download link that I got from signing up. Is it publicly available now, or is there a release date?
[00:44:09.790] Josh Nelson Youssef: It's all publicly available at greeneyedmonsterar.com on both Android and on iOS.
[00:44:16.588] Kent Bye: OK, well, I will shout from the rooftops for people to go check this out and to have people see the experience and then listen to this as a way to kind of digest it and unpack it. Because I think there's a lot that's going on here to kind of bring all this stuff together. So yeah, like I said, a spatial poem that's many layers that you can keep going in deep and deep and deep. And yeah, Josh and Mary, thanks so much for joining me today on the podcast to help unpack it all.
[00:44:37.923] Josh Nelson Youssef: Thank you so much. It's such an honor.
[00:44:39.985] Youssef Mary Chieffo: Yeah, such a great conversation. It's just so great to hear all your thoughts and, yeah, share a bit about ours. Thank you.
[00:44:46.977] Kent Bye: So that was Josh Nelson-Yusuf, he's a co-creator and director of Hayago, the Green-Eyed Monster, as well as Mir Chifo, co-creator and performer within Hayago, the Green-Eyed Monster. So, I have a number of different takeaways about this interview, is that first of all, well, the piece itself is, if you watch it, you may not quite understand everything that's going on. One of those challenges of storytelling within augmented reality is that you can digest material that's coming from other places. So in this case, it's taking a Shakespeare play of Othello and focusing on one character of Iago and the relational dynamics of Iago as Aigo is going around and infecting people with jealousy and you see the character names, but if you're not familiar with Othello and these characters, then there may be a lot about this experience that kind of goes over your head. But at the same time, there's lots of ways that they're playing with scale. It's a tabletop experience. You're seeing characters move around. They describe it as a musical AR rock operetta. It's an animated villain origin story for a Shakespeare-infused project. And so because of that, there's like many layers of the ending that you can start to peel off. And there's a lot about this experience that I heard from the creators that I would have never picked up just from watching it. And so I think that's the interesting thing is that they're taking existing media where you could watch this piece or maybe inspire people to go watch this piece if they're not familiar with it, and then start to unpack each of the additional layers of a piece like this. So. Yeah, it's a great song, short little piece. I recommend checking it out and very much enjoyed hearing a lot more from both Josh and Mary about their process and trying to reduce down this, it's almost like a character study of looking at a very specific character and then some of the different dynamics of that character and trying to describe different aspects of PTSD, microaggressions, and doing a gender swap, and you know, lots of other aspects that if you're not familiar with, then you may have missed it, which, you know, I put myself in that camp as I went through this experience. So anyway, that's all I have for today, and I just wanted to thank you for listening to the Voices of VR podcast, and if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
