Queerskins VR is an embodied story that explores the range of experiences from love to loss, and the shame, regret, and fears around sexual identity. It’s got a magical realism aesthetic that transports you back to the 80s AIDS crisis as two devout Catholic parents drive to the grave of their gay son. You can hear their conversations as you rifle through a box of diary entries and objects that represent a lifetime of memories. It uses a wide range of volumetric capture approaches from Depthkit to photogrammetry,and stylized 360 video that results in an extremely compelling and transportative experience.
I talked with Queerskins writer and director Illya Szilak and creative and technical director Cyril Tsiboulski at the Games for Change Conference in June 2019 about their journey into VR, their production process, their accompanying installation and photo documentary project, and their future plans of other embodied stories they want to tell.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. So continuing on in my series of looking at the VR for Good movement, and specifically looking at some of the projects that have different technological or architectural innovations, today's episode is with the creators of Queer Skins, which debuted at Tribeca 2018. I had a chance to see it at XR for Change in 2019. I've had a chance to talk to the creators, Ilya and Cyril. They're both interactive storytellers. They did an experience called Reconstructing Mayakovsky. And so they've been innovating and trying to figure out ways to use the immersive technologies in a way that are allowing you to use your embodiment and use interactivity in different ways. And so In this experience, you're kind of traveling in a car and you're essentially in the backseat. And in the backseat, there's a box with all these different possessions and belongings. And it's presumably of somebody who has died and passed away. And you get to look at these different objects and see that this person, based upon the context of the conversation in front of you with this person's parents, was a homosexual who was not necessarily accepted by his parents. And the parents are on the way of driving to this person's grave. So it's a very intimate, intense experience, and using a lot of different technological aspects. They're using the DepKit for volumetric capture, and it's very specific aesthetic. And they're also using embodiment in a very specific way. You're interacting with these different objects. So anyway, we'll be kind of unpacking that a little bit more, but I wanted to give just like a broader context of what the experience was. But that's what we're covering on today's episode of Voices of VR podcast. So this interview with Ilya and Cyril happened on Monday, June 17th, 2019, at the XR for Change conference in New York City, New York. So with that, let's go ahead and dive right in.
[00:01:58.294] Illya Szilak: I'm Ilya Silak. I'm the writer director of Queer Skin's A Love Story, and I'm an interactive storyteller working, started online doing interactive narratives and now moving into VR and AR.
[00:02:10.545] Cyril Tsiboulski: And I'm Cyril Cybulski, and I was the creative and technical director on Queer Skin's A Laugh Story, and I run an interactive design studio called Cloud Red in Brooklyn.
[00:02:19.048] Kent Bye: Great. So maybe you could each give me a bit more context as to your background and your journey into immersive media.
[00:02:26.347] Illya Szilak: OK, well, I'm actually still a practicing physician. So I work at Rikers Island Correctional Facility in New York City. And I started writing interactive stories online in the mid-2000s. And then I found Cyril. to actualize it and we came out with an interactive narrative in 2008 called Reconstructing Mayakovsky, which amazingly is still being taught on university level as an experimental narrative game and electronic literature. So that's how we found each other and we've been working together ever since. We moved into VR with something, so I always considered the online interactive narratives to be installations online in a way. And so I always thought spatially about it, then got interested in VR and I was one of the Oculus Launchpad fellows, the first class of people. And I came back from that and I said to Cyril, okay, so how about we find someone who can, because he didn't know Unity at the time, how about we can find somebody who can make this for us in Unity? And he said, well, how about I just learn Unity? Yeah, that's that's pretty much how we got here
[00:03:52.553] Cyril Tsiboulski: So it's actually exciting to be here at Parsons New School. I graduated from Parsons in 2004 from the Digital Communications and Media program. And a couple of years after that, I started a design studio with a business partner. And we've been doing a lot of interactive work. And then, as Ilya said, we met in 2008. And it's just really exciting to work on projects with her because she's, I mean, super smart, but also brings a really unique perspective, I think, to storytelling that I got really excited about helping her visualize it and bring it to life. We usually decide on what tools we're going to use that are best suited for the story. We started with online multimedia novels. They can be experienced in a non-linear way and are just, you know, super engaging and interactive and I think it was a natural
[00:04:44.902] Kent Bye: transition to VR because we're we're really interested in presence and how to make things interactive And I'm wondering if you could give a bit more Backstory and context for queer skins and how that story came about for you. I
[00:05:02.227] Illya Szilak: I'm going to give you the one that doesn't freak people out. I am a physician. When I was doing my residency in the 90s, I actually worked with a lot of HIV patients, and I work with HIV patients now. Sebastian, the character who dies in Queerskin's love story, is really an amalgamation of people that I knew, and also autobiographical sort of information, very personal information that I wanted to explore through the point of view of this other character. So I ended up, everyone thinks he's real, but I think it's because I just know him so well, because I wrote for the online interactive, I wrote 40,000 words of his diary, So, yeah, I know him.
[00:05:50.768] Kent Bye: Yeah, and I'm really struck by the blend of different volumetric techniques because there's a depth kit, really clear depth kit, and then you're kind of like in a CGI car and you're interacting with these objects and going through this person's history through these objects in this box. But also the landscape was amazing just to see how the volumetric feel and part of me was like, how did they do this? Because I've never seen anything that looks quite this good because it's, it's something that looks very smooth and uniform, but also technically I was like, well, how did they even do this? So just curious to hear a bit more about the volumetric fusion that you did here in Queerscans to create the aesthetic that you did.
[00:06:32.840] Cyril Tsiboulski: Yeah, I mean, we were really fortunate to work with a lot of really talented people on the project. We did combine a lot of different techniques and technologies in it, and why not, I guess.
[00:06:46.425] Illya Szilak: Well, we were looking for a magical realist kind of aesthetic, so it just became... really important that we found that sweet spot between artifice and reality and not go too far over into kitsch and also certainly try to avoid that slick CG sort of appearance as much as possible because it's a really profoundly affecting story and that was just going to be the wrong way to go. But we also wanted to recognize that this is, you know, a semi-imaginary space and allow people to sort of aesthetically experience a little bit of transcendence.
[00:07:23.335] Kent Bye: Yeah, and it's like a shimmer or a glow or I guess the quality of light I was really struck by that how you're able to convey that I've never seen something like that in VR because it's something very unique in that way
[00:07:36.260] Cyril Tsiboulski: Yeah, and that was one of the challenges. Our DP, Corey Allen, actually programmed the lights on set to match the 360 video that we shot on location in rural Missouri to light the actors just right so that the exterior video perfectly matched the volumetric assets that we were capturing on a green screen stage. We also worked with Juan Salvo who is a colorist who actually made all of the different elements come together. The 360 video, the volumetric characters, the interior of the car, all of that had to be brought in visually together.
[00:08:15.578] Illya Szilak: And he actually, he added a little bit of green to the video to make it almost feel like you're driving through a home movie. And I mean, for people who know film, I mean, I was really looking at rear screen projection, which is exactly what this looks like, and then really wanted to get a kind of hyper real coloration. So that's, you know, which is sort of classic for like 1950s melodramas. So that's Juan's work also in bringing that together.
[00:08:44.873] Kent Bye: And I think right now in our culture we have, I'd say, an explosion of these ethical and moral dilemmas, of these different worldviews and value systems that are really at odds with each other in some ways, getting to the point of all-out war. And I feel like this is an exploration of ethical and moral dilemmas, especially when it comes to sexual identity and religion. And so, just curious to hear a little bit more about, you know, addressing this very sensitive topic and the way that you went about trying to talk about these moral dilemmas that you're exploring here.
[00:09:20.783] Illya Szilak: So it was really important that we didn't create a story where you could pretend to be someone who you weren't. It was really important to create a kind of story mechanic that allowed you to bring all of your history, all of your biases, all your point of view and your imagination into the story and actually make it The crux of the storytelling is that the visitor gets to interact with this box of objects, personal belongings from the man who's died, and it's through interacting with those objects, photographs in the diary, that they actually come to know or construct in their own mind who this man was. So there's a drama playing out in the front seat, but there's very little actual tangible information that's transmitted. You get a lot of emotion. You can feel the conflict between the parents. You know that the son, you know, they were ashamed of him in some way, but who he actually was comes out through the visitor's own construction of who he was. And so that kind of openness was really important for us because we don't want to tell you how to feel about this story.
[00:10:27.440] Kent Bye: And I'm curious to hear a bit more in terms of the reactions that you had from people. Because as I was watching that, I was imagining if people actually lived through this in certain ways, what their reaction to the story was.
[00:10:41.898] Cyril Tsiboulski: It's been quite a range, you know, from people just prying in headset to people just ripping the headset off and just walking out. And I guess one moment we showed the piece at Seagraph last year, 2018, and one gentleman, an older gentleman, came up to experience the piece and He was actually just kind of standing around and kind of talking to me about games and how long he's been coming to see Graf and probably talked to me for about 20 minutes and then he left without saying anything and then came back again five minutes later and the headset was open so he went in and and then just kind of again walked away. It was just really kind of a really strange experience. And then after he watched the experience, he just disappeared. And then maybe like an hour later, he came back and just gave me a hug and then said, thank you for being here. And that was just so touching. And I don't know to this day what was going through his head or what he was feeling, but it was just, I think it's an emotional experience for a lot of people.
[00:11:46.404] Illya Szilak: So we called it Queer Skins for a reason. One was because I wanted people to have to say queer and skin together and I mean it's also a play on a term for gay skinheads but that's not why I used it. So to some extent it pushes some people away, the name, but it also really acts as a beacon for queer people of all kinds. it's almost like a safe space in a film festival or in an exhibition where people come in knowing that in all likelihood this is going to be okay for them and that has been really heartening like especially at Tribeca I mean so many people coming and then through the crowdsource photography project that we have as part of this sharing their stories with us it was really a profound experience.
[00:12:35.521] Cyril Tsiboulski: Oh, sorry, I was just going to add to that. At the LA Film Festival, we had a young girl who came out to her mother right after watching the piece. She was there with her mom. I think she was probably 17, 18. And yeah, that was really... So after her mom watched it, she came out? Yeah, yeah.
[00:12:53.468] Illya Szilak: Exactly. Yeah, and we've had people who have sent friends who are coming out to this piece because they just wanted to show them that people were talking about these things, were making in this space, and that there were people who shared experiences that maybe they were the same thing that they were going through.
[00:13:14.174] Kent Bye: Can you elaborate a bit more as to what this crowdsourced photography project is that's related to this?
[00:13:19.744] Cyril Tsiboulski: Yeah, as part of the installation, we really like having the experience presented as part of an interactive physical installation. And we've done this at a couple of different festivals. We started at Tribeca and we also had it in Toronto, where we worked with an artist-photographer to photograph visitors to the experience. after they had gone through the experience. And what we did, we asked the participants to select an object from the installation that spoke to their own stories of love and loss. And then we anonymously photographed, so no face, participants with these objects. And the stories have been really incredible. And I think it goes back to what Ilya was talking about. the object and how we construct the identity of this person who died in the story. People really projected their own stories onto the objects that they found and their own histories. And maybe you can talk a little bit about some of the stories that we heard.
[00:14:18.890] Illya Szilak: Yeah. I mean, I think that the objects and having people choose the object and then be photographed with it allowed people to do something really profound, which is to connect their own story to this story that we were telling. And so for me, thinking through how we wanted to approach, for instance, empathy, having this become a natural connection between the story of a family that may be nothing like your own family, that the story of someone dying of AIDS may be no way related to your own life, even on a subconscious level, come to the conclusion that maybe your story of love and loss and their story, this man's story of love and loss, are not maybe that different. And that, I think, to have it be performed by the visitor themselves was really profound. So we had people sharing intimate stories that we just would never have, I mean, we didn't really have a right to, honestly, but they, it gave them something and it gave us something and we were treated everything with such respect. For instance, we had one at Tribeca, we had one, it was towards closing and this beautiful woman came in and we joked about her boyfriend getting lost in some like robot shooter game and she just walked off and and so she went through the installation and VR and she came back with this really like mass-produced clear Garfield the cat comic McDonald's coffee mug from the 1980s and I said I I love this coffee mug, but I just gotta ask you, why did you pick it? Because it's really not, you know, I mean, there's got to be a story here. And so she got photographed with it. And it's a beautiful picture. It's sort of at her chest level. She's holding this glass coffee mug with both hands. She's got black fingernail polish, perfectly manicured. And she wrote, that she picked this mug because her sister had one just like it. And her sister had died of leukemia when they were teenagers. And after the sister died, she felt that she could actually come out as a trans woman to her family. And so this mug was associated with loss, but was also associated with her own liberation. And honestly, it was really hard to keep from crying. And it wasn't just once that this happened. It happened over and over and over again. It was so much that we just looked at each other and we said, what is happening here? And I think that people are so wanting connection. And I think that virtual reality, especially if done in this way and with the possibility of installation, physical installation or physical performance creates novel spaces for intimacy. And I think that that's what VR itself, what we're really interested in doing in VR itself. So the next installation of Queer Skins, we have four episodes scripted. We are actually going to, do you want to talk a little bit about that?
[00:17:27.020] Cyril Tsiboulski: Yeah, I can talk a little bit about what the next chapter is and how that's going to manifest in the installation part. So in the next episode, it's going to be a standalone piece, but it continues the story that we started in chapter one. Mary Helen, who's the mother of the man who dies, starts reading his diary. And as she's reading the diary, we reconstruct one of the scenes in the diary as she's imagining it. And in that scene, her son and his lover meet on a beautiful beach outside of LA, and they start this dance. It's going to be an interactive dance where, depending on how you as a visitor move in the space, the virtual dancers will respond. It's going to be shot with 360 volumetric video. And we're really interested in creating this dialogue between you as the visitor and the dancers. And we're not expecting you to dance, and most people probably won't dance. But we're really interested in how the dancers can affect your movement as well, and vice versa. And we're thinking also, as part of that, creating a performance piece where real dancers can move in the space with the virtual dancers that will be projected onto screens in the space.
[00:18:42.237] Kent Bye: The thing that was really striking in watching this piece was that you're almost creating a new ritual for death and for grieving and literally driving to like the cemetery and you're going through all these found objects and I think the stories that you're sharing about this interactive component of encouraging people to pick their own objects to be photographed with it shows the power of how our memories are projected onto these various different objects and how A box of objects really represents a whole lifetime of memories that have certain connection to people's lives. And so it's very intimate to be able to shuffle through those objects, but to then think about our own lives and our own objects that in some ways capture the essence of who we are.
[00:19:24.323] Illya Szilak: It was really important that we did not have CG objects in this. We used archival objects that were curated by me and purchased on eBay. But the whole idea that we are now taking someone else's important objects that have a history with them, a history we don't know, inserting those stories into our story was really, really critical. For instance, in this installation, we and Tribeca created a set of artist multiples. We bought archival, mass-produced paper objects. In this installation, we have photographs that I bought in bulk off of eBay from the 70s and 80s. Other people's families. People from all over the world's families, honestly. And then hand-stamped I hand stamped. It was a long time. There was a labor involved. Hundreds of these with text from his diary. So those are takeaways that you can bring home this memento with you, which is your experience, but then you also have this other story that I don't even know whose story it is. And I think that playing with those kinds of novel forms of connection and sharing, I think is what we really want to explore in our work.
[00:20:37.782] Kent Bye: Great. And finally, what do you each think the ultimate potential of immersive storytelling and immersive technologies are, and what it might be able to enable?
[00:20:53.465] Illya Szilak: Yeah, I mean, I just gave this talk on empathy, but I do really think that the potential is that, I mean, if you really want me to go down a rabbit hole, I think the potential is that we actually can teach ourselves to think differently. to actually be more open, to be more courageous, to be more accepting, to be more kind, to be more loving. Do I believe that? I absolutely believe it. I think it's going to take some really damn good storytellers. I don't know that we've reached anywhere near the potential that we have. It's a very exciting time to be in that space, because we don't know what the potential is. But just everything I was talking about with body cognition, I think that that actually shows what the potential of VR is, is really rewiring our brains for good.
[00:21:41.150] Cyril Tsiboulski: It pretty much sums it up, if I can add to that. It's brilliant.
[00:21:47.812] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the immersive community?
[00:21:51.714] Illya Szilak: Yeah, be brave and make really, really good stories that push boundaries. And don't be afraid to fail, because you never know what might happen.
[00:22:03.059] Kent Bye: OK, great. Well, thank you so much for joining me. So thank you. Thank you so much.
[00:22:06.947] Illya Szilak: Thanks so much, Ken.
[00:22:08.548] Kent Bye: So that was Ilya Silyak. She's the writer-director of Queer Skins and an interactive storyteller. And Cyril Sibulski, he's a creative and technical director and interactive designer at CloudRed Story. So I have a number of different takeaways about this interview is that, first of all, Well, I think part of different aspects of the story is that you're trying to give this aspect of interactive narrative. And so they're taking these archival objects, they're scanning them through photogrammetry, very polished textures, they really feel like they're these objects that you're looking at, and they, they feel like they're actually from this person's life. did a great job of building out this character. She said she wrote over 40,000 words of this person's diary, this fictional character, who's really an amalgamation of a lot of different people that she's seen her day jobs as a practicing physician at Rutgers Correctional Facility. And so she sees a lot of these different stories, and I think she aggregates them and then puts them into these fictionalized depictions. But You're in this back seat of the car, listening to the parents in the front seat. They're using the volumetric capture of depth kits, so you have this volumetric sense of them, but they're lamenting and talking about different aspects of regret around the different shame that they had around their child, who was an out homosexual, and they didn't necessarily accept him. And I think they're kind of reckoning with that. So you get to know more about the story of this character through reading different aspects of his diary going through these different objects And I think the other striking thing about this experience was just like the installation was really fleshed out I mean just lots of different photos and objects and they had this whole Companion like photo project that went along with it so that people could come out of seeing this experience, you know enter into the story world of the story and then look at different objects that they may connect with and then take a picture of themselves that they could then either elaborate on further or on the back of it would you know have like these photos that were there you know that she got from ebay and other places that then stamped on the diaries with further information about these fictional characters that are trying to tap into these universal experiences. So just thinking about this as a experience that is VR for good, because it's trying to talk about these different taboo topics, you know, for people who are not necessarily like accepted or they're especially alienated from their family. They were talking about how this experience was traveling around and people saw it and they wanted to show their parents. And so imagine coming out to your parent after seeing this whole immersive experience and then being able to see that happen within the context of someone who's already passed away and you're kind of like recognizing the regret in these parents and then being able to then come out to your parent in that way. And what a beautiful context to be able to set, to be able to have different conversations like that. So yeah, just talking about different aspects of sexuality and sexual identification and to be able to tell these different types of stories and try to allow you to connect to archival objects that are just part of the general pop culture and things that you may have an affinity for or identify with, you know. It sounds like they really took a lot of time to be able to bring these different objects in to give you this sense of realism. and the overall experience had this kind of shimmer it was like this magical realism where you could tell it was a virtual reality experience but it felt like a dream-like quality where the word that comes to mind is this kind of glow or shimmer where it had this kind of stylized film grain to make it feel like you're in a home movie which was very surreal. It's like all these different shaders and ways to make you transported into this another time and place. And I thought it did a really great job of doing that and just felt like this really magical quality. And I don't see that a lot in a lot of different experiences. So there's just a lot of really fine attention to the detail. And I think they're really successful and be able to kind of take you back to another time and place. They've been fundraising and actually shot a big part of their sequel to this in the like the Intel capture studio. So being able to actually have like full dome volumetric capture. There was an experience at Sundance last year called Runnin' by Kira Benzing. working with Reggie Watts and a number of different dancers, being able to have lots of people in the space at the same time. So it sounds like they want to start to explore different aspects of embodiment and all these pre-recorded different volumetric captures and how can you start to have these interactive dances with these pre-recorded dances. So excited to see where they take this whole series in the future. Real innovators in the interactive story space with the Reconstructing Mayakovsky piece that they put together. It's this flash site that you can check out. And yeah, just excited to see other ways that they figure out how to include different aspects of interactivity and embodiment within immersive storytelling. So that's all that I have for today. And I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listen-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. so you can become a member and donate today at patreon.com slash voices of er thanks for listening
