All Kinds of Limbo featured the ability to show a volumetric-captured, virtual reality performance to 20 users within the Oculus Quest. Nubiya Brandon narrates her experience of being a mixed-race woman in the United Kingdom, and takes a tour through the musical genres of reggae, grime, classical, and calypso as the theatrical staging is shifted through different worlds.
I had a chance to unpack the experience with Brandon as well as Toby Coffey, who is the head of digital development at National Theatre where he runs the immersive storytelling studio. We talk about the evolution of this project, experiential design process, how they designed for large-scale social interactions, and how the National Theatre started an entire publishing arm in order to publish projects like All Kinds of Limbo, which can be heard on Spotify.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
The National Theater collaborated with Accenture Interactive, and here’s a video they produced about All Kinds of Limbo.
Here’s a behind-the-scenes video from the National Theatre
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye and welcome to The Voices of VR Podcast. So continuing on in my series of looking at the XR experiences from Sundance 2020, specifically the immersive storytelling innovations, the technological innovations, as well as the experiential design process. So in today's episode, I take a look at All Kinds of Limbo, which is a performance piece from the National Theater. So specifically Nubia Brandon, she's the writer and performer in All Kinds of Limbo, talking about her mixed race heritage in the United Kingdom. And then Toby Coffey, he's the head of digital development at the National Theater. And so this was a piece actually that you could have 15 to 20 people with an Oculus Quest and you walk in to this experience and it's like a music performance. It's like a music video or like a live performance. You're seeing somebody perform theatrically and you have other people in this social VR experience. You see a white line of you as a body in a shadow and they're trying to recreate this sense of a concert to give you the sense of social presence, but yet. you're all watching this performance that's captured in volumetric capture. But also just shifting from different theatrical sets and so that way it's kind of a blend between a music video and more of a theatrical performance in spoken word. So lots of really interesting things that are going on here and so I had a chance to talk to Nubia and Toby to talk about how this project came about and the evolution and all the things that they were trying to do in this piece called All Kinds of Limbo. So that's what we'll be covering on today's episode of the Voices of VR podcast. So this interview with Nubia and Toby happened on Monday, January 27th, 2020 at the Sundance Film Festival in Park City, Utah. So with that, let's go ahead and dive right in.
[00:01:52.730] Nubiya Brandon: Hi, my name's Nubia Brandon, and I'm one of the writers and the performer in All Kinds of Limbo, a new immersive piece from the National Theatre.
[00:02:02.263] Toby Coffey: So my name is Toby Coffey. I'm head of digital development at the National Theatre, where we run the immersive storytelling studio. And we're here with All Kinds of Limbo, which is a communal VR performance piece.
[00:02:13.696] Kent Bye: Great. So maybe each of you could give me a bit more context as to your background and your journey into this immersive space.
[00:02:20.455] Nubiya Brandon: Wow, I was really brought into this in quite a peculiar way and very luckily. So originally we were kind of going alongside Small Island and there's some history behind that. So Rafi, the composer, was doing some workshops in Calypso. We met on tour and he was reading Small Island and then he asked me in an email to sort of express what my cultural worth was or what I felt my heritage was. From that, I sent him quite a colourful and poetic email and he said, oh, I sent that to the National, hope you don't mind. And that was how I met Mark, who's basically our angel at the National. And initially, I think we were kind of put forward to maybe do some music for the play Small Island, which, if you don't know, is a book by Angela Levy, really incredible writer. And it was about the Windrush generation that came over in the 40s. and the first time that they mixed as well with white communities and their experiences of oppression in that era. And it was a fantastic play. So because they found someone better suited to write the music, and Ben is incredible, we were then asked to do something completely new. And at first, I didn't know what we were doing. Honestly, they said VR, and they said create something, but that could have been anything at that point. And we stood looking at each other on the Thames, sort of thinking, so, uh... I guess I'll just call you with some ideas. We had a second meeting and I think that they were very interested in tying together the Calypso generation of the 40s to the grime generation now in London, which then, when I went back and did some digging, I felt that you just couldn't tie it together culturally, socially, it's just very different. And so I started a timeline with a playlist and kind of went along different eras of music and movements that have happened between the 40s to now and why that makes each group not related to each other, like ethnically as well not related to each other. So in the end after that, after we had like this big history lesson, they just said well why don't you just write about your story? something that I really have avoided for a very long time because it's quite hard because I'm just a Brit. That's it. And I guess for me, I always wanted to challenge that idea of what the English Rose is and what that means to me. Why not be someone like myself? Because I have all the credentials. And so, yeah, I got into it and it just flew out of us this piece and then I'd never worked in VR before like I still have a wind round phone at home like enough said and so for me this was just incredible and it was very bizarre just watching myself perform and I've never seen like obviously you never see yourself to play a gig so it's like my first experience seeing me as a storyteller telling me my own story and And it was really, really beautiful. And the second time around that we've come back for Sundance, we've really revamped the whole piece. So there's been some visuals. We had some great new brains that came into the project that had watched the original and were really moved by it from their own personal experiences in life. But also they were very, you know, they're on the techie side. They knew what they were doing and they had some fantastic ideas. And we really discussed and put things forward. So this time around, it really felt like what we were initially trying to achieve. which is just so much more vibrant. So I'd say that's a little bit of how I got in and a little bit about myself, yeah.
[00:05:47.970] Toby Coffey: So in terms of my journey into VR, actually it probably started with my undergrad degree. So my dissertation, which was in 1994, was about how VR in the future could be used to rehabilitate offenders and treat people with certain cognitive issues. Off the back of watching Lord Morrow Man, which obviously influences all VR ever since. And then, you know, a big hop and a skip and a jump from then to now, being Head of Digital Development at the National Theatre includes lots of kinds of audience engagement. You know, traditionally it's been things like websites, app development, that kind of thing, digital publishing. We started with doing some installation work, I think in 2012. when we had a kind of immersive experience called Me and My Shadow, in which it's a kind of international dance piece where you've got an audience member, one in Istanbul, one in London, one in Paris and one in Mons, and they can all see each other in a life-size virtual world that's projected onto a full-size screen, a life-size screen rather. And that was kind of the first dip into that type of installation work, and then, subsequent to that, we then Actually it was in 2015 we had a new artistic director Rufus Norris and new exec director Lisa Berger and what happened with that is I started looking at 360 film and was really interested that virtual reality was coming around again and Rufus had actually been reading Creativity Inc at the time which is a story of Pixar and he was very taken with the wonderful things that can be done with technology but he says it's very much when you bring the storytellers into that technological framework that that's where the magic happens basically and kind of with that in mind I showed him a 360 film and he he did the film you know at the point in time it was a cardboard it was on my phone it was streaming from somewhere I was hoping nobody was going to text while he was watching it kind of thing because this is my first meeting with the new artistic director And he kind of just looked at me for 10 seconds afterwards and then said, we've got to do something with this and we've got to do it now. He said, you know, this is, we've got to find out what the potential for dramatic storytelling is and what it means for us and what we mean as a theatre as a kind of discipline for that. And quite quickly from off the back of that, he had a show called Wonder.land, which was a musical adaptation of Alice in Wonderland with Damon Albarn from The Gorillas and Blur, Moira Buffini, the writer, and Rufus was directing. And basically it was a musical version of Alice in Wonderland for the digital age. So Alice is actually an avatar of a girl in high school called Ali. the rabbit hole is the internet, and wonder.land is Wonderland. And I said to Reeves, look, this is the dream story world to do a piece of VR, in which he agreed. So we did a, what in effect is a VR music video for one of the songs. And what happens is, you fall down the rabbit hole, land into Wonderland, which starts off quite dark, and then it all gets really bright and trippy. And then you get serenaded by Che, the Cheshire Cat. And it's a four minute ephemeral piece of VR music. And in the show, Ali is getting bullied at school and so she goes and hides in the toilets and goes into Wonderland on her phone. So we recreated the school toilets front of house for audiences to do. So they had to go into Wonderland in the same way that she did, i.e. sat on the toilet. So we had like five toilets and one hand dryer and people queued up to do the VR in the same way that they'd normally queue up to go to the toilet. And we had 90,000 people do the VR over five months, which was just great. And as a kind of litmus test, A, to see whether this type of content works, but also as an audience engagement model, it really, really kind of proved itself. We've done a number of projects since then. Home Amir, which was a verbatim documentary piece about the Kali jungle. And Draw Me Close, which is something that's had a number of iterations. It started, we showed that at Tribeca Film Festival in 2017. It's a collaboration with the National Film Board of Canada. We then showed a second version in Venice, and then we showed the full audience experience in January 2019, and that was a one-hour audience experience from their very original version, which was a six-minute piece. And then that'll have its North American premiere in Toronto at Salt & Pepper later this year, and then we're looking at how we push that out further. But then, yes, I'm very, very fortunate to have a very close relationship with both Rufus and Lisa, the directors of the theatre, and when we knew that Small Island, the play was coming up that Nubia referred to, we said this should be another show that we do something alongside. You know, sometimes we do completely independent work, like Draw Me Close, and sometimes we'll do work attached to a show. And Rufus had said, don't do 1948. He will have done that on stage with the show, which is when that's based. And at the end of the play, there's a child of mixed races born. And he said, actually, what you should be doing is something that celebrates multicultural heritage and mixed heritage and how that reflected in society today. and I knew that this workshop had happened which looked at the musical influence all the way through from Windrush to now and I really liked that concept and could we commission that as a piece of music and that then became the backbone of the project which is fantastic and you've got this musical heritage over several decades in the actual score of the music itself and then Nubia's story as her today and growing up is a really nice juxtaposition because it's two levels of heritage told in a completely different way but obviously work so well together. And then obviously Nubia is an incredible performer and so we knew that that was going to be central to the whole project. and to the audience experience. For me, there were two sides to the project. There was the creative side, which we've talked about and what that would look like. From a technical perspective, I briefed our technical partners, Accenture, with the question in 2019, which is when we started the project, how can we create a communal performance, a communal holographic performance. And I thought VR was where we would end up, but I wanted us to go through a research project process to say how could it be done. And so this is where we've got to. We're in Sundance. We've an audience of up to 20 in Quest headsets. And we did the volumetric capture with Dimension. Then All Seeing Guy, who are very close collaborators from a creative technology perspective, developed the environment with our designers at the National Theatre as well. Obviously, Rafi and Nubia were central to the creative content itself. And we've totally achieved what I wanted to achieve, which is a communal performance. When we showed it at the National Theatre, we were very much rested on the ceremony of performance. So there was a show on the hour, 20 past 22. You bought a ticket online, as you would do for any show. When you put the headsets on at that point, you could hear the orchestra warming up. So there were lots of nods to, this is a piece of performance. And I very much wanted audiences to go away feeling as though they'd seen a piece of performance as much as they'd seen a piece of VR. So actually when we hear people clapping at the end of all kinds of limbo, it's kind of nice because it feels like people are falling into that kind of frame of mind. And what we also have with all kinds of limbo now is that infrastructure is, we always have in the very beginning, this should be something that can be easily tolerable and transferable. So all kinds of limbo. We are looking at a North American tour and international venues that want to take it, which is great. But also it's a framework that we can bring other creatives in and say, look, we've made this type of communal VR performance experience. What might you do if we worked with you on this? So it's a very interesting starting point for new work as well.
[00:13:43.588] Kent Bye: Yeah, no, that's a really good overview of the evolution and history of the project. And for me, one of the most striking things was the social dimension and being able to have this sense of social presence with all these people in the context of volumetric capture and seeing you perform. And there's another aspect where I saw it twice. The first time, there was 20 people, and it felt a little hard to move around. And the second time, it was a little bit easier. And it's nice to do it with a lot of people who are dancing around. And so there's a group energy that can start to form if you have the right audience, and they're also moving around. I think there's certain aspects of seeing the direct feedback for people as they're sort of dancing or the open question of like, how do you encourage people to engage or what's the desired action? But I really love the feeling of the group energy that I got out of that. And it was also difficult for me to parse a lot of the lyrics and the message the first time because that felt like it was so visually overwhelming. I was paying attention on VR stuff and it was almost like, slipping into a dream with all this poetry. And then I was like, oh, wait, I've lost the plot here. I was so viscerally responding to the music and feeling like I was in a music video in this performance that I was like, oh, wait, what's the deeper message? And so I had to actually go through again. But maybe you could give a little bit more context as to that deeper story that you were trying to tell with All Kinds of Limbo.
[00:14:53.363] Nubiya Brandon: Well, just as well, just to go back to that point you made earlier, I think that maybe that's what you're experiencing is that some people are either completely absorbed by like rhythm and visuals, which I feel like that would be just one side of it. And then some people who are more still are probably listening to more of like trying to gather the lyrical content, because it was a mixture of people either coming out saying you're a fantastic performer or that really moved me what you said. So potentially, if I watched, maybe it might be a good experiment. Lyrically, I really had to challenge myself. As I said before, I'd actually really avoided speaking about this for quite some time just because I don't like people asking. It almost seems rude in a way. But I really had to think about where I felt that I'd been put in my life and turned away from by certain groups of people, I think. It's felt quite difficult growing up being mixed race without any real black heritage. And the reason why you hear some things like reggae, it's not actually because I'm Jamaican, it's in England, across the UK, dance halls have always been a thing because Jamaican heritage is such a big part of our culture, like white or black. So I used to go to all these dance halls and I would sing my ones and twos over sound systems there. And so that's where that came from, and I was listening to a lot of things like Lynton Queasy Johnson, his album in 1974, I want to say, Bass Culture, he did with Dennis Bavell, who's like the godfather of reggae, and even Herbie Hancock came over and did a documentary about him and his work and stuff, it was really incredible. And how Dennis does it, so you hear in like the root section at the beginning, there's like first reggae, when I mean like the suit, there's like this climbing on the like rhythm that like Linton does, so like I wrote these monologues and then me and Rafi went back and we were like just clapping into different rhythms. I was playing a lot of vinyls at mine, we were listening back to like different albums and songs like Colton and the Shoes and stuff like that. So it's weird, like Toby said, it's like two stories going on at the same time. I was creating a timeline, but also talking about myself. So, like, say, for instance, you know, we start off with the classical music. I love Vaughan Williams because he just depicts, like, the perfect essence of, like, the British countryside. And I wanted my face as a character on top of that music to show that point that I'm not supposed to be just an urban influence because I'm black in Britain, right? And then after that we get into the roots and it's starting to explain that change culturally that I was talking about. So the Calypso generation, for example, in the 40s, these were all very fairly conservative. They were still Church of England taught because they came from colonial islands. They knew those very straight, narrow values that British people did, because they had exactly the same teachings. And then you get to the roots generation and really they're just like less tolerant of the austerity that they're experiencing because they're really getting the brunt end of stuff. These young, young Jamaicans that are now like cockney Jamaicans as well. So that was like showing that translation like, you know, I'm speaking like this and this way, but then, you know, we get into the root section, you can see the change. But also I was, I guess a lot of the sections of the song were, they probably have more like oomph to them. It was me really being acrimonious. I had like a big chip on my shoulder about a few things and it was nice to kind of like express that to close a book on it, if you know what I mean. And then I had to show some like gratitude in the lyrics to like, My boyfriend who has been my best friend for years and he is Caucasian and he has given me so much confidence in myself and I think it's very natural that a lot of us have built in colourism and featurism. Even if we're proud of who we are, it's just a natural thing to have every now and then because you're told that you're not right or you're ugly in certain senses sometimes. And he was like that first person in my life that really like gave me this confidence and what a beautiful thing because he doesn't look like me. He will never experience my life like that. But for him to always be by me and to take on my pain as his own pain and to help me take that journey culturally will always be a huge blessing to me. And I do feel like that's what, especially in London, all of my friends were all English, but my friend underneath is Vietnamese, Chinese, but he's a South East London boy. It's so normal, we're all so different. So I paid homage to those people. And then, as well, the Calypso song was actually a joke song that was written in my kitchen with my friend Alex, who was in hysterics when he heard it. But that was the first song that we started working on, because I actually wrote it as a joke and then thought, oh, I could actually put this in an album. But then when me and Raf were like, where do we start? I was like, look, I've got This Is Just Perfect, right? And it was a bit of like a tongue in cheek to like, some of the lyrics are like, you know, no king or queen could have said it much better. You know what I mean? Like, how do you know what goes on in our streets, if you know what I mean? you know, take a look at my Spice Racket, it's just about saying how blended and mixed London is, the UK is, I can't just like square it down to London like that, that the UK is, so that's what that song was about. So yeah, I'd really try to fit in as much as possible, but the whole point of it being called All Kinds of Limbo is because there is no real conclusion to this topic. It's a very big discussion that we've really tried to narrow down in 12 minutes that I don't have the answers to, to give you closure on. It's still something that I am discussing with myself and that was the point of calling the piece that. All kinds of limbo. I've always felt like I'm in the middle of everything. On the class spectrum, on the racial spectrum of things, within my family and my personal life I've always been in between things. and very unconventional in life. So I really wanted to push that. And I feel like when people have watched it, they actually are very surprised that people really understood that that's what I meant. And that was really great. So yeah, there we go.
[00:20:44.825] Kent Bye: Yeah, there's a certain amount of poetry and abstraction. It's almost like going into a dream and seeing these different references. And I think a challenging part of this piece is that if someone doesn't have any deeper context around the Calypso generation or whatnot, it's from the United States. It's not something I'm familiar with. And I feel like there's certain aspects where you're starting to give me a direct experience of that. But there's also parts where there may be cultural references that are very specific to a region or a context that may be at the point where maybe I lose the thread about something because I don't understand the reference. But overall, I get the deeper sense of what that project was about, that exile or alienation that happens. And that, I don't know, it feels like it's this reclamation of trying to communicate these different aspects of culture. Taking that more poetic approach allowing people's imaginations to fill in the gaps and I feel like that's a part of that But it's still this challenge of you know how specific to a specific regional culture is something that
[00:21:40.290] Toby Coffey: Yeah, it's a very interesting point and we went through a number of iterations of how we saw the project working. At the end of the day, what we want to do is give audiences a performance that is enjoyable and this comes back even back to the very first Wonders.land experience. Finding out what is the right level of information that people can take on board in a VR experience because the experience itself is quite overwhelming and you've got the space to move around etc. there are so many, many messages in it and there is a lot of history. And so what we're going to be doing for the tour is really blowing out the context, but not within the VR. There'll be peripheral information around it, both in terms of website material, we'll probably do a couple of radio documentaries, look at the musical evolution, which is so interesting in its own right, the kind of cultural, societal evolution. There's so many things, you know. And actually, you know, what's interesting about showing it in North America, this is based on migration, a migration issue that started in the late 1940s and is still relevant and awful today. And, you know, to be coming into North America with an issue about migration like that is very prevalent, I think. And then also the history of UK black music is just not known in the States, as, you know, Nubia told us. So to bring both of those things over and not do it in a preachy way, do it in a way that's kind of spearheaded by this incredibly beautiful piece of creative work but that's got the underlying messages. It's just finding the right balance. You can't load that all into VR because that then becomes a documentary and that's not what this piece is. It's a creative reference to a number of different issues and cultural references.
[00:23:20.055] Kent Bye: Yeah, that's the thing I've been noticing in these different 360 video pieces is that once people experience them, then they have a broader context to be able to dive into different aspects of that experience. And so to be able to share different aspects of your culture like that, and then maybe set a deeper context for people to either ask more questions or to have an embodied experience that gives them these primary metaphors that allows them to just understand the deeper through line a little bit more.
[00:23:42.246] Nubiya Brandon: Well, just to chime in as well, and just what Toby was kind of saying, I think that like, so on my last visit, so my family are actually half American as well, like mostly like the Bronx and stuff like that, and then they've all spread to warmer places over time. On speaking to my cousins, I realised that same thing, like they don't know a lot, and my cousin said something really interesting to me once, she was like, yeah, that's because black America doesn't care. And I was like, but she didn't mean it in like a, I don't care about you. She was just like, their own issue is so big. But it's funny to me and interesting because our story is the beginning of their story. We have to go back to like the docks in Liverpool where slaves were imported, sold in pens, the ships would refuel and then go to the Americas or go to wherever they're being sent. So that's always been really interesting. We've always been very heavily educated about The slave trade and how it's happened and worked and how America has changed Probably because of our doings, you know in the beginning and all the rest of it But yeah, I just found it very interesting that why not? I wanted to maybe start this discussion and get things flowing and you know educate people not in a way of like Oh, well, you should know about us and blah blah I just would like to build that gap. I've seen more grime artists work with a lot of American artists. Ocean Wisdom had Method Man on his new album. Ocean Wisdom's from Brighton, this double-time MC. And Slow Thai as well, I think that he's doing really well in America as well. And you saw people like A$AP Rocky working with Skepta, who's one of the old school, he's one of the original grime MCs. And Dizzee Rascal, obviously he's in LA. You know what I mean? You see that bridge there. I was trying to open it up a little bit more outside of just urban context as well. I think that as well, I was really up for this challenge. I saw that the film Babylon, which is like, I would never have played reggae if I hadn't have watched that film. That was like Bible for me. And it was about the sound system culture in Southeastern Brixton, like Southeast London and Brixton, Deptford, places like that. So it had like Aswad in the 70s when they were really, really good. Dennis Bavell did a lot of the music. It's a really phenomenal soundtrack, so definitely listen. And even a couple of the characters I saw in Small Island were actually in that film back in the day, and they were shocked that I knew about that. The same person who made Quadrophenia created that film as well. So that had just been remastered and re-released, and it had a really good reception in New York. And I remember saying to Toby, I want to jump on the back of that, because that was one of the things that I brought in as, like, influence and inspiration I asked everybody in our group, can you please watch this? This is like one of the most iconic films about London. It is so raw, how it's shot. It's really amazing. So that was a really big, that inspired me to want to talk about it out here. And, you know, I guess, you know, I've just visited my family for the first time in a very long time. It was like 18 years, like the separation. And it made me, the warm reception that I had, Also, they found me very intriguing, which was really funny. I felt like an alien around them. But it made me want to have that conversation more because I had people that care about on that side of it. They were very interested. It's just like they just didn't know because people just think of us as like the Queen and tea and, you know, like, I don't know, everybody probably thinks we go to a really good school that looks like Hogwarts and like, do you know what I mean? And that's not really the case. It is like, I'm very proud to come from a city that is the biggest melting pot in the world. and not in terms of like, there's nothing segregated about it, like, you know, your rich neighbourhoods and your poor neighbourhoods are, you know, you have a high-rise flat next to like the most pristine like Victorian house, and those classes do tend to mix in the communities, and I'm really proud of that, so I really want to express more about what we're about so that we can like get rid of some stereotypes that may be unhealthy and have maybe caused like a great pause on people not learning this information sooner, so yeah.
[00:27:42.506] Kent Bye: Well, this piece has a lot of really interesting combinations of different genres, of affordances, of different mediums. It reminds me of a music video. It's like theater and performance. It's got the music video aspect of it as well. You're kind of taken through all these different ways of being in a social VR experience, and so you can move around. have a little bit of agency of seeing yourself relative to other people as you're dancing to create this synchrony and group experience and you're embodied in these I guess lines that are white lines that are very abstract but you can just get the outline of that that's another person and yeah just the deeper rhythm and the story that is coming through with the lyrics and the almost like poetry and like a poetry jam with all the different music genres as well. So, but there's also not only the different genres of these things, but the ways that you implement it technologically and it's kind of fusing all these together. So I'm just curious to hear a bit about that process of bringing it all together.
[00:28:35.197] Toby Coffey: Yeah, I mean it kind of. We were aiming for a performance, so they all seemed to blend together quite naturally in a way. It's interesting talking about the avatars and the evolution of the avatars and also the evolution of what the audience engagement should be. Because right at the very beginning people were saying, you cannot get 10 people in a headset to be in the same space together. And I was like, well no, but we're going to get 20. And there were a lot of misconceptions about what could be done, what should be done. And I think this is a real problem that the industry has, is that it creates these myths that people then abide rather than challenge. And first of all with the idea that people said they're just going to be walking in different directions, they're going to bump into each other and all this. If they're walking in different directions, then we've done a really bad job because they should all be focusing on the performance. And the performance will, at any point in time, be at one place. So that's where the audience should be choreographed around that. Which is exactly what happens, and it's fine. And then the other thing is that people can't say no, no people are going to bush up against each other or bump into each other and one day I just came in from work and said you know what I've probably bumped into 10 people and bushed up against 50 on my way in on the tube today but we don't take that as being abnormal we've just become used to it but yet this new technology has come along and we've created this golden rule that you can't in any way bump into or bush up against somebody else so what we did to address that was just as part of the onboarding say you might brush up against someone it's okay don't worry about it and you take that fear out of the situation because the last thing that we wanted to do was have audiences that were afraid to engage with it because they might do something they thought was a new social nomer or something do you know what I mean so once you take that away as well then people can be more relaxed and we encourage people to dance and you know again if you're out dancing you're gonna bump into other people do you know what I mean so it's just to make people feel more comfortable with that We had a lot of discussion about the avatars as well. And what I really wanted to do is have something that represented where a person was, but no more information than that. And with the movement that people do, kind of forwards or backwards, you get a general shift in the movement of the avatars, which reminds you that it's communal experience. But I definitely didn't want to have a full body avatar in which we could see another person dancing because Nubia is the person I want to be the performer, not the guy over there with the crazy dancing. And so I wanted to keep it, that's how we keep it in that way. It felt like that kind of strip of light with a kind of shadow of light on the floor seemed to work the simplest and most elegant. Nonny de la Pena came and did it, and the first thing she said when it started was, oh my god, the avatars, which I thought was going to get a really harsh critique. But she said, that is great, that's the type of avatar that you need for a communal experience. Also, the height of it is at chest level, so there are no sightline issues. If you've got 10 people all stood in the line, the person at the back of that line can see as much as the person at the front of the line. So on all levels, creatively, it's such been an interesting discussion. I know Rafi, when we were working on the sound development, Rafi is a musician who is used to having his track locked in the studio and it will therefore sound exactly like that every time it happens. and then all of a sudden we commissioned him to make this piece and then bring it into a VR world where the audio has been spatialised and he's like, this is not how I remember. And actually it changes as you move around the space. So it was a very interesting discussion between Rafi, the musician, and Gareth Fry, the sound designer, about what we should be doing and how you allow the music to come through as best it should from the original recording, but also how you let an audience get a sense of this is a live space. And so we were very spatialized at the beginning, and we're less spatialized now, such that you get the whole body of the music, but you do notice differences as you move around the space. So, I mean, all in all, it's been led by audience experience and performance, and then how we make the technology work behind that kind of ambition.
[00:32:35.699] Kent Bye: Great. And for each of you, what do you each want to experience in VR?
[00:32:40.741] Nubiya Brandon: Do you know, this project was the first time I'd used VR and how bizarre like looking at yourself, but I've just like had my eyes opened, especially here at the festival. There's so many interesting ways of like doing stuff. It's like a whole other level. So for me, I'm just going in completely blind to experience like as much as possible. and you know I think that I really would love to see more people telling stories as I've done and I hope to sort of connect with more people who are doing that and you know it's gonna be really hard to like top any music video after that like for me anyway so so maybe if we can just do all my music videos
[00:33:23.340] Toby Coffey: Oh wow, I've not even thought of that before. What would I like to see in VR? This is quite a shallow answer I think, but I'd actually really like to experience a skydive in VR, but with some kind of additional environmental factors thrown in there. Because I would very much like to skydive, but there is no way I'm throwing myself out of a plane. But I feel like if I did it in VR a couple of times, then I might be able to kind of psych up the energy. I think more relevantly social work is something I'm very interested in seeing. A lot of people will bring in an element like social or haptic or whatever because it's an opportunity and you can do it. We very much make sure that the technology earns its place. But I think, you know, coming out of an age where everybody has just complained that VR is very isolated and antisocial, that, you know, I think there's real potential in there for people to be even more social than they could have been before with people that they're not in the same space with. So I think that's going to be a really exciting opportunity across the board, not just in storytelling, but in all different worlds.
[00:34:24.178] Kent Bye: Great. And finally, what do you each think is the ultimate potential of virtual reality? And what am I able to enable?
[00:34:33.105] Toby Coffey: So I would say. You know, I think when we look at science fiction, there's stuff coming, you know, the Avatar chessboard in Star Wars. I think it was the 2020 Japanese games. They'd said that originally they were going to volumetrically capture sports events, and then we broadcast them back out to stadiums around the world. But the technology to do that didn't exist at the time, but people are now actively working on it. So I think there are things like that where you will either be able to turn up to a stadium and see a huge stage show or another piece of performance or bring that into your own personal living space. You know, even just the way people talk about it in all kinds of limbo, the design really opens up your mind in the space and you don't feel like you're in the same kind of enclosure as it were. So my original degree was Psychology, Artificial Intelligence and Human-Computer Interaction. I think from a psychological perspective, well on all of it to be honest, we're just scratching the surface. I wouldn't want to limit it too much by saying exactly this is what I think we should do.
[00:35:31.234] Nubiya Brandon: I will probably just say I was by fate drafted onto this ship and I think that I really don't like have like expectations of it. I think I'm just fortunate just to sort of be on this journey with Toby very luckily. Like I said before, I'm just running into it blind. This is such a new world for me. I'm the sort of person that just played records. I've always been a bit of an old soul, so this is really unconventional for myself. And I've really enjoyed it, so that's just fantastic to have that. But one thing that I'll take away from this piece is just that and I know this sounds kind of vain saying this because it's myself but like I got this experience where I got to like know myself as a performer and weirdly felt like my own pain and I found that so I don't know what people really experience like live shows like when they come to like watch you and I feel like the whole reason you go to see certain artists is because you can see their emotions like oozing out of them and that's what I felt from this and how amazing is it gonna be just to really explore and develop this whole, have we created a new theater form, if you know what I mean, like, you know, a way that you get emotion out from an audience, you know, in this way. Does it change because of the VR aspect? I don't know, I'd love to get into that. So yeah, that's, yeah.
[00:36:49.402] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the immersive community?
[00:36:52.877] Toby Coffey: Yeah I think on this particular project what's also good just to let people know is and you know to your point about like the absorption of the lyrics and the music all in one go as well as the VR we released this as a track onto like streaming channels so it's on Spotify etc etc it's the first release of what would be National Theatre Records which is really exciting so you know this whole thing has been a real kind of creative evolution in a number of different ways which is Really exciting. But yeah audiences can listen to the track all kinds of limbo on Spotify, etc, etc Awesome.
[00:37:25.726] Kent Bye: Well, I just wanted to thank you for all the work that you're doing in this space and yeah Really got this sense of deep social presence and yeah, just excited to see where this all goes with the future of performance So, thank you Thanks so much.
[00:37:37.435] Toby Coffey: Thank you very much indeed
[00:37:38.908] Kent Bye: So that was Nubia Brandon. She's a writer and performer in all kinds of limbo, as well as Toby coffee. He's the head of digital development at the national theater where he runs the immersive storytelling studio. So I have a number of different takeaways about this interview is that first of all, well, just to take a step back and to look at all kinds of limbo and to have this, this social presence of being in a VR experience with 20 other people who are co-located with me in the same experience. I don't think I've ever had an experience quite like that anywhere else to have that many people in VR at the same time. And so people were represented as this white line, and I would have liked to have seen a little bit more movement, and so you get this sense of people moving, but because they wanted to put the emphasis on the performer of Nubia, I think they were trying to minimize the different distractions that you may have from people that are walking around. The real feature of this experience is trying to have you focus on this volumetric capture. But I think they can tweak it in some ways. Um, and when I did it, there was a little bit of an offset. And so you have the risk of bumping into people. But I think, you know, one of the things that Toby said is like, just to encourage people to feel free to move around a little bit, because you know, this is still the early days of VR and it's not going to kill you to kind of bump into somebody unless of course you're like running at full speed towards each other. But. You know, that's generally not recommended in any VR experience. But just to be able to have this sense of being able to dynamically move around, and I didn't necessarily get the sense that other people could see me moving around dancing because this was a type of experience where I would have really loved to see this group synchrony within the experience. And because there may have been a delay and a de-emphasis of that as a group experience, and I think they're missing out on some of that social dimension of the synchrony of people just even moving in rhythm to the music as it's playing and unfolding. So this was a extremely impressive experience within itself of just this Windows mixed reality volumetric capture and this performance piece by Nubia Brandon. And, you know, she talks about her own process of what she was trying to communicate. You know, one of the main messages from Nubia is she does feel like she's in all kinds of limbo where there's parts of her identity that feels like it doesn't necessarily always have a root or a grounding within the culture and feel like she's kind of trapped between these different worlds and that. This piece she's able to walk through her own story and her own timeline but to kind of step through these different aspects that there's no real tradition or black heritage that she's really drawing upon. There's like this dance hall culture and Jamaican heritage that's part of the UK heritage and the base culture that was coming around in that time. You know, in this piece, she's really following these two different stories of her own personal story, but also the story of the evolution of the Calypso generation in the 40s, the Root generation, and then going up into today and just, you know, sending out her gratitude to her boyfriend and that, you know, she's just trying to fit in as much as possible. But yet, you know, there's no real conclusion to this topic and that she doesn't really necessarily have closure on it for herself and so it's sort of fitting that the title within itself is All Kinds of Limbo and I think that's the deeper message that she's trying to get on both in the lyrics and the music but also in the music video as well. For me it is sometimes hard to parse music and language in immersive experiences especially as I'm paying attention to something. I actually went through it two times and then there's actually available on Spotify so you can listen to it and so When I listen to music, I tend to get overwhelmed by the gestalt, and so it's difficult for me to parse down into the words and the meaning of the song until I get a chance to listen to it a number of times and really meditate on it. But having listened to it again a number of other times, I can really see the different aspects of the story that Nubia is telling here. now in talking to toby there are certain aspects of this being a part of the black music that was happening in the united kingdom so the jamaican influences and all these other things like calypso and all these movements that i'm not necessarily familiar with and so there's a lot of very specific regional cultural context and references that I'm not familiar with. And it was interesting to hear Toby say, like, you know, it's not meant to try to cover the comprehensive history of all of this. All of these influences is basically the culmination of a lot of those influences into this performance piece that becomes a tour through a lot of these musical genres embodied through the music, but also the story as it is unfolding through all these times as well. And to really get the full context to all the surrounding information and the history that there's gonna be other media that are out there, whether that's, you know, Nubia mentioned Babylon, which was a film that came out back in the 80s, and then was actually re-released just recently, just because it was such a significant film in the evolution of Jamaican music and United Kingdom, that was actually released here in the United States just within the last couple of years. And so there's certain aspects of the culture that through movies or through a radio documentary or other ways to kind of get at some of this broader story. Because you know, originally, it was going to be like an evolution through these different time periods. And then I think Nubia put together this timeline in history, trying to prove why that was not going to necessarily be good approach to trying to talk about this and so they decided instead to really flip it on to her own personal story and her own personal evolution and to try to get at some of those Larger issues through the lens of her own experiences. And so I think it just worked really well it's quite compelling and it's one of those experiences that feels like a a full live performance mixed with a lot of the best aspects of a music video, but also kind of like a theater element. But because it's in virtual reality, you're able to turn this theatrical elements into a music video. So I think that's an interesting blend because it actually feels like you're watching a live performance, but yet it's recorded. So then you could do all sorts of other things with it being in virtual reality. And it's the type of experience that going through it a couple of times, I just thought this is really tight and I could watch this, you know, 10 times in a row and not get tired of it just because it's such a well-produced piece, marrying the virtual reality affordances with the music and this whole piece that she put together. And Toby said that this is the first time that they've actually released anything on their album. They've actually created a new publishing arm within the national theater to be able to start to put out different types of music that's created through projects like this. So this is an experience where it is a social experience. And so hopefully it'll be able to travel around to different places and you'll get a chance at some point to be able to see it, but just to see how you could put that many people, 20 different people within the same room, Oculus quest. I'm looking forward to the time in the future where you see a lot more of these different interactions. And I think trying to find that good balance between. What is the different types of player agency or embodiment that can give you the sense of someone else's presence without having to create something that's going to obscure or block someone else's view? The great thing about having 20 people in VR is that, you know, if you had 20 or a hundred or a thousand people and you're all physically co-located, then you would start to have people with different heights, different sizes, and it would start to occlude the main attraction here, which is Nubia and the performer. And so just to have a way of abstracting out your embodiment into this line where it's not necessarily getting in your way, but still be able to have enough of your embodiment shown to give you the sense of presence in identity and expression. And to, like I said earlier, to have the sense of group synchrony, which I think would be very cool to see a type of experience like this and to have people really get into it like as if it was a music video and people were really jamming out and dancing it around. I think if there's any one complaint that I have with people when they go in an experience like this is that as they put on the VR headset they're being occluded by what's around them and so I think people are generally just extra conservative from not moving around a lot. I've done a lot of VR and I just like to move around and dance around and so I would just like to see the type of experience with 20 other people that are just really rocking out and jamming out and to see how that translates into a virtual reality experience like this and to see how that can give you this extra sense of this group synchrony as you're watching this type of experience of all kinds of limbo. So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast, and if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a Unleashed to Support a podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So, you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.