I talk with Winslow Porter at Tribeca 2017 about Tree VR, which is an experience catalyzed by the Rainforest Alliance where you embody a tree and experience an accelerated life cycle that comes to a premature end as human activity is catalyzing the destruction of the Amazon Rainforest. We talk about all of the haptics, smell, and the logistics of implementing a variety of 4D effects. There is also an impressive list of collaborators and funders on this project as it has traveled around the world for the past couple of years.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. So continuing on my series of looking at VR for good movement, today's episode is with Winslow Porter. He's the co-director of Tree. So Tree, I first saw at Tribeca back in 2017. It's just at the Impact Reality Summit in Seattle, and they were showing Tree there after it traveled around all over the world. And the two creators, Melissa and Winslow, were there. And they're working on their third installment, which is an augmented reality experience that they had been working on for a number of years. They're really waiting on the technology to get Advanced enough and I think they're they're at the point now that they're able to start to build it out So Winslow and Lisa they had a trilogy that first start with giant which I first saw at Sundance 2016 and then tree which was at Sundance and then Tribeca in 2017 I had a chance to catch up with Winslow to talk a little bit about all the different Aspects of haptics and smell and ways that they're trying to tell the story of the rainforest and different trees and you know in this experience, you see a tree and you become a tree and that you see the whole life cycle of that tree. At the end, the tree actually burns down. And so it's just trying to speak to the larger thing that's happening in the rainforest now, the destruction of the rainforest. And, you know, you have this direct embodied experience of actually becoming the tree, embodying the tree, and then going through the whole life cycle of the tree, ending with the tree's life being ended at the hands of men who are trying to clear cutting the forest for mining. So that's what we're coming on today's episode of Oasis of Yara podcast. So this interview with Winslow happened on Friday, April 21st, 2017 at the Tribeca film festival in New York city, New York. So with that, let's go ahead and dive right in.
[00:01:52.618] Winslow Porter: My name is Winslow Porter and I am one of the directors along with Milica Zets on a project called tree. The last project we did, it's a project called giant. And so this is the second part of a trilogy.
[00:02:03.724] Kent Bye: Maybe you could just tell the story of how this project came about and what you were really trying to say.
[00:02:08.768] Winslow Porter: Sure, so I guess it starts with Giant. Milica actually envisioned Giant as the first part of a trilogy where you're in a basement with a family during their last moments. There's bombs that are approaching and you get to witness that with the haptic chair which really makes you feel like you're there. We really believe that any type of physical embodiment of the virtual world really helps immerse you in that. So with that, we wanted to make a project. Giant is about the atrocities that human beings do to each other. And then with Tree, we wanted to talk about the harm that man is doing to Mother Nature. And so with Tree, which is actually originally inspired by a sound composition called Nightmare of a Tree by Alexander Prodich, who was also the sound designer on this project, or the composer, I should say. And also he was one of the sound designers for Giant as well. In his piece in 2002, Nightmare of a Tree, you get to witness what it's like from a tree growing from seed to full size, but it takes a slightly different arc where you get cut down in that experience. And ours, I won't spoil it, but you you meet a different fate of far too many rainforest trees. And in Tree, in the actual installation, we ask people to grab a seed, plant it in a bed of soil, they put the headset on and realize they've transformed into that tree or into that seed. And then they grow as a seedling through the soil and eventually they get use of their arms as they grow and see sort of like as the sun hits them that they are expanding and sort of witnessing the majesty of nature all around them. until they reach the very top, because we're modeling this after a specific tree called the Kapok tree, which is a sacred tree in all sorts of areas of the Amazon, but particularly we chose a place in the Peruvian rainforest. We worked with the Rainforest Alliance to really find a tree that meant a lot spiritually to, in this case, the people of Peru. As you witness it growing through the top, it's an emergent tree, so it's the tallest in the rainforest. It goes above the canopy. You then also witness, you hear the sound of humans, and it doesn't bode well for the tree after that. But along the way, in order to simulate being this tree, We have all different types of haptic components, or 4D elements as people say. You know, which aren't really that new when you think about it. You know, people have been doing this commonly in the 50s. We have a scent track where we worked with a company called International Flavors and Fragrances. You know, and I just love the term scent track. It sounds a lot better than smell-o-vision. But it's a very similar thing where at certain times we are able to trigger different events and we're able to bring you, as soon as you start the experience, we send out something called Earth Pete, a scent that's custom made for this experience where you really feel this like dank soil and part of your mind knows that you're not there, you know, you didn't just transform into a seed, but another part of your mind smells it and it's convinced and it wants to go along for the ride. And a common thing in VR, just like a lot of other types of storytelling, is convincing people that they're somewhere else, but doing it through, in a sense, it's all magic. We're back of house here at Tribeca Film Festival, and you can see all the different computers that we have, but obviously we don't want to show people how we do the tricks. And Trick may be making it sound like we're trying to deceive people, but it's all in good fun. And so we also have haptics in the ground, too, which are able to vibrate your feet, and also a sub-pack, which allows you to feel the growth of the seed and also the moment at the end where you meet your fate. We also have a fan, which simulates when you go above the canopy, this really peaceful moment, which makes it all the more heartbreaking for the end of the experience. And we have two heaters, which really simulate when some people get close to them when they're near the ground, their brain tells them that something's very wrong. Everyone reacts differently, but the heat we find, in contrast with the cooling fan, is very effective to be able to immerse people in this Peruvian rainforest. And then we also light a match at the end of the experience too, which there's nothing that can really simulate that smell quite like it. People think we use some crazy high-tech device for that, but at the end of the day it's just a book of matches.
[00:05:57.918] Kent Bye: Yeah, interesting. So let's take a step back and talk about the distribution of this just as we're talking about this just to see how people might be able to see it and also kind of pointing out the fact that seeing it as an installation you get to see all these 4D effects that I think take it to the next level of level of presence but yet are there other plans to get it out there and distributed it?
[00:06:17.500] Winslow Porter: So that's a really good question. Do we want it to be seen by hundreds of thousands of people or hundreds of people? I guess thousands because the centers are opening up very quickly so it's easier to be able to get more people through. But there's still an issue with scaling. I guess it's a similar situation of do you want to go to the movie theater or do you want to be able to witness something in your home theater? Or, you know, is it something that you can watch on your phone? There's sort of different ways to be able to consume it. Obviously, there's something that with more of the elements, it's a richer experience. But it really, for this message, for Tree, we wanted to be able to impact as many people as possible. So we do want to be able to create an at-home version where we can transfer the haptics, you know, into different controllers that are there. And, you know, sub-pack is becoming a lot more common in homes. like they have partnerships with different headsets which will allow that, but ideally, you know, being able to have this in centers where, you know, we can set up multiple sessions where maybe there's even multiple people experiencing it in one place where they can really have a moment together afterwards where they can sort of commiserate or sort of feel like they all experienced something similar. So yeah, I mean, definitely there's strengths to each one, but also, you know, the drawbacks of understanding that, yeah, we can't be delivering this. I would love to be able to make a scent box that could have all three scents, but you know, that's something that, maybe that's further down the road. I'll put that on a different timeline.
[00:07:35.040] Kent Bye: Yeah, I just talked to Jackie Moray who's working on a scent collar where you essentially have something around your neck and she was showing that off at VRLA and I think that right now if you look at the five senses, the visual sense really dominates and so that's what a lot of attention has been going on. We have the specialized audio that is basically what you can do with visual and audio is a lot. And then the haptics, you know, having this rumbling platform that you're on as well as the sub-pack, you know, some of this is getting consumerized that people will be able to have it in their home, but other things are kind of the digital out-of-home entertainment scale of actually feeling the rumbling underneath your feet. It's a subtle thing, but I think it makes a big difference. And then the smell, it sort of takes it to the next level in terms of putting you in the scene. At the very beginning, as I was underneath the ground and smelling it, I just felt like this, oh wow, this is really, it put me there in a way. Although, you know, since I've done a lot of VR, I'm like, oh, they're doing something with sense. I mean, so it's like, it's hard for me to completely be immersed with that. So, I mean, just in, as people were talking here, as we're talking about this, people might be able to put together that there's heat and smoke and fire and But there's a certain amount of human interaction that you have to have an individual standing there and lighting a match right underneath you. And so that is another level of interaction that you're not going to be able to get, and it's not automated. So it's an installation piece in that way. So I'm curious, as you're designing this, you're having these high-touch, non-scalable components mixed with the whole things that could be consumerized.
[00:09:03.537] Winslow Porter: Yeah, I mean, well, when we have the installation, you know, we want to be able to have as many different senses, you know, activated as possible. And, you know, when we do it, too, it's I think it's we don't like to show exactly what's going on the screen. So there's no spoilers because as soon as someone sees any representation of what the work is, they will already form an opinion and I think get a sense that they've already experienced it or have an idea. So we wanted to abstract that by having the projection on the wall. And then there's a certain amount of theatrics that are involved with us having to light the match. Just in general, when we onboard them to tell them to grab the seed, that they should plant it however they want, to take a deep breath, to be able to embody what that is, the experience begins as soon as they hear about the experience. And then when they step to the platform, that's sort of the next step. It's not just putting the headset on and taking it off, but being able to help them, to guide them through it. Yeah, it almost reminds me of a little bit of immersive theatre, you know, involved with it. And then everybody too, like one of our executive producers has a different way of lighting the match, Justin Durazo. He has his own technique and we both sort of like have learned through doing it with multiple people. There's something really nice about having it customised. We try to move the heaters too for when the people are in different areas. Have the sense, you know, if people immediately are like sort of more receptive to it. We'll give them a little bit more We'll bring a little bit closer. You know, these things could be automated. I actually have looked into what it would entail having a match lighting machine It's maybe a little bit more work than what we were expecting but I think there's something really nice about being there and also it piques people's curiosity as to you know, as to what they will be seeing. They see me down there lighting a match and they think to themselves, they're like, oh wow, I wonder what that's like inside the headset.
[00:10:40.351] Kent Bye: So can you talk a bit about the battle station behind here, what people have to do in order to run this experience?
[00:10:45.516] Winslow Porter: Sure, yeah, we were very fortunate to be able to meet Shin and Yidan from MIT Media Lab, who are standing right next to us here in the back of house, where they designed a patch in Maxim SP, which basically allows the master computer that's running the experience, you know, the audio and the headset, to be able to send a message, something called a UDP message, to Maxim SP, and it receives a bang, basically, it's an on switch, and then that starts a timeline on our show controller computer, We have an HP laptop over to the right, which basically is the brains of all the haptics. It allows the low frequencies to get sent to the floor, to the vest. It also is able to tell, basically using something called a relay switch, which is just turning a power supply on and off, so that you have a duration in which you can send that message. So we tell, you know, the fan, okay, be at 30 seconds, at 240 seconds into the experience. And then we also have another computer to the left, which is playing the projection on the back wall, the sort of abstracted journey of the tree. And we also have an audio track, an ambient audio track that plays through it as well.
[00:11:49.217] Kent Bye: Usually when you do a VR experience, everything's driven by one computer, but what it sounds like is that there's multiple computers driving different parts of the experience as well as another computer that's driving the part of the experience where other people are kind of watching it.
[00:12:00.972] Winslow Porter: Yeah, exactly. You know, eventually this will all be done either on a computer and a phone or just all on a phone itself. But because we're sending so many messages, it would get sort of confused. And also, we've got a pretty sizable GPU in there right now, but we don't also want to have one computer handling too many tasks because it'll bring the frame rate down. So yeah, it's very modular right now, but I think it's all going to be moved towards sort of a unified solution.
[00:12:26.432] Kent Bye: And can you talk a bit about Sundance this year? Because I know that there was some initiatives to be able to have more environmentally themed experiences and if there was some involvement there in this development of this project.
[00:12:38.252] Winslow Porter: Yeah, I mean, Kamal was, you know, a big proponent early on of the trilogy, so she actually helped us get our act together, putting together a deck, because we were touring so much with Giant. So Milica and I were, you know, she inspired us to really sort of unify on what the journey of, you know, the user experience would be. And so then we actually were fortunate enough to get a Rauschenberg grant, because we were part of the new climate section. And we're also, we partnered with the Redford Center, actually, to be able to help with our funding. They were our fiscal sponsor, which was really great. They believed in the project as well. And it's also one thing to be able to talk about VR. It's another thing, you know, as soon as we were able to get people from the Redford Center into the headset, we were able to show Jamie. And it's one thing to see a deck and another thing to be able to witness this tree's life in eight minutes. You know, an experience that would normally take a hundred years or more. and also be able to see the expressions of people's faces, you know, after they got out of it, so that we were really, you know, I think that people definitely think about trees very differently, because we're giving them a unique perspective. So for this too, we wanted to be able to leave people with a sort of a, not a feeling of helplessness, but to want to sort of engage them, to galvanize their efforts. So, you know, we work closely with the Rainforest Alliance as well, and they helped us identify the problems that are in this area of Peru, which is Oftentimes the deforestation is caused by palm oil, by livestock, and actually gold mines where they slash and burn large pieces of land. So we wanted to be able to educate that. So in the experience we actually give you the seed back, sort of as a seed of hope, as a reminder to take this seed as a reminder to keep our forest standing, is what it says on it. And we printed it on recycled paper with eco-friendly ink. Actually our business cards are made from sustainable cotton. as well, so we're very conscious about all the little things that we can do. And actually Sundance was using wood from the previous year, so a lot of attention to detail when it came to that type of thing. But yeah, as far as future festivals, it just was announced today, because it's Earth Day, that we are part of the VR for Impact with HTC, which is great, because we very much look forward to being able to work with them and also work with schools and with other educational centers, museums, libraries, to have this experience in as many places as possible. Because for us, yeah, just getting people inside the headset is what is most important to us as creators, as storytellers.
[00:14:59.965] Kent Bye: Yeah, and to me, when I think about some of the ultimate potential of VR, I think of being able to help us connect to ourselves, to connect to each other, but also to connect to the Earth and the cosmos. And so this experience in particular, you're using embodiment in VR to be able to cultivate empathy and connection to the Earth. So I'm just curious to hear your process of design of how to allow your first person embodiment to be able to cultivate that connection.
[00:15:26.053] Winslow Porter: Yeah, I mean, that's a really, that's a pretty heady, you know, thing to think about, using computers and a digital form of storytelling to bring people closer to nature. But I think it's definitely possible because oftentimes we don't even have a moment to think about it. We feel also really helpless and sort of a learned helplessness of, you know, that this is going to happen. A statistic that we found really shocking is that in one finding, it says that, you know, we won't have rainforests in 2050 anymore. And that's, you know, that does, that's not that far away from now. And, you know, That's if we continue at our current rate. So I think making people understand that there is something that they can do, to understand that the issue is extremely complicated. But also, when we showed it at Sundance, a lot of people came up to us and said that they looked at trees differently. That they were telling their friends, and their friends were like, oh, you're crazy. Stop thinking about being in a tree. But they're like, no, you have to try it. Because once you do that it allows you to, it's a paradigm shift for people and that's really, that's the difference of putting a screen in front of them that they can sort of more passively interact with. Whereas VR, yeah, we're sort of dissolving the screen and it's all around you and you can obviously not open your eyes if you don't want to, but for the most part it's happening and you're witnessing it and it can feel very real.
[00:16:35.818] Kent Bye: And so what do you want to experience in VR then?
[00:16:38.579] Winslow Porter: I don't know. I mean, it's a good question. I think that, you know, with the eyes in the animal piece, we're big fans of a marshmallow laser feast, you know, being able to witness through different types of particle effects in a software that they use called V4 or VVVV. You know, they were able to do something totally different where I never even imagined what it would be like to be a dragonfly or a frog or an owl. But, you know, after that, I think that everyone is pretty profoundly impacted by it. It's an art piece, but at the same time, it's something totally new. It's even hard to compare it to other things. So I think that in general, too, people should always be approaching VR for, is this the best tool? And I think that's a question that a lot of people ask. And so I think for Tree especially, I don't know of too many other, I think a sound piece, because that's what it originally was, is a good way to do it. But I think also VR is the appropriate tool for telling the story. As far as future projects that I would like to see in this, I mean, that's a great question of what else is something that is best told in VR. And I think if I knew right now that I'd probably be hard at work or, you know, Milica and I would be hard at work crafting a deck for that. But we are, you know, interested in the third part of the installment, which right now we're calling Breath. where we want to be able to use augmented reality, but the tools for that aren't quite ready yet, in which case we might be prototyping with untethered VR headsets to be able to simulate what AR would be like, so that when everything's ready, all the assets are there, we'd be using Unreal again, because we really enjoy the team there, and also the way it looks, the lighting and shadows from that are just really top-notch.
[00:18:17.436] Kent Bye: Great. And finally, what do you think is kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:18:25.440] Winslow Porter: That's something I think a lot of people are thinking about. I think it has the potential to... Catherine Bigelow has a piece here, The Protectors, very profoundly influenced by the movie Strange Days, thinking of what that would be like to truly experience somebody else's memories. And they have an interesting story where they're able to use it to solve a murder mystery. The thing is that with a new platform, with a new medium, you know, there's sort of infinite possibilities and the closer it becomes to looking real, the more we have a chance of profoundly impacting people and, you know, it's really up to the people creating the content as to how they want to do that, you know. With video games, you know, there's games like, say, something like Flower, which allows you to be able to, you know, have a more sort of abstracted idea of what it would be like to be sort of pollen or leaves floating around a space. And then you also have, you know, your classic first-person shooters. Or just like with films, you know, there's, you know, a wide variety of genres to be able to communicate sort of the human experience or other experiences. So I think it's really, the future's really bright and, you know, it's also overwhelming for what is possible inside of virtual reality, augmented reality, mixed reality, whatever people are calling it these days. I think it'll all sort of start blending together. Because at the end of the day, virtual reality is just the ability to take someone to another place. And we've been doing that since we've been telling stories as human beings around the campfire. And what is virtual reality to someone who can't see? Is it a convincing sound piece? Is it like a really good record? Because it's able to bring you somewhere. Something that also someone said about Tree is that the piece could exist without the visuals, if you're able to experience everything and just hear it. So for me, that's really interesting to think about. So I think that, yeah, being able to basically explore being alive and also, you know, what it's like to embody, to have a sense of embodiment when it comes to other beings and, you know, what is virtual surreality? Why do we have to compare everything right now to the one-to-one relationship with our everyday lives, you know? What happens if we are totally disembodied and able to communicate with other people through something that, you know, is so new that we can't even describe it? That sounds yeah As you can tell I've been been working very late nights here at the festival So that's sort of my stream of consciousness feelings on that Awesome. Well, thank you so much. Thank you pleasure
[00:20:48.625] Kent Bye: So that was Wenzel Porter. He's the co-director of Tree, which premiered at Sundance in 2017. And I talked to him at Tribeca in 2017. So I have a number of different takeaways about this interview is that first of all, well, I really like how they were experimenting with embodiment here to tell the story, because to really show the growth of a tree, they're really accelerating time. They're really playing with timescales as well, because trees often outlive human beings. And so to be able to see the full life cycle of a tree, And then as you are in the tree, you're able to kind of actually embody the different branches and look around. And, you know, there's a big part of this experience I'd say is the different haptic experience, whether it's like the rumbling floor or the sub pack or the different smells that they have, their lighting matches in front of your nose and kind of blowing out the smoke. And so just a lot of ways that they're trying to engage into a full sensory experience. And it was interesting to see behind the scenes at Tribeca, how many different separate computers, I mean, as you do the experience, you don't think about like, Oh, there's actually like three different computer systems that are running just to even drive this experience because you know, the, the haptics and the, the floor and the actual VR experience itself are through the two different computers. And then the third one is for the audience to be able to see some type of projection map. It was interesting to hear that Winslow was specifically saying they didn't want people to see anything about the experience because they found that if people see something in a 2D screen, they're like, Oh, I know what that's about. And they want to have a little bit more of a mystery so that people would want to actually experience it and not have any spoilers as to what happens. This experience has been out there for a number of years. And so at the end, you have this tree that's burning down and it's trying to point to this larger issue of like the destruction of nature, destruction of earth. And how far can we take this idea of trying to use these virtual reality and immersive technologies to give you these embodied experiences to allow you to connect to nature around us? And then can that instill a deeper connection to seeing how we're a part of a larger ecosystem? Lots of different VR for Good experiences that are trying to do that same type of experience. One that comes to mind most vividly is the marshmallow lizard feast which when so pointed out with the in the eyes of the animal actually at Tribeca 2017 there was actually a marshmallow lizard feast experience there called tree hugger, which was a very similar kind of like depiction of a tree and you're touching the tree, but it's a lot different artistic style using a point cloud representation and that marshmallow is your feast piece ended up coming into the we live in an ocean and air which was at the such a gallery later the next year and But this beast tree is a part of the larger trilogy, which, you know, the first one was giant, which was a lot about war and men against each other. And this one was men against nature in the sense of, you know, destroying nature. And the last one, they wanted to try to see different aspects of our breath and how our breath is connecting us to this larger ecosystem. There's a number of different VR experiences that are doing that. I know there's one called Breathe that's going to be at Sundance this year. Martial Lancer Feast actually is doing another experience around breath as well and actually going inside of the body and seeing how the breath is inside of the body and how they can start to depict innards of the body as this ecosystem. But this next experience for both Winslow and Lizza, they wanted to try to have the augmented reality experience on the phone. So being able to push this out to as many phones out there that have augmented reality. and use the affordances of augmented reality, which I think they've been waiting actually for a couple of years. And it's, I think at the point now where they can start to build on it and start to deploy out and do the features that they've been waiting on. So super curious to see how that ends up, but lots of different people involved in the funding here. There's the Redford center, the Rauschenberg grant, and then the VR impact from HTC that I think was a big part of helping to take it all over the world and show it all over the different place. So just had a lot of support from a lot of different funders and I think the VR for good Experiences if you get into the right area There's number of different funding resources because it's just need more art more content to be able to cover these different type of topics and so sounds like they were able to get some of those grants to be able to produce this piece and Continuing on to produce their next piece. So there I think they're still in the phase of fundraising right now. So and reach out to them if you want to get more involved in helping support their work in the future. So that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
