Polymorf’s Symbiosis is a multi-sensory VR experience that uses soft-body haptics, a bespoke, microdose smell dispersal system, and different tastes. The piece debuted at IDFA DocLab 2021 (see my previous interview here), and it has a sold out run in Portland, Oregon from November 12 to February 12 at the Portland Art Museum’s Center for an Untold Tomorrow (PAM CUT). There are 6 different characters that you can embody in this “speculative fabulation” inspired by the last chapter of Donna Haraway’s Staying with the Trouble book where she suggests that humans start to combine the DNA of endangered species into humans in order to cultivate a deeper kinship with the world around us. Polymorf takes this idea to the logical extreme as you embodied these human animal/hybrids in the distant future. I had a chance to experience 3 of the 6 different characters at the PAM CUT press preview, and then catch up with the co-creators Marcel van Brakel and Mark Meeuwenoord during my trip to IDFA DocLab 2022 in Amsterdam to unpack their journey, process, and some of my experiential design feedback and critiques.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye and welcome to the Voices of VR podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. And you can support the podcast at patreon.com slash Voices of VR. So in today's episode, I'm continuing on my series of looking at different pieces that were showing at IFA DocLab. Actually, this was a piece that showed last year, 2021 at IFA DocLab, but also happens to be showing in Portland, Oregon from November of 2022 into February 2023 at the Portland Art Museum Center for Untold Tomorrows. So there's a piece called Symbiosis that I had a chance right before my trip to DocLab to see three of the different six characters and then the creators are actually from Amsterdam so I was able to run into them and do an interview. I actually did an interview with Polymorph last year about this project but hadn't had a chance to see it since it was this whole multi-sensory experience. So this is a piece that is, first of all, inspired by some philosophical writings from Donna Haraway's Staying with the Trouble, where she talks about how can we actually just stay with the problems that we have in the world and not try to escape, but also to imagine different potential futures. And part of that imaginal space is to find other ways that we can build empathy with these creatures that are around us. Her ideas that she explores in the last chapter of her book is this idea of what if we were to blend our DNA with other animals, would that be able to cultivate a deeper type of kinship with the world around us? Polymorph took that idea, this speculative fabulation, and said, OK, let's take this out to its logical extreme and have these different characters that you're embodying. that you have different multi sensory experiences. So you're wearing this soft body haptic suit, you have different smells, they also have different tastes that you're eating. And so they're trying to extrapolate and experiment with these different forms of embodiment into these deeper philosophical ideas of how can we build a kinship with the world around us. So there's lots of different things that are going on with this piece and I wanted to have a chance to talk to them and give a larger context and I'll have some more thoughts at the end as there's six different characters and there's a wide range of different types of experiences and so I'll share some of my thoughts of being able to experience some of the multiple characters. So, that's what we're covering on today's episode of the Voices of VR podcast. So, this interview with Mark and Marcel happened on Sunday, November 13th, 2022, at IFADocLab in Amsterdam, Netherlands. So, with that, let's go ahead and dive right in.
[00:02:38.279] Marcel van Brakel: So I'm Marcel van Brakel and I'm founder of Polymorph. I have a background as a filmmaker and a theatre maker but now we're more and more into interactive art and with Polymorph we kind of research the human body, XR and AR, so like new technologies and the body and what's the relationship between the body and technology.
[00:02:59.895] Brakel Mark Meeuwenoord: My name is Mark Marienhoort, I'm also in Polymorph together with Marcel. My background is actually philosophy and creative technologies. And yeah, that's what I do.
[00:03:14.340] Kent Bye: So yeah, I live in Portland, Oregon, where the Polymorph Symbiosis is opening up there. It's having an extended run there, I think until February or so. And so I had a chance just before I came to IFA to try out three of the different characters and got some more information on the other characters. But there seems to be six different choices that the audience has to be able to experience a multi-sensory, immersive, virtual reality experience with sound and visuals, but also haptic suits. And there's smells as well as with taste and eating. And the version I saw, the food was held up in customs, and so it wasn't there yet. So I saw four of the five different senses. But yeah, maybe you could give a bit more context as to symbiosis and how this project came about.
[00:03:58.653] Marcel van Brakel: Well, so Symbiosis, like you explained, is this multi-user VR experience where the user can pick a character and with the character comes a soft robotic suit that kind of changes your body architecture into a non-human species or a post-human species. And with the project, we want to kind of research what happens if we go away from the human-centric position where we control everything and have all the power and go to a more symbiotic relationship with other kinds of nature, maybe also future nature. that might include nanorobotics or AIs or other kind of things with agency. And to really have a thought experiment about how that changes maybe your emotional state, your kinships, your relationships with people, political structure. So for that we built like six different stories with six different hybrids consisting out of human DNA combined with endangered species that we want to save or get a kinship with. And as an audience you can choose one of these to experience these speculations.
[00:05:02.672] Brakel Mark Meeuwenoord: The project is inspired by a work of Donna Haraway in her latest book, Staying with the Trouble, Making Kin in the Tertulli Scene. And it's one of the suggestions of fabulations, as she calls it, is to combine ourselves or our DNA with endangered species to be responsible. The offspring, we have this kind of sense of responsibility taking care of, becoming with these other forms of kin, of other species. Oddkins, he calls it, I think. And then having this more symbiotic relation, as Marcel said, with your surroundings, with your ecologies, with other beings, with other critters. So that's the more, I think, the more ideological, intellectual part of the experience. And then next to that is also just being there and experiencing a very physical VR piece, because VR is a lot about losing your body or getting rid of the body and then making a new one in virtual reality. But we like to introduce or what we try to do is reintroduce the physicality of our own body or maybe a body of another thing into the VR piece again. And that's why we use sand and taste and these robotic elements that you wear.
[00:06:18.653] Kent Bye: Yeah, I'd love to have you expand a little bit on some of the technology because this goes above and beyond, say, just a normal virtual reality experience where you put on a headset and you're already taking care of both the visuals as well as the spatialized sound. But here you have these whole haptic suits that people are choosing what character they get in. And then I did three of the six different characters. And so I had some experience of some of the different haptic variations in each of the different characters. And then you have the smells as well. And I don't know if each of the different characters are smelling different things as well. You have this tube that's at the bottom of the VR HMD that is dispersing smells at different points of the narrative. So maybe you could talk about that process of developing both this haptic experience as well as the smell experience within Symbiosis.
[00:07:00.428] Brakel Mark Meeuwenoord: The first thing I think what's interesting to say actually is what we discovered. So VR is built for a certain kind of body, like a human standing straight with his arms like kind of spread to the sides. I think it's called. And what we tried is explore different kinds of shapes of different kinds of bodily architectures. So we would maybe lie on the floor or we would tie each other up. We have one of the characters actually a suit that people are connected to each other. And these haptics, yeah, they control that in a sort of sense. But that didn't work technically, because when you lay down, the trackers won't see you in the VR. And when you make different body postures, the technology doesn't really, it's not really suited to do that. So we had to figure out how to take care of that. But also I think the whole thing, maybe you saw of course, is it's not only the VR, you know, it's the suits, it's the scent, it's the smell, but it's also this environment, this ecology, and there's all sorts of pneumatic cylinders actually connected to the suits via air, and they control pockets in the suits. And you're connected to these technologies via air, with air, with this soft thing that we breathe also, you know, that's also a very interesting aspect I think in the piece. So you're connected in this whole, literally this symbiotic relation in the piece itself, connected to the technology.
[00:08:29.143] Marcel van Brakel: Yeah, and I think it's also a long journey of trial and error. First copying soft robotic systems, then making our own variations and kind of discover how to do that in a way that it doesn't break. Because at Pemkut it's there for three months, it's really a long time. and we need to have it sterile and replaceable. So that was a long journey also to kind of research and R&D that. And with the smells, it's also completely different. So not only every suit is different, but also the smell design is completely different for each character. And also this food snack arrangement that you get is totally unique for the character. So for us it was really interesting, actually kind of a trick we learned from a previous Project Famous desk where we had like multiple stories parallel to each other so the audience when they came out of the experience they immediately started to share stories and because they were curious about what other people would have experienced and they wanted to share that. So in that sense it also connects people. after the experience by thinking about the story, what they experienced on a physical level, on an emotional level and to kind of also think about how this future might look like and can you participate on it. And also within the smell design we use a certain smell that kind of stimulates the natural produce of the hormone oxytocin in the human body. And that's really an awesome thing because oxytocin is kind of created when people are pregnant and a woman is pregnant to kind of create bondage, a kinship with the family, for the father to bond with the child, for the mother to bond with the father. So it's telling to the brain, this is my kin group, this is my society, this is my family. And if you smell it with people who are not part of that kinship, it tells the brain, okay, they're not part of your group. And if you're part of the group, it means that the brain is also more susceptible and more willing to share resources to help each other, to empathize with each other. So in that way, it's really interesting to kind of also use this almost hormonal manipulation of the brain to facilitate connection between the people in the experience.
[00:10:38.862] Kent Bye: Yeah, I had a chance to go to the press preview where they had different slots. So I was able to see three characters. And my first impression that I have is that each of these experiences is widely different. You can get whole other different types of experiences. And so there seems to be a negotiation process that happens at the beginning where The docent that's there gives a brief introduction to each of the six different characters. There's the slime mode, and the monarch bonafide, and the toad. And then you have this multi-body entity that has the head, the body, and the Tamagotchi AI food gatherer. So each of these six different characters are widely different. So I'd love to hear your take of how you describe each of these different characters because there's a bit of a negotiation process that has to happen amongst the audience to say who wants to do what type of experience. So I'd love to hear some of your reflections of what you were trying to do with each of these different characters.
[00:11:32.318] Marcel van Brakel: So like with the multibody, that was kind of the last character we added to the total. And with the multibody, I wanted to kind of think through what is your consciousness actually? Is this voice in your head? But maybe in the future, we're able to kind of merge brain or merge consciousness. And how would that be like? So that was like the basic idea. And from that also came the idea that, okay, if we merge brain, we might also merge body. So within the installation, people are really interconnected, physically connected to each other. But in the story, they have this monologue interior that's really different than all of the other stories, where it's this coalition of voices that's talking to each other and making a decision together. But it's also the AI is like a part of this coalition. It's trying to influence or it's taking out data out of this conversation. And if you play the AI, you will have the same story world and the same presence in the same digital part of the same world, but you will experience a completely abstract and a completely different matter. You're actually listening in to the algorithms kind of making decisions, tapping into the conscious stream of your partners that have this completely other perception of reality. So that was kind of complex because of course if you're a butterfly you have a different set of sensory apparatus with which you make sense of the world. And of course we need to sometimes compromise to the audience that you can still have an empathetic relationship with a creature. But we kind of played around that, for instance, slime mold, of course, is a one cellular being, it cannot see. But having a VR, we still want to have a visual compound. So we choose to design the slime mold world in a more abstract manner. The slime mold can only see the chemical trails of the stuff that it's in. So in that sense, we wanted to kind of plant the seed that If you change body, you also change the perceptual system and with that comes also a different understanding of the world and maybe also a different kind of thinking about that world.
[00:13:31.448] Brakel Mark Meeuwenoord: I think you start out, and maybe this is a too heavy word, but you start out negotiating with each other what you're going to be in that world. But I think the real discussion is after that, that you exchange what you experienced, and that maybe if you were the slime mold or the AI, they had this completely off, kind of vague or maybe abstract experience in the VR, at least. And you negotiate that perspective on the world, basically, on our world. And I think that's an interesting thing, because of that, especially the AI is super abstract and radical, It's not really story-driven, it's almost abstract, there's almost no story, only repetition of some stuff, and that can be annoying even. And I think if you bring that to the discussion, After the experience with your co-experienced people, the people you actually experienced that, then you have something to tell. That's an even important experience or perspective as the others. Or at least that's what we maybe should talk about. What does the AI have to tell us?
[00:14:44.656] Marcel van Brakel: Also, because I think every story has its own theme, where the story is kind of following another thread in this becoming-with story. Some of our stories are really emotional and more empathy-driven, and other stories more reflective, about changing identity, for instance. Like, how would it be if you could not only change your sex, but also complete your body architecture, and what kind of new problems might that spark, and how do you overcome these problems? Yeah, thinking about the future where your body might be a fluid thing that you can maybe change in the way that you can change your avatar now on the internet. I think it's really interesting to kind of, yeah, it has all of these different styles also.
[00:15:30.188] Kent Bye: Yeah, so the first time that I did it, I did The Monarch Butterfly, which I felt like, in terms of the stories that I saw, was probably the most story-driven in the sense that, I mean, each of them have different stories in different ways, but I felt like the combination of taking me into this speculative future and to give me a narrative to hang on to as a character is moving through a space and Unpacking some of the deeper philosophical aspects of the body and what's it mean to merge with other entities? And so yeah, you have this kind of speculative sci-fi aspect that's getting into the philosophical aspects but introducing me into this potential future and then I did the slime mold and which was much more like you're in a sleeping bag and I'd say in terms of embodied experience that was the one that had the most visceral like haptic experience. It was pushing on all different aspects of my body and in more of an asymmetrical way than you know the Monarch butterfly was a little bit more of a symmetrical experience in my legs and a little bit of upper body, but the slime mold was much more immersive in the haptic sense. But also the visuals of that were a little bit more sparse, so more abstracted, but giving me some point cloud representations of the forest, and then after I saw the video of the cane toad, I was... recognizing that there was a little bit of a parallel between going through the world through the Toad's perspective, you get the same journey but in a much more abstracted for what the slime mold could perceive. And the Toad, I understand at least those three initial characters of the slime mold and the butterfly as well as the Toad were the ones that you launched here last year. with symbiosis, but the Toad, I didn't have a chance to experience the haptics of the Toad, but having something on the throat I thought was interesting in terms of like having different body parts that you're really stimulating. And then I did the Tamagotchi AI character, the third, and I guess I had that experience of frustration, almost feeling like had I been going only once and choosing and then I got that, I would feel like all the other characters are getting so much richer of a story and a narrative and it was almost like, a repetitive vision of the world that I was getting a little bit of annoyed. So I guess that's the intent, but I was sort of confused. It's like, why did the creators create what feels like such a divergent experience from these other experiences relative to the more story-driven or ones that have a little bit more of a hook in terms of what you might expect from a mediated experience or a story? And this was so abstract. And then once I saw the video of what the multibody saw from the head perspective, I understood what you were doing with the Tamagotchi in terms of creating that abstraction of that world. But in the absence of knowing what they're seeing or having any direct way of communicating with them, it's hard to know how that translates. If that was the only experience I got, then I would be sort of feeling like that was confusing or annoying.
[00:18:15.357] Marcel van Brakel: But I think it's confusing to be an AI.
[00:18:19.621] Brakel Mark Meeuwenoord: Without being easy, you're explaining exactly the way we think of it. And again, of course, it's a debate. Do you want to have an experience that's full for everyone? Or do you want to have everyone, in the end, think about the topics you want to think about? And I think if someone is annoyed in the group and actually finds a position in that group to speak about that and to communicate that, and the others listen actually, then I think you made a step or something, on that level at least. And then also, of course, we think of it as a whole. And we actually also see the thing, you know, it is an issue, you know, only being the AI. We had a discussion, especially in the States, you know, here in Holland, it's way more easy. People don't complain and they just do it all, VR. I mean, they were way more critical over there in Portland. with you guys, which is good because it also gave us this thing. What is important for us? Is it the individual stories, the individual experiences people buy a ticket for? Or is it the whole piece as a thing? And is it the complete story? and should be presented as that and stick to that plan. So I'm really happy that you actually felt that because then for us also it's an exploration in what do you do as artists and where do you think you're hitting the right thing. Is that collective? Yeah, that's the collective then.
[00:19:50.749] Marcel van Brakel: I think that's an important thing. In the beginning we also framed it as performative VR, like a more theatrical version of it, where actually not having the same story, but having a parallel perspective on the same world in the same moment in time, is also an important part of the experience. Because that's true, we have all these gridders here next to us, we're not aware of them, but they have the same moment in time and a totally different perception. and at the same time they also feel the impact of the stuff that we designed for them. You started with this idea of the slime mold and the toad are connected and that's really funny because they are designed to be like that. Actually the slime mold is traveling on the back of the toad for a certain moment in the story where they're really literally connected but they have completely Not knowing, they're completely unaware of their... They're kind of a bit aware of each other's presence, but not necessarily understandable. So sometimes we also have feedback. Okay, yeah, I was on the slime world, but where was it? And I think that's really funny because if you look at our own skin even, it's full with bacteria. They live there and they help us and they have their own life and they have their own version of reality at the same parallel in time, at the same moment that we have it.
[00:21:09.794] Kent Bye: I had a question about the multibody, because I only had one opportunity to do one part of the three-part entity. Because there's the head that they describe it as to have the agency to look around, and then the body. Is the body able to look around as well, or is the body able to only see the perspective of the head, which I would imagine from a VR perspective could potentially make someone extremely motion sick? So what is the perspective of the body of the multibody then?
[00:21:33.852] Marcel van Brakel: You're completely right. It's the last version. The body is following what the head sees. It's basically blind and it's getting its information from the head. But with your hands you're able to manipulate the head. So this is this negotiation with two people. You have to kind of... figure out. Sometimes we had two family members in the same suit and it was kind of easy. It was almost like a dance, very subtle and they wanted to manipulate each other. But if you're there with a complete stranger it's a different relationship and also you have to kind of feel what can I do, what I want to do, what you don't want to do. But yeah, it is a bit more challenging if you're only in the body in the experience.
[00:22:16.560] Brakel Mark Meeuwenoord: I think actually the version now, the first version we designed in the studio, Marcel and me, was more extreme. You know, there was one body controls the other body even more, physically also more. But, you know, to compromise a bit on people not knowing each other in the same suit, connecting to each other, meaning that normally it would be like holding my hands next to your face now and completely control your head. So controlling your view, basically, but with my hands, but you actually feel that I'm doing that. And the VR is really dominant, of course, because now I'm moving my head and changing your perspective with moving my head. But it would be even more with using my hands to move your head that you actually would feel that. So it's more of a bodily experience and less of an only visual experience. And you can still choose to do that in the suit.
[00:23:10.909] Kent Bye: Yeah, because the head is kind of facing directly the body, so they're face to face, and the Tamagotchi is kind of behind the body, facing in the other direction and attached in the back. And so you have this three people standing together, physically connected. And I guess the challenge for me in the multibody is that I have no sense of awareness of what's happening in the other body. I guess there's a larger question for each of these different characters. Because there's this big haptic suit, I was expecting to have a virtual representation of my body within the experience, but there was no virtual representation of the bodies. And so I'm this ghostly character in the scene, but yet feeling all these haptic effects. And so there's a little bit of a disconnect for me in terms of having what's called the virtual body ownership illusion, where I start to really feel like I'm identifying with the body that I'm having. In the absence of not having a virtual representation of my body, I felt like there was sort of a ghostly presence of this haptic presence without understanding the full context of what my body representation in this world was. I noticed that there was Vive trackers on one of the suits with the monarch butterfly in particular, but not having the virtual body I guess was a bit of a shock, but I heard they were trying to stabilize it or I don't know if you have plans in the future to have the representations of the body because I think that would help ground me into what my body representation is and to see maybe even some visual feedback to help correlate what's happening with what I'm feeling with the haptic feedback versus what the virtual representation of that to help identify with these other entities.
[00:24:35.992] Marcel van Brakel: You're totally right. The initial design included seeing your own body as a body and also see the back and have animation for it and also have agency on it. But just to be honest, that's still something we're struggling with. Like Mark already told us, if you make a skeleton of a non-existent creature, the trackers won't work that easily. So we have all kinds of issues with that, but we're still working on that also. So we hope, I don't know, I cannot promise when it will be finished, but we hope somewhere in the future we can also offer that into the experience. Because we think it's also really important, like you said, that you can really identify with the body shape. And potentially it's already prepared, but we still, also because we're not there in Portland, we cannot troubleshoot when things go wrong. We have to play it safe for the moment, so we preferred a stable system instead of including that feature into the experience for Portland.
[00:25:36.257] Kent Bye: So one of the things that was a little upsetting in the sense that, you know, the food was being held up in customs for whatever reason, so I didn't get a chance to have the final sense of the taste, but it seems like as I was watching the videos, there's different moments where the character is instructed to kind of open their mouth or to receive different food, maybe two or three times for each of the different characters at different moments. So the docents that are there have these plates to feed the different characters different types of food. So maybe you could just describe generally what you were trying to do with the food that was being delivered during symbiosis and how you were hoping that was going to play into effect with the different smells and also all the other senses with the touch and the smell and the vision.
[00:26:17.068] Marcel van Brakel: Yeah, so for us, one of the starting points for creating Symbiosis was our interest in combining food and VR. So there was a long-wished desire to make such a thing. So for each character, we kind of created two food snacks. We partnered up with Michelin-star restaurant in Holland, who kind of custom-created these different food sensations for the experience. And within the VR, the story is told in a first-person perspective, so it's like this strain of thought. And within that, there are certain clues that some stuff might be edible. So people are instructed that if you hear such a prompt to open their mouth, and then you might get a snack. And for all of the experience, we have two or three moments that we make this thing happen. So they're all vegan, and then of certain moments, of course, you need to surrender to the situation where you don't know what you will eat. And that's also a thing that we work with a lot that we think it's also very interesting to kind of put your audience into this vulnerable position where you're open and you have to also cross a border because frankly it's quite something to open up the body for something that you don't know what it is and you have to fully trust the team and the project that we don't fool you or play around with you. So we want to be as respectful as possible, but we designed the food snacks really also in a speculative way, in a way that's not always about tasting nice, but tasting interesting or tasting futuristic or unknown. And for us, it's also when we started like this question mark, how does it work in VR? How do people be open for that or susceptible for that or not?
[00:27:57.435] Brakel Mark Meeuwenoord: In a more content way, thinking about this perspective we have as a human and controlling stuff around us and deciding actually what's being eaten by us and how we produce food. It's about the food dilemma itself. Who is eating who or what is eating what? And how do you negotiate it? We are kind of on the top of the food chain, so to say, but there's all sorts of negotiations where we decide not to eat animals, but why do we maybe still eat plants? And what are plants? And how do we think about that in future constructs? Basically, I think that's also very, very important, actually, why food is a part of the experience, you know, and next to the vulnerability that it brings, you know, if you open your mouth without actually, that's a big step, you know, you have to really do that. And we like to have this bodily part of being vulnerable and being perceptive to losing control, giving up control, actually, in the experience. And the food is one part of that.
[00:28:59.714] Marcel van Brakel: Also when I did the multi-body myself, it's like this sea scenario where you're in this sea and the first snack is like it's made out of cooked seawater, like really condensed and it's a gel. And it's so special to kind of have the sea in your mouth when you're there. It's adding so much to the experience itself for you to kind of be in the moment and be in the spot that we want to place you. I think it was way more powerful than I anticipated. And then sometimes it's also, yeah, it's this dilemma. So within the store we have this symbiosis that kind of voluntarily share their body with the audience as an offering, because that also prevents us from killing other things to eat. And then they regenerate the body, and it's this kind of priest who see that as a spiritual task. But then to eat the human flesh of these critters, or the flesh of these critters, it's also a big step and people react very differently to it. Some really love it and some people think it's really hard.
[00:30:03.771] Brakel Mark Meeuwenoord: But it's grounded also in the way our own bodies work, of course, come to think of it. The mother that produces the milk for the child. It's a part of our own body that's actually being the child grows of her body, of course, but also when the child is born there's this ritual of feeding it from its own body. I think that's a very strong thing, you know, and actually it's super normal in that sense. But if you project it in a broader kinship kind of situation, then it becomes something else and you have to really take a step to accept that, I think.
[00:30:36.277] Kent Bye: And the other thing I noticed that was unique about this project was that you have these little plastic bulbs that have liquid that are dispersing smells at different intervals. And I think a larger challenge with smell dispersal is that once you put a smell into the room, sometimes it's hard to kind of edit a smell out. But I felt like the system that you have was able to deliver enough of those particles to give me the sense of the smell and to give me a way of actually having different smells over time without having, after going across multiple characters, smelling too much of what other people were smelling. So you're able to put a tube that's very close to the nose on the bottom of the VR HMD that you have on these Vive Pros that you're using. Maybe you could just talk about that process of, like, how do you think about describing those smells? and how to actually deliver it in a way that you are able to somewhat edit the smell over time in a way that is able to give distinct smells that disperse but also not interfere too much from the other characters.
[00:31:36.034] Marcel van Brakel: Of course we already worked on smells with for instance famous deaths where we had kind of the same technologic system but we transported the smells over a longer distance and of course in that project you're in a small chamber, a cooling cell in which we need more smell to pump in the space to have you smell it. But building from that, we came with the idea to put the smells closer to the body so there's less smell needed for you to have these sensations. It's actually really a micro-dose that we can give you and that's fairly quite simple. We can control the amount of air that we push through these valves and these canisters of smell. So, since we need not so much of a quantity, it stays with you and doesn't pollute into the space. Of course, after a long run, there will be some smell in the space and it's hard to get that out, but we have way more control over it than a couple of years ago.
[00:32:32.554] Kent Bye: And then, like, how do you think about, you know, describing the smells? How do you name it or how do you work with creating contrast with smell?
[00:32:39.600] Marcel van Brakel: Actually, I think working with the smell within the immersive experience is a lot like your sound editing. So sometimes you use smell as an atmospheric quality. So you can smell the space where you're at, the woods or the floor. And sometimes we use it as a special effect to kind of maybe directly probe your emotional parts in your brain stimulate them with also not only nice smells in a lot of like VR or XR experience it's also about pleasing the or having a nice environment for the audience but sometimes we also have a different smell and as a child It's also a learned thing that we kind of associate certain smells with things that are not pleasant or not good. But if you ask a child to kind of describe if their poo smells bad, they might not have this negative or positive judgment over it. They just experience it as something interesting. So, yeah, I think that's also interesting that we're trying to mix like the familiar with the unfamiliar. And it's actually quite hard to find the right smells, because sometimes we want a woody smell and you buy it, because we buy from some distributors, you get this perfume kind of thing that really doesn't smell like the real thing. So the perfume industry, of course, that's their business to make smells not realistic. So for us, it's always a journey to kind of find these smells that are kind of spot on.
[00:34:10.697] Kent Bye: Any other thoughts on working with smell, and how do you start to use language or talk about how it fits into the larger experiential design process?
[00:34:18.342] Brakel Mark Meeuwenoord: Well, it's super interesting to work with it. It's super hard. As you said, it's hard to control. It's way hard to control. It's super hard. But it's also very powerful. It precedes the cognition, and therefore you can direct emotion with it. I think we really took another step in controlling it in this piece with symbiosis, but also with making it a part of the story world and the aesthetics of the whole piece. You wear the smells in sort of external glands, which you described, there are these bulbs that are on the suit, but they're also functional. Because you're wearing them, the scents themselves, the containers actually are close to where they should be released. So I think that helps the whole system, the technological system of it. In previous works, the containers were a long way from the place that it should be released. So there's contamination in the line. So all these things you have to take into account. And what also is very interesting I think, so in different places people also maybe probably culturally react differently to the amount of sand actually being released. So we had this discussion and we set it up, we made the build up. You can debug a lot of stuff online. VR-wise, you can do stuff in all sorts of elements of the software. But it's very hard to actually, over a distance, to debug some kind of thing in scent, you know? Oh, what do you smell? You can ask someone, but it's like, I cannot see the numbers. So that was kind of a... So we had to actually adjust this week, we had to adjust some of the settings in the smell. talking to different people smelling and then like calculating an average in that and also thinking about so... Even having language to describe it can be difficult. Especially that, you know, there's so little vocabulary on talking about scent, you know. We're all amateurs on that, at least in our culture. We're super amateurs in talking about scent. There's some people that can do that really good, the perfumers and in the industry, but most people cannot. So that's interesting to work with, I have to say.
[00:36:32.590] Kent Bye: Yeah, even like wine tasters or people who have a whole vocabulary for describing taste in food or wine. So yeah, coming up with a vocabulary, I think, will be a part of this multi-sensory future as we continue to expand that out to find ways to talk about it as well. For me, I have trouble articulating it because it's at this kind of almost, like you said, precognitive or subconscious level that is impacting me maybe in drawing up memories, but it's I remember there was a piece called Sweet Dreams that was by Marshmello Laserfeast at Sundance and they had the taste of pop rocks and they asked me like what memories came up and it wasn't until they asked me what memories came up that then the memory came up and so then it was like a question of like oh wow to what degree are these senses working at this subconscious embodied level that is so subtle that it's like pre-language that we can't even articulate it exactly so it's It's one of those things, like you mentioned, very similar to sound design, that it's kind of in the background and stuff that's kind of blending in and stuff that you're not necessarily even articulating. But as you are laying in all these different aspects of reality, I think that's something that's on the frontiers of where things will go in the future. But yeah, just to kind of wrap things up here, I'm curious what each of you think the ultimate potential of immersive storytelling in a multi-sensory way might be and what it might be able to enable.
[00:37:54.906] Marcel van Brakel: I think the more we work with the technology, the more we also realize that we're this biochemical, electrochemical thing as a body. And that the technology also has to find ways to tap into that in another way than only this outside technology that has to have an interface. And it's also feeling kind of cold and techno. and I think to move towards a more feminine relationship with technology. For instance, within our project, the main technology is of course the soft robotics, or maybe not the main, but one of the important ones, that is feminine. It's not this hard metal like robotics, it's soft and feminine, and it's softer in interaction with the body. And I think I'm really curious about how we can develop that even more, or even maybe in a more radical way. Yeah, and I think for us, working in this field of AR, it's much about play and about experimentation, about figuring out stuff and also do stuff that's not been done before to kind of see what happens, actually, instead of, we don't know exactly how XR will develop, but I think it's interesting to kind of take it to places where it's not now. There's a lot of emphasis on VR. We're here at DocLab and it's all about VR, VR, VR, VR. And of course it's a really cool medium and we work with it, but for us it's just one of the options and not the answer.
[00:39:25.133] Kent Bye: Yeah, just a quick note on the language of feminine. I tend to think of it as like in terms of Chinese philosophy metaphors for like the yang and the yin, where the yang is much more outwardly facing and the yin is much more receptive. And so I feel like there's a sense of the embodied and emotional presence that's more receptive in a way that is receiving sensory input from the environment. Not necessarily to put like gendered language of feminine, but at least that's how I think about it in terms of like more receptive modes of technology. So yeah. But yeah, just for you, any thoughts on the ultimate potential of multi-sensory immersive experiences?
[00:39:56.923] Brakel Mark Meeuwenoord: It's impossible to actually answer that question because it's so... If I would say one thing, we should do less. because I feel overprivileged with the amount of access we have to all sorts of technology and technologies to come. And I think we should really, I don't know, it's a really double kind of schizophrenic almost kind of situation because you feel you can do all these things with it and you have access to it as an artist and more and more and more. Today we were discussing with scientists and we can do together with science and there's all these enormous oceans of possibilities. And I feel it's overprivileged or something. So I really think we should also appropriate these technologies to size them down or something, to be more humble with them. and not see it as like a super power tool for we can do anything and we can tell this very important stories as also we do, you know? But how can we size it down and make it less, do less with it and be more with less or something? I don't know. I'm struggling with the thought, but I feel really emotionally connected to that idea that I don't really understand yet. But yeah, there's something to do less.
[00:41:16.832] Kent Bye: Well, I had a chance to do these characters and they had a little booklet that I appreciated reading through afterwards. It has a lot more about the world that you've created and so there's a lot of world building that's going on and different stories. I feel like there's the different characters that you're able to kind of dive in and to get these different experiences and then this symbiotic aspect of the audience coming in and negotiating from before and after trying to figure out what character they want to have based upon the temperamental experiences they might want to have and then the unpacking of it all and trying to share each of the different stories and characters. I feel like there's a lot there for audiences to unpack here in Portland, Oregon until February. And so any other final thoughts or messages that you'd like to send to the immersive community as we wrap up here?
[00:41:58.530] Marcel van Brakel: Now we want to invite them to join in and to experience it and we're really proud that also Pampkut took a risk to have this kind of cutting-edge experiment. You might even say they're in this play for such a long time because it also takes guts because some things cannot work and people can be disappointed if things go wrong. But it's really special and I think also we notice when we're in America with new pieces there's a lot of enthusiasm and people really want to try out and to experience and they have an open mindset for it. It's really cool for us to experience. So yeah, I hope you all come.
[00:42:38.247] Brakel Mark Meeuwenoord: Yeah, yeah, thank you. Thank you Matt, Matt Henderson. Thank you for being with us in the project and also a lot of other people there, but because you're in the community there and Pampkwart of course, but... And Amy Dodson as well. Amy, she really stuck her neck out, you know, she's really out there and I remember speaking to her last year at IDFA. We had a really nice dinner and And now we're there, you know, and she made that possible. So, and the whole team there, it's been amazing. But, you know, for the visitors, you know, be open and honest about it. Be critical. I hope it's a piece that touches you. And I also hope that it's not the whole thing in the world to tell about these things, but it's a small thing we did. And I hope people engage with that and have an honest experience in it.
[00:43:27.485] Kent Bye: I think it's at PAMCUT, which I think is the Portland Art Museum's Center for Untold Tomorrow. It's 935 Salmon Street Northwest there in Portland, and just down the street from the main Portland Art Museum. It's on the corner on Salmon Street for anybody that's there in Portland. Go check it out until February. And yeah, thanks for helping unpack it all today on my podcast. So thank you.
[00:43:48.002] Marcel van Brakel: Thank you so much for having us.
[00:43:49.823] Brakel Mark Meeuwenoord: Thank you, Kent. Thank you.
[00:43:52.811] Kent Bye: So that was Marcel van Brackel as well as Mark Myrnord. They're both a part of Polymorph that created a piece called Symbiosis that showed at IFA DocLab 2021 and is currently showing at Portland Art Museum's Center for Untold Tomorrows from November up until February of 2023. I have a number of different takeaways about this interview. First of all, there's a lot of really innovative technology that they're integrating here, from the soft-body haptic suits into the different smells that are coming up, integrating taste in different ways. Unfortunately, when I saw it, the food that was being shipped over from Amsterdam was held up in customs, so I didn't have a chance to see how that played into my own experience. But I spent some time looking through this world-building that they're creating. There's Donald Airways' Staying With the Trouble as a book. And then they created a pamphlet of the symbiosis that goes into all the backstory and the philosophy and the background and actually has the script that they have for each of these different characters. And just in looking at Staying with the Trouble, there is this concept that is a philosophical thought experiment in some ways. And in their interpretation of it, they're integrating it, not just the integration with other species to create this kinship, but the integration with the technology as well is like the nanorobotics and this other kind of more speculative, transhumanist interpretation and spin on Donna Haraway, which I'm not sure if she has as much. I haven't read all of Staying with the Trouble, but from what I saw, it was more focused on building kinship with other multi-species futures and how to create empathy and thinking about how do we have humanity think more about how to be in relationship to the world around us. So there's actually a lot of rich philosophical ideas. And I think some of the different characters explore that in more rich detail, like the toad, as well as the monarch butterfly. I think probably explore those in the most specific way. But again, as you go through these environments, a lot of the narration that's coming in is through the words that are coming in. And so there's a leaning heavily on the script that you're hearing as you're going through this world. And it's interpreting all these different dimensions of the world that they're creating. And, yeah, the slime mold is a little bit more of an abstraction of this, but I think the visceral haptic experience was really strong. And then the multi-body, again, kind of gets into all these different experimental aspects that I think are appropriate for a film festival, but I'm not sure how well some of these different types of ideas that they were trying to experiment, how that translates into a general audience. You know, just the idea of having another person move the head of another person and how do you connect the embodiments of other people, but as a Tamagotchi AI, I had no context as to what the rest of my multibody was seeing, so there was no way for me to contextualize what I was seeing or hearing in that experience and how that connected to others. One thing that I'll say is that there is a wide variety of different types of experience. They created one of the Tamagotchi AI, which, experiencing it and hearing their explanations, I'm not entirely convinced that this is a good idea, to do this type of experience that's going to be deliberately annoying to the users, in order to create this social dynamic afterwards, where they're trying to have these different conversations of that experience, and to have something that is so abstract and so lacking in the story, and something that is repetitive in a sense that it's just repeating singular words, and it's not a satisfying narrative experience like the other experiences were. And so their intent is to deliberately create this type of annoyance in order to have different types of conversations at the end. But I think in terms of just trying to promote the structures and forms of immersive storytelling and VR as a medium, to have one experience where you're deliberately avoiding some of the affordances of the medium, it's a little bit experimental. That would be OK in a film festival context. But when you start to broaden it out to the larger mainstream, this may be someone's first and only VR experience. And they kind of walk away from it like, wow, that was a bit of a nothing burger. I think there's a larger risk of saying that there are individuals that are doing it, so they're trying to focus on just the collective experience, but you're sacrificing someone's enjoyment just to create the potential of this larger collective dynamic. That's the one big critique that I have around it. I think it would also be a lot more powerful if you did have a first-person embodiment, because you are a ghostly apparition in these experiences. And so, as you're experiencing these different haptic and sensory experiences, and the absence of having a type of virtual representation of a body in these experiences takes away from me feeling really deeply present in these different worlds. It kind of creates an abstraction of this. I'm a ghostly, omniscient perspective, but yet I'm experiencing all these visceral haptic experiences. I do think the full embodiment of these experiences would take it to the next level. So what they were saying is that they were trying to get away from just the normal anthropomorphic skeletal embodiments that you have in VR. And you do have to start with that because we do have these limbs and they were talking about trying to lay down, but they ended up that they weren't able to integrate these full body track embodied experiences, especially as they have like just a couple of white houses and there's six different people and they're able to get the VR experiences and have the head tracking. But have the full body tracking and all the different trackers and five trackers and keeping all the the batteries and you know it's just a bit of a logistical nightmare at that scale and they didn't want to create more instability into the system which i totally understand but i think there is this aspect of when you have this really deep haptic experience without having any sort of body representation as to what's happening then it takes away from some of that. The smells that they had, there wasn't any sort of associated. They're kind of unique smells. When you have unique smells, it's not necessarily tapping into any of my memories about what those smells mean. They were just kind of more of a novel smell. I guess the tastes were also trying to be novel in that sense. I didn't have a chance to experience that component, so I can't really speak to it. But from some people I talked to last year, They weren't necessarily a fan of some of the different tastes just because they didn't taste good. And so, again, that's them experimenting and deliberately using these more negative contrasts of something that isn't always pleasant and how does that play into the larger experience. So, yeah, like I said, there's a lot of really interesting technology in the back end for the air pneumatics in this off-body haptic suit. And, you know, it's a very ambitious project and experimenting with all this multisensory aspect. And I think there is a An interesting philosophical context, I'd recommend folks to go check out Donna Haraway's Staying with the Trouble to read into more of the inspiration and to get their book to read more about it. There is this potential to create this social dynamic after you experience it to compare and contrast some of the different experiences that people may have and some of their takeaways and emotions. So yeah, that's Symbiosis by Polymorph. It's currently showing at PAMCUT. That's the Portland Art Museum Center for Untold Tomorrows. Amy Dotson and the whole crew there. Hope to catch up with her at some point just to hear how the different art museums are starting to bring these different experiences and what some of the reactions to this piece have been as there's been the Venice Film Festival VR expanded for last couple of years have had satellite exhibitions here at the Portland Art Museum. And yeah, just exciting to hear some of the different immersive storytelling Experiences getting outlets to some of these more regional art museums and happens to be here right here in my backyard So glad to be able to check it out and to unpack it a little bit with the creators So that's all I have for today And I just wanted to thank you for listening to the voices of ER podcast And if you enjoy the podcast and please do spread the word tell your friends and consider becoming a member of the patreon This is a this is border podcast and I do rely upon donations from people like yourself in order to continue bringing this coverage So you could become a member and donate today at patreon.com slash voices of ER Thanks for listening.
