Chelley Sherman is an XR artist who has been working in VR for 4 years, creating audio-visual performances, and exploring how to created heightened, highly-perceptive experiences. Sherman says that her experience with scoliosis early in her life severely impacted how she perceived her body, and that one of her first experiences in VR completely transformed her sense of identity and body representation. She was able to see a volumetric capture of her body using Depthkit technology, which was able to giver her a completely new perspective on her body in what she describes a form of phantom limb therapy or mirror box therapy. She’s been hooked on the visceral power of VR ever since pushing the boundaries of perception by creating art that explores the cross-section of the mind and body, with a particular focus on trauma therapies that use techniques like EMDR, brain entrainment, and vibrotactical experiences using the SubPac and other haptics technologies like UltraHaptics.
Sherman was the featured VR artist at the immersive screening of the UN Women Global Film Festival in San Mateo, CA in May 2019, and I had a chance to sit down with her to learn more about her journey into VR and some context about of some her creative explorations. We talk about the sonic architecture of spaces, the importance of embodiment & visceral haptics, and some of her explorations in the frontiers of neuroscience and chaos theory looking at topics ranging from creating differential growth patterns, coral reefs, diffusion reaction patterns, self-organizing patterns, flocking Patterns, swarm patterns, and dynamical cognitive systems. We also talk about the challenges around disassociation, including how multi-participant VR experiences with an AR app on a phone still feel isolated. Finally, we talk about the need for more sophisticated critical discourse about virtual reality work, and the unique challenges around solidifying a brand and identity around success and career advancement versus the need for anonymity for creative experimentation in a context that allows for innovation being born out of many iterations with failures that include fast cycles of candid and authentic feedback.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
Here’s a video of Sherman’s Dispersion piece that was showing at the UN Women Global Film Festival, as well as at the Immersive Design Summit party before the opening day.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So continuing on in my series of talking to virtual reality artists about their process, today's episode features Shelly Sherman, who's an XR artist who's been working on VR for about four years now. So Shelly's got a very distinct style of creating these really lifelike entities and forms and sculptures. And I first saw one of her pieces back at the Art of Dying show back in 2015, where she had a piece there that was kind of like these multiple layers of inception. She's somebody who first got into virtual reality, really changed her own sense of her body image and self-image. And she tells a story of that and also just very interested in more of the cognitive science and neuroscience perspectives and mashing that up with the different mathematical formulas and kind of blending this together with this highly technical fusion of the mind and the body. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Shelly happened on Thursday, May 16th, 2019 at the UN Women Global Film Festival, where there was a whole VR night and she was featured as the featured artist there. And that was happening in San Mateo, California. So with that, let's go ahead and dive right in.
[00:01:30.852] Chelley Sherman: So my name's Shelly Sherman. I'm an XR artist here in the Bay. And I've been predominantly working in VR for the past, I guess now about four years, but also doing things like AV performances and interactive installations. But I've been sort of like merging all of my previous experiences and expertise kind of in this realm to sort of figure out how to heighten these highly perceptory experiences.
[00:01:57.400] Kent Bye: Can you talk a bit more about your background and your journey into virtual reality?
[00:02:02.104] Chelley Sherman: Yeah, it's a very interesting one that I talk about a lot. So when I, I was first working with a friend of mine and he was doing a project, I'd never touched anything in VR before. I was working with mobile apps and doing installations and a lot of audio visual performances. But he had asked me to help him with a piece where we were taking a volumetric image using the depth kit in VR to do a kind of performance, an AV performance in VR. So I actually I have scoliosis and I was operated on when I was younger So we were testing these videos of ourself and I was able to get this three-dimensional movie capture of myself and kind of walk around myself and I realized that the way that I started to perceive my body in VR and and in my own body immediately changed. So something that I didn't quite understand or have words for, I always had a bit of this body dysmorphia. And just having your entire spine fused at 13 years old and having these portions of your body atrophy and things were asymmetrical, I didn't really have a good sense of ownership in my body. And being able to see myself outside of myself in this virtual space, not as like a mirrored image reflection, but, you know, kind of be like disembodied from my own self and objectively, you know, walk around myself. I had this much better understanding. of my body and that, outside of the experience that we were working on, it hit me in such a profound way where I realized that this medium that I didn't fully understand or wasn't completely sold on was just incredibly powerful and I wanted to continue to explore perception. And what I didn't understand at that point, you know, I had always like read books by like Vyavarman Chandran at a young age and realized that what I was experiencing was like phantom limb therapy, mirror box therapy, and having this like disembodied version of myself, my brain was immediately able to reimagine, reconfigure my own self in this physical world, in this physical space. Yeah, so I kind of continued to research different kinds of perceptual illusions, vibrotactile illusions, out-of-body experiences in VR, things like how you would incorporate rubber hand illusions in VR, and how I could also utilize those experiences to heighten these perceptory experiences in not only a visual format, visual context, but also using
[00:04:39.194] Kent Bye: Sound and light therapy techniques as well kind of like EMDR and just you know been sort of like exploring that realm since I Think the first time I saw one of your pieces was in 2015 at the death and dying show and that October and Wondering if you could talk a bit about the piece that you had there
[00:04:55.778] Chelley Sherman: Yeah, so yeah, I was for the Art of Dying event and I had created this piece called Das East, where you're traversing through different sort of like Bardo's and using portals as a way of how to move through a virtual space without controllers. So you kind of enter this realm, you're disembodied, you don't have an avatar. This is sort of like an afterlife experience. It's very unnerving, unsettling, and kind of using these light and sound therapy techniques and also just being able to move throughout a space and kind of like create these calming effects and experience a narrative where you have kind of like your own agency to move freely within a space and circle through these different portals or dimensions. I had a friend of mine who has since passed but he did the sound design for this piece as well because he also had a very deep understanding like on a sonic level how to really tap into that narrative and work with this kind of like audio stimulus within a virtual space and I thought you know, like more perfect than virtual reality and using spatial audio to really place these cues, these sonic cues in VR and really use your entire body to move through and explore these different monuments within the experience. And just like seeing that, you know, showing it around and presenting it, getting this feedback from a lot of people that it was very visceral how they were able to feel a lot of these somewhat geometries, kind of like Hobweb sort of geometries, move through them in a space. And I learned a lot more even out of that as sort of research every single time I'd show it out, kind of getting feedback from people, how much deeper the experience was for them.
[00:06:42.034] Kent Bye: Yeah, it's probably about three and a half years ago or so that I saw that but what I remember is that it was like my body moving through space but it was almost like this labyrinth. It reminded me a little bit of like Inception where you have like VR worlds within VR worlds within VR worlds like this experience of walking through this space but yet coming out of VR being like completely lost and I really was like that like kind of being disoriented by going into a virtual space and coming out and actually not knowing what direction you're facing just because you got so turned around in the experience. And I felt like you're able to do that to some extent. But you're talking a little bit about EMDR and these different techniques to induce these different levels of brain entrainment. And I've experimented a lot with different things like Hemisync and Holosync with binaural beats, which is being able to actually induce brainwave states that are from that level of entrainment. Is that kind of what we're trying to do in terms of more of a visual entrainment? Or maybe you could talk a bit about what you're doing and looking at in terms of either the EMDR therapies, eye movement.
[00:07:38.773] Chelley Sherman: Eye movement desensitization reprocessing.
[00:07:41.474] Kent Bye: Right, so EMDR and then all these other techniques, how you're kind of tying all that together.
[00:07:45.077] Chelley Sherman: Yeah. Well, there's something, the more that I start working on these experiences, I realize a lot of it just is programmed from a background of doing audiovisual work and really working in these performative, what's called rave experiences, where people are so tuned into the sonic architecture of a space and more concerned with body movement. You have these artists that are so concentrated on like creating like a palette of percussiveness and just like emotion through like the work, the audio work that they're creating and producing. And, you know, with the Dasis experience, not even necessarily just like the binaural audio, So the thought behind it is having this outwardly stimulus, which is either, you know, lights flashing back and forth from your view, having not binaural audio per se, but you're having like a ticking back and forth from the left side to the right side of the ear or some kind of like vibrotactile response within the hands. And what that does as you're going through a traumatic or uncomfortable engagement or thought of an experience, it taxes the working memory. And so to be able to move through these sort of like chaotic experiences and find a stasis where your body isn't as responsive, so you're kind of like engaging in a somatic response of these very uncomfortable experiences. And so what I thought was really interesting too, just being a spectator of like observing the people that are within my own experiences, seeing how I can, force them to move and interact with the space by these audio cues. At the end of the experience, there's this one sort of portal that's like an altar on the ground, but what you have to do is get on your knees and sort of crawl through it. So I'm creating, this is weird. I'm watching people engage with it, and I'm like, it's weird, I kind of feel like God. I'm forcing them to get down into the space, and not through any kind of UI, but as a, They're engaging with the space where they feel like they have free agency, but really trying to understand what piques people's interest in certain kinds of movements and let that just be a more natural, free-flowing engagement with the piece.
[00:09:58.593] Kent Bye: Yeah, as you were talking about the working memory and kind of overloading it with the EMDR techniques and I know that EMDR is used quite a bit in treating trauma and so I could just imagine how just VR as a medium itself is like really overloading the working memory all the time and that maybe there's something there in terms of the therapeutic effects of VR to be able to potentially do either explicitly exposure therapy or just more generally be able to give people a context that allows them to move in a way that is more ecstatic in the sense that it's Different from what their normal level of stasis is but that they're able to have this context that allows them to tap into moving around in a different way, but also when you start to add in music and That's where I find that's super fascinating, where you're able to now visualize different components of the music and give this whole full sensory experience that is really trying to inspire people to either be connected to the music in a new way or to move in a new way. And I'm just curious to hear a little bit more about your background in this cross-section between the art and the live performance and music.
[00:11:02.138] Chelley Sherman: Yeah, you know, so it's funny because it's also very much tied into that like initial experience that I told you of not just like body dysmorphia, but what happened after I had surgery. And I noticed that, you know, there's so much in a very like fundamental like foundational level. that I had these like tensions really tied into my body that I didn't really understand. So coming from this like AV background and noticing that like in these spaces, that's where I would feel better. It's cathartic. Like even last night, I was out seeing a performance of some friends of mine that was it was just absolutely like chaotic and loud and very, like, sub-bass heavy. And, you know, we had, like, a very intense weekend that I told you about. And so, you know, like, just being able to engage in those spaces and being on the West Coast and people are so much more in tuned with the mind-body connection of, like, where traumas are, like, doing, like, body work. And, you know, I came from a very wary of the woo. In and like highly scientific background, but you see a lot of this like merging now in trauma therapy clinics They're really pushing to have people go do yoga and get into like Eastern medicine or body work and moving, you know throughout the body and yeah, and so the way that that work started to develop a little bit further was in the dispersion piece that I created and I what I really wanted to do was sort of like create in this sonic sculptural environment where it's picking up off of your movement and interaction within the space also using like a feedback synthesis technique that My collaborator and good friend of mine, Chris Latina, who's an engineer at Dolby Labs, had created. So it's this high-dimensional version of not just a one-to-one experience of sound to light, sound to some kind of movement. What you do in like any kind of like AV performance, but there's a constant feedback of this sonic sculpture that then sort of creates a life of its own and I Occasionally use like a sub pack and I've been using sub pack vests in this experience for right now like a cheap way of kind of like integrating that fibro-tactile response to kind of like have this like sense of body ownership, but then see this sculpture sort of like move away from you and create this like really pretty psychedelic experience of this piece that you're kind of communicating with, but it's still like a bit of an extension of you. And with that as well, it's always like a research and exploration because the same thing every single time I show it. I get feedback and listen to how like the sensation that it's caused for other people and their own engagement with it and using that as a way to kind of like keep developing the meaning behind the works as well.
[00:13:55.889] Kent Bye: Well, I wanted to ask you a bit about an algo rave because I've heard about the algo raves at South by Southwest where someone who had just gone to an algo rave. So what is an algo rave?
[00:14:07.993] Chelley Sherman: Yeah, so an algo rave, it's all code-generated music. So you don't have software. The people that are performing are live coding. And it's funny because we talk about these rave experiences where it's cathartic and everyone's dancing. I don't get that as much. because it is very chin-strokey and really heady. And it's really funny because the last one that we did a talk at, I'm looking around and we're all just a bunch of nerds, music nerds, whatever, and looking around and I'm like, wow, this is so funny because there's certain genres of music that it begs for a certain kind of dance and a certain kind of movement and that's sort of what you're coercing with the music that you're putting out there. You see people just typing away, and then they're only moving their neck and twitching their neck around and twitching their eyes. And I'm like, total algorave style, total algorave aesthetic.
[00:15:10.009] Kent Bye: It sounds like it could go horribly wrong, or just like you're watching somebody live code. But yet, is it more like experimental music, or is it more rhythmic? Or what does it sound like?
[00:15:19.547] Chelley Sherman: depends on how it's been coded. It can be rhythmic, but it is very experimental. And it's a lot of very intentional listening. Because working with software to, let's say I'm live coding a VR experience. It might not be as polished. But that's kind of part of it. And in these algorithms, too, what you're doing is you're watching people code this experience. So it's not this multi-sensory festival rave. you're really trying to understand the inner workings of this computer, so it's all computer music. And yeah, it's more hacker style, and it's a completely different engagement with music and a festival setting, whatever, and yeah, so they're different.
[00:16:08.440] Kent Bye: Well, I haven't had a chance to go to an algo rave, but as I hear a description, I can imagine what might some of it be like, is like you're watching them code and that you're trying to, in some ways, make a prediction as to what's going to happen based upon what you can understand if you understand what the programming language is. If you don't understand the programming languages, it may be completely lost on you what they're actually doing. But there seems to be a certain amount of having correlated to what they're typing and what they're doing, and then seeing how the sound's different once they do that.
[00:16:37.653] Chelley Sherman: Yeah, totally. And you know, so like, let's say the last one that we'd went to Mark fell, who's a great musician and composer. However, he didn't have anything projected. So it was very intentional listening. You knew he was typing away at something, but there was nothing really to help you connect and understand what was going on, because he's more of like a master at that kind of craft. And not to say that the rest of them aren't, but you know, it's the intention of how you want the audience to listen. But yeah, you know, like you do kind of have to have an understanding of the programming language. Like, my friends and I who do code and do program, we're reading. You can see them. They're like little notes hidden within the structure of their code. And so it can be really funny, because it's like they're kind of talking to you. Or there will just be messages that these error codes will just start filling up the screen. And it's just like, it's really silly. It just reminds me of old, I don't know, just weird LAN party culture. Just like we're like hacker culture, you know, so it's not like polished or refined as you know Something that's been like heavily mastered you're like appreciating like what is going on in a computer? You know, like what is being synthesized in a computer and like under the hood of everything else that you know is being built So it's yeah, you know, it's it's like a club. It's like a club for programmers. I
[00:18:03.612] Kent Bye: Well, I know I know there's been a lot of like just live streaming so people are often playing games But there's some people who do like creative coding and like just are making stuff in unity But they're live streaming it and you may be able to kind of tune in but this seems like there's this element of the live performance But also the dialogue that happens with the audience there but also this deeper pattern that is in your work in other ways, which is just the algorithmic nature of Trying to see these different patterns of surprise or chaos or algorithms the sort of like the unpredictability of an experience and so where's that go back to in your work in terms of like that algorithmic aspect of Having a little bit of chaos or unpredictability to it
[00:18:45.378] Chelley Sherman: Yeah, well, I mean, the human mind, it seeks pattern. It is built on self-organization. And I got really in doing more, not visual programming, but doing more 3D graphics, realizing that, doing like growth systems, you know, like trying to create differential growth systems, things that look like what you would find in nature, like coral reef patterns or like diffusion reaction patterns. And you find them in all aspects of nature, you know, like you go out and you're like walking and hiking and you find these similar patterns. And I was like, I always wanted to recreate that in the textures of my work and the textural palette of my work. And then I started to notice like, you know, I got really into self-organization patterns on a cognitive level, just being able to predict human patterns, and really fascinated just on a basic level of how we could recreate flocking patterns and swarm patterns in these computational systems and simulate them. And I always go back to really like exploring like cognitive science and trying to emulate these things in the pieces that I'm working on. So I had started reading up on like dynamical cognitive systems and how, you know, even within the human mind. the way that we experience chaos, it can be simulated within the brain. Researchers are trying to figure out how to use those exact same patterns that we're talking about as a way to create stasis in dissociative experiences. To me, that is my biggest trip, to see how these systems that happen in a macrocosmic scale, within how galaxy clusters and universes are forming, to a microcosmic scale. for me, like Albert Einstein trying to find the general relativity, I think is just really fascinating to try to incorporate that in so many facets, just symbolically in the work, and keep exploring it through that way.
[00:20:47.019] Kent Bye: Yeah, I'm talking to Sam Roberts. He's a programmer at Indicade. He was talking about how games can allow you to have a direct embodied experience of complex systems, where you're able to make choices and take action, but then you're able to see how your actions may be propagated over time in a complex, nonlinear system. And the more that you play the game and make different choices, then you start to understand the topography of that landscape of the different possibilities and the causal chains of you taking action and this is the effect. But there's some aspects of life that get so complicated that you really need these non-linear chaos theory ways of exploring more probabilistic structures of something like quantum mechanics as an example where it's something that you may not have a good intuition with until you have a chance to actually play with it and see how it kind of works. And I feel like that's a little bit of what VR affords us is all these new ways to have these spatial metaphors of either mathematics or these concepts or ideas that we've never had a direct interactive ability to be able to express our agency and play with in any way. But that, by opening up that possibility, I feel like there's all these new realms of possibility for artists like yourself to pull in all these deeper concepts and then to start to play with them in your art pieces. And then, I guess the question is, to what end? What would you hope that the viewer who's experiencing it would be able to get out of that?
[00:22:12.407] Chelley Sherman: Yeah, so I'm glad that you said that it gives you, that's exactly what it's doing. It's giving you a higher intuition of your own input into the system. So let's say we're looking at light and sound synthesis. If you're listening to a waveform, if you have some kind of visual feedback, you have a higher intuition of what you're listening to. And that's exactly the same concept of how I'm trying to engage these topics as well. So the way that you interact in the games that you were just talking about is giving you an ability to perceive not only a better perception of themselves, but the way that they fit into kind of the larger system of humanity and what their impact is, not just on a political or sociopolitical level, but really just with the way that you engage in the world on a very fundamental level.
[00:23:07.213] Kent Bye: Yeah, and I'm curious what type of things you're looking at now, because it sounds like you're doing this deep research into a lot of these higher order math structures or cognitive science issues. But what are you looking at to be able to feed into your work right now?
[00:23:20.045] Chelley Sherman: So what I was talking about before, dynamical systems in cognitive science, how I can use that and sort of abstract that both in light and sound therapy techniques. And one thing that has been a big concern for me in VR is how highly dissociative it is. And so being able to kind of reintegrate these dissociative experiences, and I guess it sort of follows me along in a lot of the pieces that I do. We move into these highly dissociative states, how we can reprocess and reintegrate and sort of create a deeper foundation, like cognitive strengthening. So that's a piece that I'm researching. It's in a research level right now and trying to develop is going back and forth and pendulating between highly chaotic and static states and how we can kind of use that as like an exercise and a tool but in an abstracted and artistic sense.
[00:24:13.566] Kent Bye: Yeah, when you were talking about being able to connect either a visual representation to what you're hearing, it made me think of David Eagleman in the Neosensory Vest, where he's able to take sound from an environment and then translate it into a series of haptic input into your body so that it's essentially in the same data structure as what your ear would translate and send to your brain, so it's like kind of a replacement for your cochlea. But you're able to turn your torso into an ear. So it's like this concept of sensory replacement, or to be able to take in and cultivate new senses to have sensory addition. But the key component part there was the visual synchrony and being able to have your brain be able to connect the dots and to notice those patterns, because our brain is a pattern-recognizing machine. as long as you're able to give it some sort of visual representation at the same time as you're listening to things, then I think there's all sorts of really interesting possibilities for this kind of like induced synesthesia that we have for being able to hear things, but able to connect it to different feelings or to different visual representations. And so I think there's a lot of really rich areas that can be done in, especially in music visualization, just because music to me is like this frontier of so much depth of mystery. You know, to some extent, Beat Saber is taking a very rhythmic approach of translating the rhythm that's in that experience into a spatial representation that you have an embodied experience of. But I feel like there's a whole other layer of, like, the depth of music theory, of the chord structures and the different levels of orchestration that has yet to be really fully explored and pushed to the limits in terms of music visualization. But that sounds like something that you've also looked into. I'm just curious to hear what your thoughts are and the potential of being able to visualize music in VR.
[00:25:56.500] Chelley Sherman: Yeah, so in regards to what you were talking about with having the haptic vest as well, that's something that I've been trying to integrate in the pieces. Dispersion is another, I have another format of it as a screen. And so what I'm looking into is using a sonic, like a haptic board. UltraHaptics has created this ultrasound board where you can move your hand through it and have a much more deeper visceral response to the sculpture in itself. And so, like hearing stories of people who've lost their vision and having implants within their tongue that use like the optic data to use like the surface of the tongue as another form of vision, you know. It's really interesting to me that we're born in these bodies and in this consciousness thinking that we're limited to just these sensations and I'd love to see, you know, how human consciousness and just the human species can evolve and adapt and readapt and really keep pushing the limits of our perception of just any sensory experience to have a much deeper dialogue with the media that we're engaging with and just the topics of discussion as well that we're engaging with as well in the works that we're talking about. Especially in the dispersion piece, what I'm trying to build are these sonic audio sources, figuring out how to draw sound within a space, not just within a waveform, how it could be drawn with these kinds of system organization algorithms. having deeper sensory experiences in this three-dimensional format but also that time format that you're talking about where you can move in time and have like free agency of like moving around these sound sources and audio structures and have a completely different perception of your experience in the space dependent on how you're also inputting information and your own data within that environment.
[00:27:49.937] Kent Bye: Yeah, and you talked about the discussion of pieces, and we were talking earlier about the emerging field of criticism within virtual and immersive experiences, and it's still very nascent, I'd say, but what would you like to see in terms of a deeper discussion about critique about different pieces of immersive art and work?
[00:28:07.412] Chelley Sherman: I really would like the communities to be more critical, not negative, but more critical. Right now we're kind of, I'm glad we're at the UN Women's Global Voices Film Festival, and there is a lot of cheering, which is great within our communities to be supportive, but then I'm also seeing a lack of really being able to dig in and critique works and figure out how they need to, what their importance is. what the implications of the work are, implication of these experiences, and where we're sort of falling short in not just the way that experiences are designed and presented, but also just the critical dialogue around the work. And so what I'm seeing now is just sort of like an oversimplification. And I've talked about this before, sort of like the Twitterfication you know, of like critical dialogue where we only have like an attention span to just sort of like sift through a little bit of information of like any of the inundation of information and experiences that we've been seeing. So we don't actually really get to get into the meat of what these experiences are about unless we're like on a one-to-one level and just having a discussion. because if we go online, we're just seeing this thing is cool and I did this and here's a pat on the back. So yeah, I think that people right now need to be much more critical of the work that they're doing and their impact, not only their own impact, but trying to push the conversation of this field and pretty much journalism in general. And yeah.
[00:29:44.535] Kent Bye: Yeah, I think it's actually been coming up a lot more in the different spaces, especially when talking about different film festival pieces or whatnot. And I have a very specific approach and strategy. And I've thought a lot about this, because in some sense, I'm engaged in a dialogue with the creators. And I feel like, in some ways, I want to undermine my own authority. And I don't want to be like a kingmaker saying, this is an amazing experience. Absolutely must see because in some ways I can really enjoy experience But I have no idea if you're gonna like have the same experience with that and so for me I always like to kind of withhold my final thoughts on my critique of an experience until I have a chance to talk to the creator and ask them to get to the essence of of what their artistic intentions are. Because I find that in some ways, even if I hate an experience, then if I talk to the creator, and then to try to isolate, this is what you wanted to create with the experience, but this is how it actually feels. So there's a disconnect there. So what is it about the experiential design that needs to be improved or modified to be able to maybe get closer to your intention, or to maybe change your intention if it's based upon bad premises, for example. But I feel like the podcasting medium is actually a great, Medium to at least engage it more as a conversation rather than like here's a linear review with all the different bullet points Because I'm a journalist I get to see a lot of things that a lot of people don't also get to see and so it's also like trying to describe the experience enough so that people could get enough information about the character and the quality of that experience to know if that's something that they might be into and and to try to figure out the language to be able to help connect what the essence and the character of an experience is with someone who would want to see it. So there's a little bit of just a connecting to an experience to the people who want to see it that I see that there's a part of that role, but also just to be critical on certain things when I need to be, but also to bring up those criticisms directly to the people who created that experience when possible. And that's not always possible, but yeah, but to sort of think of it more of as a unfolding conversation that is mini conversations that are unfolding and following. And that just as you do user feedback and user testing, that you get some feedback, but seeing it more as a process to see how things can still continue and change and grow and evolve.
[00:32:00.126] Chelley Sherman: Yeah, I think your approach is really important. And you know, because you don't want to walk into an experience with a bias on what the intention was, and then you're kind of like under the lens of what their intention was in building a piece, but then you could lose the ability to be as critical. And so you do want to understand where things have fallen short. And so I do kind of wonder, though, in any sense, having any platform being showcased, even I notice now, if my name is posted somewhere online or if I'm doing a podcast and I'm on your site, I already, the way that people view and perceive my work in general, they're going to already have a sort of biased opinion based off of just having a complete blank slate and putting them into the experience as well. In some of the work that I do too, I'm now a lot more interested in sort of going under different monikers, not having my own name attached to a piece so I can actually get legitimate feedback. Because I've noticed that the longer I've been in the field, people have been engaging with me a lot differently and might not be telling me valuable information that I would like to know and like to hear. And I'm also kind of like navigating this realm, trying to survive as an artist and then continue to, you know, advance my career. So the kind of like the whole structure of this is sort of, you know, it's a little backwards because we're, you know, we're trying to thrive and we're trying to strive and put our names around the work and like just any conversation, anyone, you know, listening about the work that I'm creating without experiencing it first, regardless of however critical you were, they're going to take that information into the piece. And so I'm kind of excited to be at a place where I can just put something out and not have my name attached to something and just do like a complete slate of like exploration as well, because I'm interested to see how that starts to change within the dialogue or even, you know, regardless of whether or not somebody can recognize the work that I do based off of whatever artistic imprint I'm gonna leave on the piece, but I think that engaging with your own work and other people's work in that way and being highly mindful of whatever biased opinion you may have in the piece is really important to kind of make sure that you're keeping that clean state of highly critical advancement of the field, so yeah.
[00:34:23.457] Kent Bye: It's always a challenge because I'm going to see these experiences that maybe sometimes only 200 people in the world will be able to see. And then there's certain concepts and ideas about that experience, but yet I don't know if people are going to see that experience or want to see that experience. And so I'm always like, my preference is for you to see this experience first before you listen to this and then listen to it. But that's not always possible.
[00:34:47.290] Chelley Sherman: It's so hard. It's so hard to feel. Because even now, I have a lot of friends that they'll see the video of what I do, but they don't have a headset. And not all of them can come to my studio. And it's not easily accessible. But they have an idea of the work, but it's not until they actually get into it where they have a very deep understanding of it. So that is one of the biggest challenges of working in VR. And even for a while, I decided that I wanted to work in more of an installation setting. go back into a performance style or social experience that's the isolated version of VR because although to me it's one of the most impactful experiences, we haven't gotten to that place where everybody can easily dive into it and also not completely just, if everybody's just wandering around in a headset, you're not all completely removed from reality too. So yeah, it's a challenge that I'm hoping is resolved in the next decade. But I know that we were talking about it earlier, too, where you're having more of these dome and sphere experiences. And while the possibilities of engagement and immersive experiences in those large-scale settings are vast, you don't get that same sensation as being completely isolated within the headset and going through a virtual experience.
[00:36:05.990] Kent Bye: I think you're the first person I've talked to who's talked about the desire or need to want to be completely anonymous of you being attached to work so that you get either more candid feedback or just more of a blank slate. Because I feel like the whole essence of so many people is to build their brand and their name. If people like Marshmallow Laserfeast as an example, I love their work. And so I'll probably go see everything they ever create because I just love them as creators and what they've done before. But at the same time, those previous experiences may be coloring what I have to say and think about their ones in the future. But yeah, it's just I haven't heard other people wanting to actually just kind of erase themselves so completely as to not even have your name attached. So do you feel like that's an issue there?
[00:36:49.078] Chelley Sherman: Absolutely. I mean, like, what happens when any person in their artistic career, like, when they develop to a certain point, you know, you have this, like, struggle and battle with the ego. And sure, you know, like, like, large scale studios are working with a lot of different artists. And I think that having a brand in that sense, where you're also facilitating the artworks and the work of different people and beings within that entity, important but building a brand then it forbids you to do any kind of like strange exploration into another space because you you know I've like noticed within my like just the like unhealthy association that I have with like my name being put out somewhere publicly the way that I've like seen how people react differently to the work that I'm doing in regards to like status and the way that I have then at some point become more fearful of speaking freely, or not even that I'm going into anything that's too chaotic or against the stream. But yeah, it challenges, at least to me, what it means to be an artist. The more that I can remove myself from engaging in a highly egotistical structure, make sure that the intention of what I'm doing is pure, at least to me. Because if I'm creating works and it's based off of my ego and my brand, it feels like I'm selling the work in itself. and just my own appreciation for how I'm engaging in continuing to create. And for me, it's less about the name or the brand, it's about the intention. And so if you can put a name on my intention, then take it. Yeah.
[00:38:34.378] Kent Bye: So for you, what are some of the either biggest open questions you're trying to answer or biggest problems that you're trying to solve?
[00:38:42.075] Chelley Sherman: I think what we had talked about earlier is the highly profound experience of being in VR with the detachment of what you have in a social setting. And even with the ability to share these experiences more openly and to more people, So figuring out how to bridge that gap, I've also explored having multi-participant VR experiences in a social setting connected by an AR app on a phone, viewing whatever is going on in the virtual world, but then you're still, in a sense, dissociated. That's like, based off of the tools that we have, that's one way of trying to solve that problem for the time being. But bridging that gap, to me, has still been very difficult because even in these social VR experiences, you don't get the sensation of being able to view somebody as very very complex bodily responses or just like feeling what like the energy in the space of a room is or you know we have Helped out with the piece the wavy our piece with image and heat and what's missing out of that context even though you're in a social setting is Just even like the heavy frequencies that you can feel like on the hairs of your arms you know like if you're like in a performance setting you're feeling like the thumping and of heavy bass in your body, and it's literally, it's reprogramming you. It's literally ringing out your entire system, and so emulating those experiences, we're just waiting for technology to catch up, and so still exploring just how far you can go in the medium, as for right now.
[00:40:29.018] Kent Bye: I actually saw that piece in somebody's lab that had a vibrotactile platform. So I actually did feel the base, but from the haptics of the ground. So I feel like that there's going to be a lot of stuff that is trying to, in some ways, mimic things. If you can't have the actual huge subwoofers, but you're able to actually have haptic platforms. But finally, what do you think the ultimate potential of virtual reality is, and what it might be able to enable?
[00:40:57.368] Chelley Sherman: Well, I think, I mean, the fact that I was able to, years ago, have this experience that completely changed the way that I perceived myself for the better and really get myself out of my mind, I think that these experiences that not only give us a better understanding of ourselves, have a better understanding of other people around us, more like empathetic experiences, But I don't even know if we can say, I think for me it's at this point where it's sort of like unlocking these mysteries that as I'm even developing these experiences, the feedback and the responses that I'm getting, I'm noticing that it's not really something that I could have planned for, that the things that I've developed or the way that we're experiencing or engaging, it's, you know. I'm excited for the things that I don't know that start to unravel, which could be kind of scary or not. I've had these really interesting sensations, too, of depersonalization, where if I'm sculpting something in medium for too long, my hands feel like they're detached from me. In the real world, I'm on my phone, and for a good 30 seconds, I'm like, you know, doing their own thing. And so, you know, so kind of, I'm really excited for people to start questioning more just the flesh that we're in. And the more that you question that, the way that we kind of like engage with our bodies and our minds and our consciousness is gonna absolutely like break open like a huge wall where there's gonna be so much crazy exploration. So that's what I'm really excited about.
[00:42:35.017] Kent Bye: Awesome. Great. Well, thank you so much for joining me today. So thank you.
[00:42:37.700] Chelley Sherman: Yeah, thank you so much. It was good to talk to you.
[00:42:41.044] Kent Bye: So that was Shelly Sherman. She's an XR artist who's been working within virtual reality for about four years now. So I have a number of different takeaways about this interview is that first of all, Well, just the fact that Shelly had scoliosis, she had a lot of different issues with body image and some body dysmorphic disorder type of symptoms. And, you know, there's something about being able to see her body within this specialized capture that really clicks something in her mind. And she kind of attributes it to this like phantom limb therapy or mirror box, where she's able to actually reevaluate her own sense of self by actually looking at herself in this volumetric capture using the depth kit technologies. So that was quite an introduction into VR. And it sounds like there's been this through line of really looking at this intersection between the mind and the body. So studying a lot of cognitive neuroscience and looking at different studies and something, a lot of issues around trauma and seeing how you could do these different techniques of the EMDR or brain entrainment and. different aspects where you're trying to really connect to your body in different ways. And she was saying that a lot of these trauma therapy centers are having people that are looking into these different Eastern traditions of yoga and these different somatic therapies, and just sees that there's something unique that she's experienced within her own life, to be able to then try to isolate what those components are, and to be able to replicate that within her art. So it just happens to be this very fascinating fusion of all these different aspects of chaos theory and unpredictability, but also these differential growth patterns and coral reefs and diffusion reaction patterns and looking at these self-organizing patterns and flocking patterns and swarming patterns and dynamical cognitive systems and different ways of studying chaos in the brain. And so just looking at all this different cognitive science research and seeing the different principles that she could take to then put those into these different forms that she's creating and then see how that modulates her brain. So really at the cutting edge of a lot of technology and mathematics and the stuff that she was showing there at the UN Women Global Film Festival was really, really quite cool. but she's really looking at this vibrotactile ways of using like the subpac to be able to give you this additional type of haptics and really wanting to have a lot more sophisticated technologies to be able to get to the point where it does actually make you feel like you're in these different environments and She talks about this experience of like the heavy base where you can actually feel the vibrations on your body and technology isn't necessarily at that point to be able to recreate that degree of fidelity. But if we were able to do that, then you start to have all sorts of ways of using these virtual immersive experiences to what she says is to reprogram your body and to repattern it in specific ways. We dove into the geekery and hacker culture of algo raves, where it sounds like there's hackers that are broadcasting with a projector up on a screen and they're live coding and dynamically creating the music on the fly. And just how this is a whole new genre of music that is coming out of this hacker culture and maker culture, but also people who have the appreciation to be able to actually read the code and be able to predict what was going to happen and then to see what actually happens. a little bit of this performative, messy, not super overproduced, but really super raw and experimental and a whole vibe that she's also a part of the scene and taking a lot of her work and starting to show it into more like video DJing and all these other audiovisual types of performances. And so there's this cross section between the work that she does and then being able to kind of recontextualize these different things into these broader music environments. She's also been experimenting with things like ultra haptics, which does these kind of like audio sound waves to be able to create different haptic experiences. It's kind of like a, an array of things that are pushing air. And it gives you like this more subtle etheric type of haptic feedback, but you can start to put these ultra haptic arrays up and start to give people like haptic feedback on different parts of their body, which. She's translating some of the stuff that she was doing within her art experiences and trying to give a little bit more of this sculptural haptic immersive experiences with them. So just really in general, just trying to look at our sensations and trying to see how consciousness can readapt to push the limits of our perception and doing lots of experimentation with sonification and 3D spatial audio. There's a whole interesting conversation there at the end about critique and the need as an artist to want to get some honest critique and feedback, because in order for her to be a successful artist, then she needs to get some sort of honest feedback. But yet, once you get up to a certain status, then is that going to have people not want to give you that authentic, honest feedback for a variety of number of different reasons? But Shelly was just talking about her desire to be able to collaborate with different people and also just do projects that just don't have her name on it so that she can So start to put work out there and maybe get some more honest and candid feedback. And finally, some of the issues of dissociation and feeling isolated. I think she's wanting to do more multiplayer, different types of interactions, but, you know, doing some things with like phone based AR, but there's still this isolated component to that doesn't feel like it's fully embodied and somewhat dissociative as well. Some of her experiments that she's been doing doesn't feel exactly all that satisfying yet, but kind of having this dream of having more of these multiplayer metaverse types of experiences, but then being able to actually have these group experiences. We had talked a little bit about these dome experiences where you're going to have these huge collective experiences, but that's going to be way different than what you can do when you're completely immersed within a virtual environment where you're able to control every different aspect of what you're experiencing, especially when you start to add in different layers of haptics. And she did talk about this imaging heap concert that she worked on a little bit. And that was by the wave and actually had a chance to see that in Mike Jones's workshop where he had this whole vibrating floor. And I was able to get a little bit more of like this feeling like I was in this dance club, like having that visceral haptic feedback of the floor shaking, but it wasn't like I was feeling the compression of the subwoofers and the moving the hairs on my arms to the degree and level that, you know, that's the thing that she hopes that technology will eventually get to something like that. So that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, post on social media, leave a review on iTunes, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon your donations in order to continue to bring you this coverage. And just $5 a month makes a huge difference. And actually, if you want to give more, please give more. The more people that give at different levels, it just helps me continue to bring you this type of coverage. So you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
