Jakob Kudsk Steensen is a Danish artist who has been creating virtual landscapes for 15 years, and has recently been creating virtual reality environments. Steensen comes from the fine arts world, and he’s been working as a VR artist with different galleries creating installation pieces that he licenses out. He was showing his latest installation piece at SXSW 2019 called Re-Animated, which was inspired by a video on YouTube of the last mating call of the Kauaʻi ʻōʻō bird that since has gone extinct due. Steensen said that this bird call and the emotion around it was haunting him until he eventually decided to re-create the environment of the bird in VR, and then he 3D-scanned bones of the bird so that he could re-animate the bird in a VR experience dedicated to Kauaʻi ʻōʻō and reflecting upon humanity’s impact on the world around us.
I had a chance to catch up with Steensen at SXSW in March 2019 to talk about his journey into VR, his process of world building, why he created a broader context for an installation piece with video and looping sounds of the last call from the Kauaʻi ʻōʻō bird, strategies for financially surviving as an independent VR artist, how he’s focusing on creating a sense of materiality and space in virtual environments, and why he strives to create calm, slow, and poetic experiences in VR.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
Here’s the last song of the Kauaʻi ʻōʻō bird that inspired Jakob to created Re-Animated:
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. So, continuing on on my series of talking to VR artists about their process, today I talked to Jakob Gustinsson. He's an artist who had a piece at South by Southwest called Reanimated. so reanimated is about this kawaii 00 bird that went extinct and jacob was watching youtube and he found this video of the very last call of the last surviving male of a species making a mating call to which was not going to be received by anybody and it was just the last call and jacob finds this video on youtube and watches it and sees this whole community around it and then he almost gets like haunted by this bird call and he has to make this piece of art. So before I dive in and get more context, I'm going to actually let you listen to the kawaii oo bird before we dive into this episode. you So that was the call of the kuaioo bird that got inside of Jacob's being and he couldn't let it go. And so then he created this whole VR experience that premiered at South by Southwest called reanimated, where he's coming up from a much more of a fine arts perspective. And so really focusing on the materiality of the experience, also having this big installation and a video screen, and you really get a multimodal story where he's trying to tell different parts of the story in many different media. So. I had a chance to sit down with Jacob to talk a bit about his process and intentions for creating this piece called Reanimated. And Jacob also has a lot of insights in terms of what it takes to become a sustainable virtual reality artist, going the fine art route and working with different gallery spaces. And it's a little bit different of an approach and business model. And he also talks a bit about that as well. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Jacob happened on Tuesday, March 12th, 2019 at South by Southwest in Austin, Texas. So with that, let's go ahead and dive right in.
[00:02:39.023] Jakob Kudsk Steensen: So my name is Jacob, and I'm a Danish artist based in New York. And I use VR to make virtual landscapes. And I've been doing this for about four years. But I've been working with virtual environments for the past 15 years. So I started as an artist modifying video games using actually the same software I use now. It's like Unreal Engine. So I used the same thing modifying video games back then. And then I went to art school. And then it just became a very native, intuitive medium for me. So here at South By I'm showing a piece called Reanimated and this piece is about an extinct bird. So I have audio recordings of the last bird that was ever seen and its mating call and it was also the last species that ever lived. So this piece takes this mating call and 3D models of the scans of the feathers of the bird and the habitat it used to live in and then it remixes all this into this new very organic reality that's connected to algorithmic music by a composer called Michael Riesman. He's the musical director with Philip Glass and there's also a small microphone on the headset so you're breathing in very subtle ways influence all the kind of humidity, the fog and the light inside the virtual landscape. So you're kind of becoming part of this work and in a very meditative, slow way. So I try to go backwards in terms of like approaching VR interactive and make it kind of slow and poetic. and go against the paradigm. So I've done many different projects before like multiplayer, interactive, all these things and I felt kind of a need to do something slow and thoughtful and kind of pause a little bit and spend a long time making like one work. So I've been working with this for like a year and a half in a community museum in Denmark and now I have it at South by which is really exciting.
[00:04:27.183] Kent Bye: Great. So maybe you could tell me the story of how you discovered this extinct bird that had the last mating call, which when you think of it, it's really quite sad to think about that. That's the last observed bird of that existence. And it's doing the mating call, searching for its mate that doesn't exist. So maybe you could tell the story of how you discovered this.
[00:04:46.731] Jakob Kudsk Steensen: Yeah, it's a very kind of emotional feeling you get listening to that and the way I found it was like four years ago about I was just on my laptop after a long workday and sitting alone and kind of just getting lost online you know like browsers and everything and YouTube clips and stuff leads to the next and then all of a sudden I find this call on YouTube and on YouTube it has half a million views and 2,000 people have written these like emotional commentaries on it almost like a burial and So it's just completely like, I don't know, like there's something weird in my brain, like sitting on my laptop, being hypnotized by this call, reading these other people, and then look at the views, like half a million. Then I'm all of a sudden imagining like half a million humans sitting alone and getting lost in browsers, listening to an extinct bird and being kind of evoked some emotion from it, having some emotion evoked from it. And that as a phenomena just obsessed me. That that is a reality we live in. That you can listen to recordings of animals that do not exist online and comment on it and be part of this kind of network. It's just something with that kind of primal history in the past, climate change, everything that's kind of about to be lost. And then these new virtual futures and how we live in them. So this work and usually all my works just comes from these random things and ideas that I just kind of encounter and then make it a core for a work. And then I just start thinking about how can I make this idea relatable to people and this feeling. And then I just start making it.
[00:06:29.783] Kent Bye: So yeah, maybe you could tell the story of how you went from hearing that call for the first time on a YouTube clip and then decide to make this virtual reality experience that's trying to reanimate this extinct bird.
[00:06:40.673] Jakob Kudsk Steensen: Yeah, so I heard this like four years ago and I probably started working on this two years ago, but the audio first appeared in another artwork I did to Rattiganimism three years ago. There's just like the call was this little element in the work, but it's just been kind of haunting me. It's like, it's this idea that just sticks. And it wouldn't leave. It wouldn't leave my mind. So I did other work in between. And then at some point, I was just, OK, I have to do this work, or this call is going to haunt me. Creatively, as an artist, it's like, usually I just have a few ideas in a notebook, and they just kind of slowly evolve. And then at some point, I feel like, OK, now it's about time to talk to museums, to apply for grants, to start thinking about collaborators and everything, and get this project out. So that happened like two years ago. And then I start this process of all my work premiere in art museums first. So I'm newer to the kind of film festival circuit for immersive media, but I find it extremely rewarding showing that context because you have genuine conversations with people approaching something like VR from many different perspectives. Whereas when a piece like this shows in an art museum first, it's more like I'm an artist in the art world using VR and immersive media. And they're not that many. There's like a few artists doing that. But you're not in a context where that is like the main interest. But at a place like South By and a festival, that is the main interest. Everybody's showing here are doing something and interested in exploring how, what can you do with Immersive Media? How can you, you know, fund it? How can it get out to people? So that's kind of how I work. And then it takes a long time for my idea to become into a production. It's like, then there's a year of talking to museums, contacting natural history foundations and grants and all these things because I'm not like a commercial studio so I don't, there's no one paying for it, I have to fundraise it to enable this kind of work and then an institution helps with that so I like that format because then it means that I can produce something like this and show it in more like a commercial affair setting and introduce an artwork Because as an artist, if I first probably just have like a big studio externally or a sponsor, I wouldn't be able to work the way I do, which is very chaotic and like erratic. It's like, oh, I sat on YouTube and I hear this. I want to experiment with this material. And then I just start doing that. It's like a lot of experimentation and practice to it that can't be pre-designed. So an artwork like this, I don't know the final form until it kind of just arrives at it through practice and experimentation.
[00:09:22.127] Kent Bye: Well, you have also a big LCD screen for people who are waiting to see the experience. And you have this quite epic journey through this very immaculately depicted island in Hawaii. And so maybe you could talk a bit about that experience and how that relates to the VR experience and how they're connected to each other.
[00:09:41.233] Jakob Kudsk Steensen: Yeah, so with this work I thought a lot about how the story and kind of the facts can be facilitated in a way that doesn't take away the more raw, intuitive and like physical element of VR. So like that's something I thought a lot about with this project. So I wanted to take the story of this bird, the Kauai 00 bird on the island of Kauai near Hawaii I wanted to take the whole story of why it became extinct and have that outside the VR work. So the VR fully only makes sense if you actually have both elements. So the way I normally show work is that as a VR work was like the ultimate depth and immersion in a physical space. And then there are things around you that also tell stories. parallel to that and introduces things that you can maybe have in mind or if you feel like there's something you don't understand you can take your time after VR and watch the video or you can see it before and then enter the headset. So that's a format that I'm just starting to arrive at that what really works for me right now as an artist is thinking of not VR as like an isolated thing but I think of it as an installation and in this installation you have the possibility of diving into this island that you see in the video And then the video tells the history of the bird. So it goes from the first person to document how in the early 1800s, mosquitoes came to the island on horses. And those horses, someone then found out in the 90s, like 1990s, almost 200 years later. It's the reason why this bird became extinct, because they got avian malaria. And then there was a hurricane. And this hurricane, normally birds on this Hawaiian plateau fly into valleys to protect themselves from hurricanes. But they couldn't do it because they knew there were mosquitoes. So not just this species, but many species became extinct in one snap because of something indirect caused by humans 200 years earlier. So I became interested in how we're kind of living in parallel times almost. Like things can happen today, that you can't analyze, you cannot predict it and it can be like a 180 change of turn of something in the environment or in life because of something that happened like hundreds of years ago because the causalities you can't really analyze always so in this video screen you go through perspectives from people in 200 years who have explored this bird and island and they journey through that history in the narrative and you get all these facts but in a sort of also more poetic way so it's not just like a documentary language i work with a guy michael silverblatt he's like actor and video game narrator so I work with him to kind of create this meditative voice that takes you through time and back to the island and through these histories and then you arrive in the present with a picture of the bird and then you walk into the VR so you kind of get all that pre-experience or after or you can go in the middle of it. It's sort of made in a way that it's looping so you can enter it at any point in time. So these are things I think a lot about like how can this become a space people occupy also outside the headset and you feel connected. So it's not just like in or out through this little device and then you exit and then okay and then you're on to the next thing. I try to create these spaces and designs that facilitates that.
[00:13:14.830] Kent Bye: Yeah, there's a like a little installation here of like a hexagon that has a whole wooden platform and I had a chance to see it during the press hours yesterday and because I was trying to see as much things as I could as quickly as I could I didn't watch the video first and so I just had a chance to watch it right before I'm sitting down to talk with you right now but also I found that the video was a little hard to hear just from this environment and without headphones it's actually hard to determine all the things that are actually being said but It seems like that if you were showing this in a museum environment where people are waiting to see it, then you have something that they can watch and pay attention to to give them a little bit more context. And I see that there's a bit of like different philosophies for how do you set the context and do the world building before you enter in the world so you can get the most out of it. And you know, sometimes I actually prefer to not have any of that and just to have it cold, just to evaluate it on how much is emergent from the VR experience itself. I know at Sundance and here at South by Southwest and Tribeca, there's a bit of creating an installation. So there's already a context of being built. But also, you're deciding to use the video to help build that context and have those different mediums work together, I guess.
[00:14:23.028] Jakob Kudsk Steensen: You know, I come from the art world, so I make experience that people would go to these museums, and normally when I install there are hundreds of people in a space, so it's like a big landscape, and then people enter it, and then they go with the mentality of spending like an hour, hour and a half. People are not going to do that in a festival. So it's sort of like, how do I make something that kind of gives an idea to the full narrative of the work, but don't give it away, and that has that element. Because nothing changes, I don't want anything to change in the VR world itself, so that remains the same. It's the same here as it is in a museum, except I optimize it for like... kind of small laptop computers. It's like a backpack, but it's basically like a laptop. So it was a lot of that that had to be changed. But it's more like, how do you make a design that's inviting and gives, if people wanted to, they could stay for a whole video and watch it here. It's kind of like, yeah, like you say, like noise and things, like it's kind of hard to hear. So there's a lot of elements that has to be kind of sacrificed in a bit. Like we also brought earphones so people could use earphones and sit there, but then you don't hear the call in this space. And then that's also really nice that you, this work emanates throughout the whole space. It's this song that happens every 10 minutes. You hear the bird song through the narrative. So there's like a bunch of things that to me it's just important to get the emotion from people and feeling the work. That's my main motivation as an artist. That's the natural history. But to me that's just a way to get thinking about how can I use VR to make people feel new ways of relating to nature and like sense that. That's the really interesting thing to me. It's just I don't get the ideas for how to do that if I don't have a specific story to tell. But the story is almost just like a It's like an entry point for a context. So in the video, for example, we also then made just this little intro text. It's on the screen for a minute. It's like a little summary of the piece. And people can read that if they stay, or they can watch some of the video and kind of get an idea of it. They can ask the staff that's like around. So I think of it holistically as a space people are around and how they experience that. That's kind of my approach, at least. And that comes from coming from art world and physical spaces because otherwise I would probably make it work. Like I don't see why I would make a VR work that's only fully in headsets and then tie it to installations. That's sort of very limiting to me. Then I would explore a whole other avenue. Then I would talk to publications. I would like try to get on Steam or try to find a way of getting art out in that space and i would design it differently with that in mind but this is not made with that in mind so it's sort of yeah that these different languages from different contexts where you come from and how you approach it and i come from the art world and so that's my approach but i would really love to soon probably explore like fully online available work. Maybe it's on the phone or something, or maybe it's VR, I don't know. But I do also become more and more curious to that format of making, of like, you know, like you want to publish it, you want to get out to people in homes and devices. But this piece never started that way. So it's like, it's kind of hard to figure out, but I love that people are experimenting with it and approaching in many different ways. It makes it kind of fun to see how people do it.
[00:17:46.133] Kent Bye: Yeah, well, the thing that was really striking to me was to see the level of detail that you had in the island that was shown in the video. And in the VR experience, there's certainly detail, but because it's real time and you have to hit a certain frame rate, it's not quite as well high resolution as you're able to do in a fully rendered out video. Maybe you could talk a bit about your process of trying to depict with that level of detail an island like that that you created this video.
[00:18:10.247] Jakob Kudsk Steensen: Yeah, totally. Also when I show this in museums, I have specially built big overclocked computers. So they have twice the details and everything. So this is like scaled down to these mobile computers and little setup. But what I do is basically that for this piece, for example, I went to Natural History Archives and then I photographed the feathers of the bird. I like 3D scan bird like guts, beaks. rocks, leaves, moss and all these things and I do that all the time. I spend like a fourth of my time all the time or a third out in landscapes yearly and I collect stuff. So I create like libraries of plants and things that I can remix together and play with.
[00:18:48.305] Kent Bye: So to me... So you're like scanning them with 3D scanners or something?
[00:18:51.807] Jakob Kudsk Steensen: Well it's a lot of more like photogrammetry and then more and more it's more the textures that obsess me. So it's becoming more and more like I don't want to replicate a tree for example. It might just be some of the bark and then I want to mix that bark with another bark and make like impossible species that feels kind of real but also a little off. So to me it's becoming more and more about a creating a sense of materiality in a virtual space that you can kind of sense so like that's a lot of you know like kind of shader programming how does it feel liquid does it feel you know like this work is kind of feeling humid and so it's like how do you create a feeling of things in the air on a screen in front of you. How to create a sense of space in between you and the object in something that's essentially a screen. So that sensory thing is becoming and is in this work specific a lot more and more central to my interests like the texture, the plants, the leaves. So I photograph them and then I use different algorithms and software to create plants and kind of spawn. So like the island is made by Satellite data of the height maps of the island. It's like a technique used in actually much of the video games a lot. They would take like this height map. It's like a black and white photo of an island and then the software converts that into a 3D model. It's kind of relatively accurate to the actual landscape. And then I just, on top of that, start painting. Like that island is literally hand-painted by me with moss and grass and different things. And I like sit and populate it with it. And then on top of that, I have these trees, so I make like, okay, maybe three ferns. And then the software can turn them and do them different scales and within certain variables, randomize them. And then I just kind of brush it out on the island. So it gives this weird organic feel that looks kind of real because of these real textures, but if you pause it, it's clearly not. Some of the trees have feathers on them instead of leaves. So that's kind of my process. It's a lot of working with organic materials and algorithms. software and like mixing things together it feels kind of real but they're only possible in a virtual space so I'm also very interested in playing with this kind of fetish for the illusion like replication of things that in itself is not interesting to me especially in VR I'm sort of like thinking well okay here I have the option to create something very different and new so why do you just want to replicate what's in front of you So I'm kind of toying with that expectation a bit. Like in this work, you land in a forest, like, okay, so for example, you have the video, it's relatively realistic, and then in the VR, you land in a forest scene, it's like relatively realistic, and then slowly that just dissolves and becomes something... completely unreal, you could say, but it's all based on actual material. So is it like real or unreal? Because all the materials are actual plants, actual bird calls, an actual interview with the last guy to see this bird alive and his memory of it. All that is real. It's just mixed together in different ways. So I'm kind of interested in playing with that expectation or fetish for realism and fidelity and things, because to me, it's more of the feeling. So when you look at something in this piece, like a wet rock, it doesn't look like a wet rock, it feels like a wet rock. So I over-dramatize a bit the wetness on the rock. I don't do it like, oh, this is what actual wetness looks like. It's not. It's more like iconized, or it's like stylized in a bit, emphasized. If something's supposed to feel wet, dry, big, spiky, you know, like, whatever, I dramatize it a bit and play with the textures to give it more contrast, like, change it a bit. But in the moment, with that much stuff around you, I think people just, it just passes through your brain and you just kind of think, oh, this is kind of, like, sort of real, you know, like, relative realism, kind of think that, but if you look at it, it's not. It's, like, really weird and organic and, like, it's not real.
[00:22:48.732] Kent Bye: Well that was mimicking my direct experience of that which is I was really trying to observe that ecosystem and notice the disturbances in that ecosystem and that concept came from a conversation that I had with a man that was at a men's retreat and he was telling me about the indigenous practice of tracking of how processes that you have to have a mental map of an ecosystem and you're looking for disturbances of changes and it's those disturbances that are Intuitively leading you towards the animal tracks and then you are able to then track the animals in that way And so it's like that deep connection with that environment. So as I was watching your piece of reanimated I was noticing. Okay, this feels like I'm having this Trip through nature. I'm gonna just notice and try to find disturbances and then I look up and then there's like something weird and then there's something that's kind of floating around, or surreal, or out of place. And I'm like, it was striking to me. And so maybe you could talk about that in terms of the consonance and dissonance cycles of building narrative tension by playing with people's expectations and then perverting it into doing things that are unexpected.
[00:23:53.282] Jakob Kudsk Steensen: Yeah, that's a great way of describing it because this is kind of what I love about VR. It's like, you know, it's like I grew up modifying video games. And what I like about those is the ability to create a pace through a space where people have the illusion of moving themselves and looking themselves. But it's actually tightly choreographed. So like a light source moves through me, reanimated in a certain way and creates like, it's weird. Just most people follow relative sort of movements through this space, through an opening in a cave and the light coming in. What do most people do? Most people look towards that. What is that? And something moves from ahead and it comes over you and out the cave and most people follow that but then I would say also like with this piece I intentionally kind of made a very calm start and you land on this forest scene and then it becomes a cave it's kind of like one directional in the beginning but then once this world starts to become a new world that's so much happening that it's impossible to see everything or experience everything in one go it's impossible because you have eyes in the back and all around you it's impossible so I wanted to create a feeling of curiosity that the interaction is you trying to explore what's happening around you not like in a sort of analytical way that's a table and man this happens but more like at the feeling of a leaf sound behind you and maybe you look there and as you do that there's something all of a sudden you're right so your sensory system has to kind of move around if you're really in the work and that's also something I do because this also shows in museums where I just install the computers and very invitingly have headsets so in like the museum context they're even more like inviting for you to pick up and I want to make something with this work that feels very smooth and intuitive and going in and out of it so just like I didn't want controllers for example I was really trying to explore and it was a lot of experimentation a lot like the first four months was just experimenting with the form how do I want to do this And then, I don't know, I try to make something that has this thing of certain narrative and journey, almost like a 360 video, you could say, but then there's this algorithmic music that would change each time you do it. And there's also like the microphone. So everything will change within like 30% variable with your breathing and voice in there. So like, I was kind of interested in this. Also, we're thinking about natural history and how things progress and become extinct and everything to have, you know, like a one time trajectory. and relative to that you can kind of explore it and relatively you're part of it but not really so it's sort of like big climate transformations it's like yeah you as an individual have an influence of it you know are you throwing plastic outside in the park are you not and you kind of have that sense of choice that people want and especially also like interactive media like oh people want to control they want to be there but i only want to do that like relatively So it's sort of the landscape determining it, and you a little bit, but not really. So that push-pull, everything kind of has in this artwork, with algorithmic music, with the breathing. And then, for example, these insects flying around you. And the insects are just programmed to follow you around in a certain part of the experience. So depending on where you move, they will also move differently. So it's like relative. interactivity a little bit. It's more like my last piece had controllers and people teleport around and everything and people, you know, were into that but it was just really hard in a museum. It's really hard. If you have big audiences that go through hundreds a day and each one has to be introduced to these controls and how to click them and everything, it becomes quite a big limitation, a barrier to your work. So I tried to like scale it back. So my first experimentation with this was actually like full body IK leap motion and you become the bird and multiply a walk around four people. But what I just kind of noticed as I began building it was that it became a lot about that then. It became more, okay, how do you relate to each other in this space as creatures? But this work is, that's not the story of it. The story of it is this reanimation of this bird and memories and emotions around that. And I want people to pay attention to that. and not the more interactive with controllers and things. I also experimented with gaze controls. You could look at something and then fly towards it. That was a little better, but still the end result was a little similar. I think people would start focusing a lot on that and questioning, why am I teleporting there and there and there? And it would be a different relationship with the piece. So I try always, I don't think that's like one ultimate design for anything. It's just like, what's the concept and how do you hyperbole and emphasize that one thing? Like usually all my work are based around one thing, then everything revolves around. The install, the narratives, everything. So I try to create depth and variation within a very simple idea. So you have many points of view on it in different ways. So I think people have very different experiences with a piece like this. It's not like one thing that's crystal clear. It's more like mystical, maybe. And I like that people feel it's mystical. I want to bring back some mystical imagination to a quite rationalized technology and industry. So there are these different elements to it that I play with.
[00:29:09.064] Kent Bye: Yeah, I remember talking to somebody from the VR industry. I think it was Pete Moss at the Seattle VR conference back in like 2015. But what he said was that he's really interested in seeing what the artists do with the medium because the artists understand there's something about blazing new neural pathways into the brain. And I'm curious how you think about that and how you experience that, because you're going from the real to the surreal. And you're giving something that people have never experienced before. So you're having a new experience that had never been created before. So what's that like for you? And what do you hope that the audience can get out of that and why that's important?
[00:29:45.052] Jakob Kudsk Steensen: Well, to me, this is a very emotional piece and story. So, like, there's the story of the bird, but I also made it, and I made it with Michael Riesman, this composer with Philip Glass, and then, like, there's a whole personal story that I'm containing private, like, I kind of lost some things, and that's the real emotion that I'm working through. So there's also just the feeling of overcoming and accepting that things can vanish and change, and that something new will happen after. That's a whole other level. Even if people don't understand the burden, they don't watch the video or anything, that's totally cool with me. That's totally fine. As long as they have that more deeper philosophical emotional connection with it. And that's my real drive as an artist. So returning to the question of how I think about artists using this type of technology, I think those are the things that you can explore and use it for. You can use it for things that are not about technology. You can use it to create some new perspectives, experiences. I don't want to emphasize technology in the works I do. I almost want you to forget it's there. I think basically what I really think about is the difference between working, because I've done some studio productions in the past, some commercial work some years ago, And the really big difference is that in that world, you usually have a story, a design, a concept, a structure, you have a producer, and you find a way to build that out. But in the art world, people play with materials and the senses. And this is the language around it. So you can do a VR piece that has no story, no end, no beginning, and I've done those before too. It's kind of what got me interested to start with. And that is the expectation that people have in that context. So by working that way, it becomes much more about just trying things out and putting it together and letting people experience that. And that's a very different approach than knowing beforehand a relative final design that you have a team to realize that. So I never know the final format of a piece until the end. I never do. And people I collaborate with, I'm trying to build more and more of these kind of structures that enables people who are very interested and fluid in these technologies and do things on their own that are really interesting, but it doesn't fit within maybe their bread and butter commercial work. So Michael Riesman was building this algorithmic music with Philip Glass. They were about to show it at Carnegie Hall. They didn't get to show it there in that concert. So, and he's, you know, we became friends and everything, wanted to do something together and it fits this work. So we had an opportunity to expand on something that he was already passionate about. And like Andy Thomas, he did this bird call visualization. He did all the latest visuals for Bjork's tour. But for a few years, he's been on his own time, just like on a small budget, traveling around, recording birds, doing animations of them. So it's sort of like, why don't what you do and get paid for it become part of this landscape? And like, I make the bulk of like 90% of all is just me geeking away for a year. But then it's like, I'm starting. And I want to do this more and more. Bring in people who are doing interesting things, you know, like stuff they upload on forums or in their own time or on YouTube and things. All these cool, interesting experiments that people do when they're very fluid with the technology to an extent where it's not about that anymore. It's kind of forgotten. It's just like painting or drawing or talking or anything, singing. And then enable that to be part of these worlds. That's the direction I hope to achieve and also showing in like a commercial context or anything like, I don't know, maybe that's, you know, studios or I don't know, but like, that's why I'm showing here too. It's kind of exploring some new avenues for that. So I started a studio called Eradic Animus for that purpose to also explore these type of productions that not necessarily bound to my name as an artist in the art world, but that can enable new things. So that's also why I'm trying to show here to build new connections and explore new ways of making But with that paradigm, where we would never do a work that's just kind of like cold hire to realize high-fidelity environment or something. I would never do that. It would have to be something that enables people who are doing cool things on their own time and make that part of these worlds. It creates structure for it. Sustainable structure for it.
[00:34:00.888] Kent Bye: Yeah, and you just came from a South by Southwest panel discussion about VR art. What were some of the messages that we were trying to tell the larger virtual reality community about coming from an art context and how that impacts how you approach virtual reality?
[00:34:15.062] Jakob Kudsk Steensen: Well, yeah, to summarize that there was kind of some of these things with one it's exploring the technology from a very intuitive and imaginative approach where you're just like Jamming away, and you don't have an end goal for it. So like final story. There's no like pitch. There's nothing of that you just Explore it and this is how artists are working in art world You need to do that in able to arrive to that final exhibition and make something compelling and new you need to just experiment So that's like one and the other one is the really big one for me that connects that is how can you work that way? and realize larger projects because once you work on larger projects suddenly you're many people working together so you also can't just have a complete crazy non-structured production it wouldn't make any sense to people in the end so it's like what i was really emphasizing in that talk on creating art in vr was that like a way of bringing people together and doing these productions, because in VR, you know, like, you need a nice sound guy, you need a programmer, you need, like, people with many different backgrounds and skills, and still do genuine experimentation. Because there's sort of, like, kind of a conflict, a push-pull between the structure of the technology, the structure of financing it, the structure of, you know, showing it and all these things, producing it, and then the reality of how, as humans, you are exploring your senses and be imaginative in a new way without necessarily an end goal in mind from the start. So that's kind of what I emphasized in that talk and how, at least for me, that's my interest in showing in festivals in this kind of context. It's showing that you can actually do that and there are people doing that in the art world. And also in the art world, show that you can actually make kind of ambitious virtual productions doesn't have to be these like one-punch small things that's not able to basically compete with a lot of stuff that's in here just because it's like made by a famous artist or something I also want to show the art world that okay when I do a show well I need like production budgets I need to like invite people in I need um the language and way of collaborating that people are doing here. And I also want to bring that to the art world. And I also want to bring the kind of intuitive, imaginative to this context. So it's like I have a foot in each world, mainly in the art world, and I'm slowly stepping into the immersive cinema world.
[00:36:42.175] Kent Bye: Yeah, well, talking to Rene Penel of Kaleidoscope VR, he's been trying to think of new models to make the venture of creating these types of either storytelling or artistic pieces of virtual reality much more sustainable for the whole ecosystem. And so he had mentioned that you were actually taking some insights from the fine arts world to sell digital prints of your piece. So maybe you could expand a bit on what lessons you're taking from the fine arts world and how you're applying it to your career as an artist to how to become more sustainable.
[00:37:09.259] Jakob Kudsk Steensen: Yeah, I mean, it's hard, but I have become relatively sustainable within the past couple of years, but it's still kind of volatile. But basically, what you do in the art world is that you edition installations. So a collection, art museum can buy this, and there are four editions for sale. and they sell at high amounts and there's no way I could make that on online sales. But it's also like okay but then if let's say I wanted to make a purely online sales work then you have to go through the whole advertising industry, you have to compete with games, you need to be part of that whole thing. I'm already in the museum context where people buy and collect the rights to show an installation and build it out. So that's how it works in the art world and you have these limited editions that are available. So it's also like scarcity, but scarcity I think is kind of a shame when you work virtually. So that's like another problem. It's a lot about experimenting with formats. So like with this, I hope to take some of it, publish like as a 360 video, try that out, get some of that out. I'm exploring how to make a music app. So we have a vinyl coming out with the recordings of this algorithm playing the music together with birds. So Michael Riesman is producing that right now. We were hoping to bring it here, but it didn't finish the print. So it's sort of like create a mixed finance model because as an artist doing limited edition sales, it means that your entire income for you can depend on Two sales, so it's also like really high risk game. That's kind of nerve-racking to play You know and then there's the whole grant writing like if you do things that are generally relevant then you can also get art grants, films grants, experimental grants like that. It's extremely competitive, it's very hard to get. But I am thinking a lot about the audience too, so I get those for projects like this because I think it has the attention to the audience that a lot of people are very good at here. In immersive cinema, people are really thinking about the experience for people. In contemporary art, sometimes it's more about the artist wanting something, to do something. And there's kind of a disconnect sometimes about the immediacy. and access, that people feel welcome, that they walk into something and they feel, okay, I'm welcome here, I'm supposed to be here, what is this? And people really go to that here. So I've taken a lot of that language and applied it to my installations that people are doing here. And that's been quite successful in museums and institutions because it means that all of a sudden they have a lot of press, they have a big audience, people really like it, like many different people. I don't know if that answered your question, but these are things that go together with how is it financed. Because if you don't do that, it's probably hard to keep convincing institutions to lay down budgets and foundations if you're not able to generate an audience interest in the piece that's able to compete with other things people also see in VR. Like a lot of finance, context, VR production exists in its own vacuum. Because they never show next to other experimental things here. Here people just like freely apply, whatever, okay, you can show. You know, like, it's really crazy diversity and things in here. And that diversity you do not have in the art world, in these type of productions. But then people are way more radical in experimentation. Way more than I am, too, like, you know. So it's something that I'm really curious to explore and I've talked to Rene about it and it's awesome that he's looking at that because I think it's really important to focus on people who make it because if you don't do that, there's nothing. you know, and how does it become sustainable? It's hard, you know, it's really hard, but I think there are a lot of potential for it. And the next thing I'm going to do is to explore how to have elements of it that people can download. Maybe it's not the whole piece, or maybe it's like I build an installation without VR, but it's immersive in other ways, projections and interactivity. And then when a museum show ends, a version of that can go to festivals and online, and it will be fully aimed at distribution so people can get access to it. So you also enter new markets, not like the collector market. But the collector market can also support really weird things. So it's about mixing things up. That's what I'm trying to do. It's really hard, but that's what I'm trying to do.
[00:41:31.729] Kent Bye: Yeah. Maybe you could tell me a bit about what type of immersive experiences do you want to have?
[00:41:39.161] Jakob Kudsk Steensen: Yeah, I mean, one, I love environmental work. A lot of my favorite things I've actually seen at festivals are just like 360 video documentaries where someone is really passionate about something happening to nature and they just go and document it and you're in it. It's like purist. It's just like, then it's not about controllers and immersion a lot. It's just like, I went here, you can look at this place changing. That's like the ecology, lots of good experiences like that. Like the Marshmallow Lace guys has a really interesting profile too, like come from a studio background, but doing it kind of artistically, but it's not like all the way to the art world. It still has that audience welcoming design approach. So I love their work. But I think what I would like to see more though is things that are poetically personal and that does present you with things that's hard to pin down in a narrative. It's also something I go against is the idea and language around, oh, VR as storytelling, only because a story can be told in many ways. And like I said before, in the art world, you don't talk so much about storytelling. You talk about concepts, exploration, experimentation. So a work doesn't need a rational story that way. The story can be, hi, I'm playing with the materiality of extinct things. That's my story. And that's what it is. You can also look at it that way. That's how it's kind of looked in art world, more than a story about a specific bird. But here it's more about the bird, because it's like a film festival context, so people understand that more. I would really like to see things that are calm, slow, poetic, thoughtful, in these kind of personal ways. To me, that's kind of the next step where things are. And I would love to see gigantic multiplayer environment where people just freely roam around together. That is the moment where it would really blow my mind. And that's what I really hope to be able to do at some point as an artist. I do not have the infrastructure to build out $100,000 systems to do that in a museum. But when it becomes, hopefully, possible to do that, that's the part where I will just go full on VR or something. That and audio. Spatial audio is to me right now the most exciting thing. I've done shaders, environments for 10-15 years and I feel like that's been driving 3D for a long time, the visual, but the audio is just crazy sensory. It can really hit you and become part of you. So I would like to do a work in AR or something that has no screen. and you just walk around and listen to things and you can explore space through audio. Why does all this spatial tracking technology has to be tied to a screen? It doesn't have to be. And I just love seeing weird stuff, weird ideas. A lot of my inspiration comes from maybe attack a video game but that's like an awesome weird plant texture so I pay attention to that one thing or you know maybe it's a complete commercial production by some big studio but maybe I have a really cool idea about how to invite people in so there's a lot of like individual pieces and things and I just look for personality when you speak to people and have done something and you feel like this is not just like a way to get funding from XYZ tech company because they need to promote and then they need some works out in the world. I'm not so interested in that. I mean, it's great if they support work, you know, that's awesome. But it's like a driving factor this whole like pitch game. I'm not so I'm not into that at all. So. Yeah, it's a lot about building sustainable internal structures and slow growth. So something I, what I've been doing for many years as an artist, slow experimentation, do it well. I'd rather do something more simple and do that well than try to just like throw in a bunch of stuff because if you're throwing a bunch of stuff, then probably it's good. I don't think that's true. Yeah.
[00:45:24.976] Kent Bye: And, um, and finally, what do you think the ultimate potential of virtual reality is and what am I able to enable?
[00:45:33.518] Jakob Kudsk Steensen: I think it's like really opening up for new experiences and I don't want to say like opening up your mind for something because I think that idea also like the idea that VR brings forth a new perspective on things you know I can relate to that but I think what you're doing with that language is that it's the idea that you use VR you bring a new perspective you take off the headset and then you have that in you But I'm looking more to view it as a way of doing things that were not at all possible anywhere before. And like kind of expanding or connecting your psychology and your body to something else and more. So that's kind of how I think about it. So I'm like in senses of your mental space and senses. an expansion of it. So it's not like VR is just connecting you to a story or a thing. You take it off and then you're kind of connected to that somehow. It's more like you're in it, and when you're in it, you are connecting in real time to something. And here you're connecting to this remixed landscape. And that's kind of how I look at things and think about it when I produce things.
[00:46:40.401] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the Immersive community?
[00:46:44.528] Jakob Kudsk Steensen: I love the immersive community, so it's more like I love the energy, the creativity that's happening across people with many backgrounds. I love that, and it's sort of like, I come from the art world, so that's where I show, but all my inspiration is not from the art world. That's the context where I can build installations, and I love that, but all my inspiration is these places here, and I love it. Even if it's more commercial or something, I just love that people are doing all this stuff, you know, experimenting, generating experimentation and trying to make it happen. I think that's awesome. And I love that.
[00:47:16.945] Kent Bye: That's my favorite part of it. Awesome. Great. Well, thank you so much. Thank you. So that was Jacob Gudsteinsson. He's a Danish artist who's using virtual reality to make virtual landscapes and doing virtual art for the past 15 years. And he had a piece called Reanimated at South by Southwest 2019. So I have a number of different takeaways about this interview is that first of all, Well, I think it was interesting to hear Jacob's perspective of really identifying as an artist and a fine artist, because I really think he's approaching virtual reality more from that fine arts perspective than from a traditional cinematic storytelling perspective. You know, he's really focused on creating ambience and environments and textures and focusing on what he says is the materiality of an experience. And so there were certain aspects that I didn't necessarily know when I saw the experience. For example, I didn't know that my breath was impacting the fog and having any sort of interactive experience. I wasn't told that before I did the experience. And it's not something that I organically discovered along the way. So there's things like that where I don't know if it would have been better if I would have known that and tried to experiment or if the design would have made that more clear of saying, you know, your breath is actually going to impact this experience in any way. So sometimes when you do different levels of interactivity and. the experiencer goes through the whole experience and has no idea that they had any sort of agency or interactivity at all, then I think that's a design challenge for how to actually make people aware of that. Or is there an opportunity for people to be able to play with that, to be able to find their traces of agency? Because as I went through the experience, I found very little of these traces of my agency, especially when it came to the breath. Now, I think because he is focusing it on these larger contexts, like in a museum context, I think, you know, he's leveraging a lot of the storytelling burden onto other mediums and modalities. And so the whole video that you would watch before you went in to see the experience, I think told a lot more of the context and the story. And then when, if you were to only watch the VR experience, I think you may actually watch it and have no idea that it was about this bird that was extinct because it's a little bit less clear. I don't think that the VR experience on its own is self-contained. I think it actually really leverages what he's trying to create there with the bird call playing in a loop every 10 minutes and then this video that you'd watch to get a little more of a backstory in the context. So I think this is a theme that I see within a lot of the different experiences at South by Southwest and Tribeca and Sundance is this whole process of onboarding and setting up the context for what you need to know before you see the experience. Now, in the experience, what I do remember, and I think what did work well is that you're starting off in this environment and you're just looking around and you're noticing things. And then he's slowly kind of dissolving the integrity of the experience. And it starts to have a little bit more of these weird, surrealistic moments where. things will start to float around or change or shift and eventually at the end you kind of have this scene where all the bones of the bird comes together and animated in a way that you get to have this experience of the bones of the bird as I was watching it I kind of assumed that this is probably like the bones of the bird but I didn't know for sure but Knowing that, you know, he scanned in all these bones and was able to kind of move things around, it was kind of interesting to be able to see that. However, it was a little bit more of his own artistic creative interpretation. And it wasn't necessarily like a documentary style, like this is what the movements of the bird actually is. It was just playing with these 3D assets and munging them together in a new and different way. he spent a lot of time of creating this world and piecing it all together and you know he's really trying to experiment with these different things of trying to create a sense of materiality in the virtual space of trying to create a feeling of things in the air adding a sense of space really playing with the textures of the plants and leaves and playing with things like algorithm music and taking whole satellite pictures and converting that into the depth map and hand painting all of the different aspects and trying to give it this real organic feel and adding surrealistic components, like having trees with feathers. And he was really just playing with fetish and replication is what he said. And he wanted to play with whatever your expectations of this environment may be. So you start and everything is coherent and realistic in some sense, and then it starts to dissolve and become more and more surrealistic. So he's mixing the real with the surrealism and the unrealism and it's more about this feeling of this weird organic and not real. So as a piece of an experience, you know, my experience is that I did find that things were interesting, but I also find myself seeing that the pace was very slow and there wasn't a lot of change or dynamic things that were happening and that It was really asking me to slow down. And I think, you know, that was the thing that he was really looking for when he creates an experience is that he wanted to create experiences that were very calm and slow and poetic and thoughtful. So as I was watching the experience, I was really curious to see where it would go. But I also thought there was a lot of things that maybe could have been shorter or a little bit more tighter. And thinking about how much of the burden of this deeper context in the story being offloaded into the larger installation didn't really necessarily feel like it was a self-contained experience that would kind of stand on its own. It really relies upon this larger context in museum installation for it to really have those other elements. And then even with that, I think it's a little bit difficult to make sure that you are connecting all the dots between all of these different aspects. Uh, so maybe that's a part of what I got out of like being in this virtual environment and it disintegrating and sort of taking this surrealistic turn it's asking us to pay attention to the connection that we could have with reality and then feeling like a dissociation or a break and I think With that, there's a sort of a metaphoric dissociation that we have with the world around us. And I think that there is sort of a dissociative quality that also comes out of the experience, but I'm not sure if that was connected into the deeper message that he was trying to get across. Cause it sounds like he was exploring this really fetishized virtualized object and trying to create this sense of materiality and space within virtual environments. And so the, you know, the other big point, just the whole business model around how does one make a living as an artist and virtual reality? And Jacob is saying that this is a completely different model, which is that essentially your entire year's income could come down to a couple of sales of trying to get a project like this sold and put and installed into some sort of museum context. So it's a completely different model. And if he were to try to bundle this up and put it onto this team store, he said, you know, he'd be competing with all these other people. And, you know, he would also have to make sure that it would be a very self-contained experience. And I think he's really leveraging and leaning on the ability to be able to tell the story from many different modalities of an installation space and in a museum space that he would have to change a lot of things that were happening within the experience in order to make it feel like it was a completely self-contained experience. So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast, and if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon your donations in order to continue to bring you this coverage. So, you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
