#1073: “Liminal Lands” VR Fine Artist Jakob Kudsk Steensen’s Unique Approach to Environmental Storytelling

Liminal Lands is an environmental VR piece commissioned by Luma Arles to document and volumetrically capture different objects and organisms within “overlooked wetlands at the edge of the Mediterranean.” It’s a piece where the sound design adapts to your movement around the space, and there are also organic textures projected onto 3D objects that are also reactive to your movements. Artist Jakob Kudsk Steensen applies a fine arts aesthetic and production philosophy to his work, and the end result creates a sort of liminal experience that is hard to fully categorize — perhaps because part of Steensen’s intent is to blur those fixed categories into a more liminal space that spans both scale, but also space and time.


This is a listener-supported podcast through the Voices of VR Patreon.

Music: Fatality

Rough Transcript

[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So continuing on my coverage of some of the immersive experiences at South by Southwest 2022, today's interview is with Jakob Kunsteinsson. He's an independent artist who did a piece called Liminal Lands. So he's doing all sorts of different types of photogrammetry of natural objects within the wetlands and change the scale and create this environmental storytelling where as you move your body around there's different aspects of the sound design that changes but also you're exploring different aspects of the wetlands through these immersive experiences. It's a piece that's coming a little bit more from the fine arts background, but environmental design. It doesn't really necessarily fit into any established category for what type of experience this even is. It's a little bit of environmental storytelling. It's a little bit of a narrative. It's a little bit of interactive gaming aspects. The name Liminal Lands is all about this in-between space. This piece itself is in that in-between space, but he's also exploring the wetlands, which are also kind of an in-between liminal lands. And he's been collaborating with different art galleries to both fund these different types of projects. And he works a little bit slower over a longer period of time to be able to sustain this type of work that doesn't really have a lot of other established pathways to be able to do this type of immersive art within the XR industry. So he's kind of walking his own pathless path and making a good run of it. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Jacob happened on Tuesday, March 15th, 2022. So with that, let's go ahead and Dive right in.

[00:01:45.124] Jakob Kudsk Steensen: I am Jacob. I am an artist. I work for the past couple of years in landscapes that people might not normally think about. So I've been focusing on wetlands and digitizing sounds and different elements and working with field biologists and especially sound artists and also a few authors and performers and doing different projects in the actual landscape. So I go out there with them and just kind of guided by the eye of a biologist that might tell you like all the soil is alive or there are certain crystallization processes or like kind of rhythms of life that you don't think about. And based of those then I work myself to 3D scan different elements and with the other artists kind of respond to those stories. and then through VR put that together into some kind of holistic entity. So I've kind of let go of any master plan in my work by now. It's kind of improvisational in landscapes and I try to increasingly bring different perspectives on it and kind of morph that together.

[00:02:47.975] Kent Bye: Okay, I saw one of your pieces.

[00:02:49.596] Jakob Kudsk Steensen: I think it was at South by Southwest of 2019 So yeah, maybe you could just give a bit more context as to your background and your journey into this space Yeah, so I worked in video game development for a while like 10 years ago and I started doing a lot of VR around 2016 and the piece you're referring to is called reanimated and So that was a project I showed 2019 a lot that was based on a conversation I had with a biologist, Dr. Pratt, on his memories of seeing a bird that's now extinct. And then through him, because he's an emeritus professor, he gave me access to different archives. So I got access to recordings of this bird that you can, almost like a ghost, you can still hear it today. And I also gained access to natural history archives where I could digitize different feathers and specimens and kind of work with almost like a ghost in the archive. So mixing his memories, the bird recording, the feathers, and create this kind of experiential, almost like a biomass that people enter through VR. And I've done many of these kind of projects. So I've also done one on an island that's sinking, which is Bora Bora, at least the periphery of it. So I've also done a VR project with that in 2015. And on that island recently, I also hosted talks with different environmental thinkers and authors in VR and then live streamed the conversation onto Twitch so people could watch it. So I'm kind of looking more and more at VR as this collaborative living space where a project is like making a virtual landscape based on specific natural histories, often overlooked natural histories, and kind of inviting people to participate within those in different ways.

[00:04:35.002] Kent Bye: Interesting. So you said that you have a gaming background, but do you also have a fine arts background or do you have like an environmental background or what connects you to this topic in other ways as well?

[00:04:43.427] Jakob Kudsk Steensen: Yeah, I have quite a mixed background. So when I was 16, I started modifying video games. But at that point, it's like when I was choosing what to study, you couldn't really study, you know, like at least in Europe, there weren't these like XR programs. There was nothing about like art and digital art. It's like if you wanted to study, anything related to video games at that point, it was to be an animator, to be a level designer, it was very kind of industrial and structured, so I went into fine arts. But as I was studying fine arts, I was building these video game worlds, and I was making paintings of the worlds I built, because that was kind of like what I was told. But then as soon as I graduated, I completely ditched that and just started making my own worlds as I really wanted to. And that's pretty much where my career actually started. So I kind of learned to follow your individual passions in the field and just really stick to it, even though at first, if it seems like there's no space for it or something, because then it might actually be innovative. So it's kind of become a mantra of mine to embrace these hunches and intuitions in the way I work. Yeah, so it's this mixture of, you know, my passion when I was studying and working with video games, but I studied foreign arts and then I started freelancing on different productions and projects, especially as a level designer. So I would make worlds. So my passion the entire time have been Landscapes and using those to kind of tell stories, but where's the world building? So a lot of VR now I see is very like explicitly narrative But in these like 3d video games and like the late 90s early 2000 the story quote-unquote was the level So it's like something might happen that might be a narrative, but you tell it through navigation like you create a Ways to look or a room can kind of be claustrophobic and small then open up and it fits elements in the story So my passion entire time has more been the space around you rather than like a character that also fits with the environmental part of my work, so it's like I grew up, I mean my dad is an environmental, I mean he works with clean energy and he's part of like this project in Denmark where there's a big group of people that are trying to protect a wetland that's about to be built over. So kind of like political activism in an environment. So I have this mixed background of these things. I just felt like growing up with video games, but usually they're all about shooting and things. But what was interesting was the emotional impact you can have by experiencing this world, the colors, the music, the navigation and everything. And then pairing that with, you know, environmental perspectives. That's kind of the niche I found myself in. And by doing it in contemporary art, it also means it has often a more like abstract component or more like a philosophical dimension. So it's not like pure documentary, for example.

[00:07:23.480] Kent Bye: It's interesting that you're walking this pathless path of blending all these things together and taking your interest in that world building and environmental design but also like this ecological environmental observation and capture and working with experts in the field and biologists and collaborating with them and bringing those aspects of those disciplines in but also in this artistic context so working with different museums to be able to show these pieces and have it as a piece for people to be in the context of looking at art and having that artistic frame of the artistic gaze to be able to look at these things. So I think having that context in a museum context allows people to be a little bit more open-minded and have a little bit, I guess, less of a looser grasp, a need for a narrative and just be more observational. Just like when you go to an art museum, you're looking at art in a way of trying to notice things and how it's evoking emotions in a way that goes beyond someone trying to tell you something about what you should be thinking. So I could see all those things coming together and so maybe you could set the stage a little bit about this latest project of how this project came about. Did it start with going to this institution or did you find yourself in this space finding these locations or maybe just describe the piece that you're showing here at South By.

[00:08:33.725] Jakob Kudsk Steensen: Yeah, so the piece is called Liminal Lands and it's based on a wetland in southern France called the Camargue. And close to the Camargue is a French art foundation called Luma Arles. They're also a museum and they also support conservation. So they support the biggest wetland in Western Europe and the conservation of it as well. So it's a very holistic institution that works across design, conservation, art, and production. So they help produce new forms of culture. So they just reached out to me and asked if I wanted to do a project. But in order to do a project, I had to do a residency there. So it's like they only produce works on site with people. So this local and global connection is very important to them. And that really fit my background, working with these different environments, but in digital media, that's kind of viral medium as well. So the VR project here was made over 10 months on site. So I lived in the city and went out every week learning about how, especially algae, salt, and the sun and water influence all life forms in this landscape. Because it's a landscape that continuously changes between freshwater and saltwater. But it's also a type of landscape that people not think about very often. But wetlands are ecosystems. I mean, the definition of a wetland is a landscape that's able to sustain fresh water. So any large civilization in the world, a city, is built close to a wetland. It's just you can't have a large group of people without access to fresh water. So it's kind of this forgotten history of a very important landscape right beneath our feet. So over these 10 months, I went out into the landscape and documented natural formations between the scale of like 1 to 10 centimeter in size. So using this like photographic macro lens that can really zoom in, I was able to lay in the mud literally and like crawl around objects and get pictures from all angles. And then that becomes the 3D models in the piece. But by working over 10 months, it's like I see the landscape transform in many different ways because it's a really radical ecosystem. So a piece of wood might just look like a tree one day, but then a month later, it's completely coated in crystals. because the weather has changed and the dynamics of the environment. So it became a project where I just learned about these very basic foundations of life, salt, algae, water, really, and document those. And in the VR work, you go through six different kind of seasons and environments. And as you move around, the environment is reacting to your movement, especially sound. And all the sounds are connected to different 3D objects and their textures. So they kind of like vibrate and change as you move around. because I'm very interested in also like using VR in a very intuitive way. So rather than having kind of like a point and click, more like a rational paradigm we're used to with technologies, I try to make people forget and not really think about that they're connected to the interaction of the work, even though they are. So there's kind of this rhythmic experience where it's like you sink into the wetland, you become the wetland kind of. It is around you and it plays a lot with Sometimes making it feel like it's kind of consuming you and around you, and sometimes more open and kind of like you're a human in control in the environment. So it kind of pushes and pulls a lot on how you feel, if it's kind of claustrophobic or open or closed, if it's dead or alive. Just the kind of sensations and things that I learned working in the landscape, I try to communicate through the form of the virtual environment to the audience. And another thing is that when the piece first showed, it showed for six months in this museum in France, it's like in a museum context, there's also like a big spatial sound system, there's like a screen that shows other parts of the work, there's a floor made of actual salt and algae that was crafted on site. So when I show my VR works, like here, they're kind of taken out of that museum context. But I like that because I think it offers like a slightly different take on it. It's kind of narrative, but not really. It's always like a little bit of a misfit, I feel. But I like that. It's like, I think it brings us, I hope at least it brings us a slight dynamic in its form or something, showing a slightly different approach of making VR. What I really hope is to find a way to start publishing the works so they're fully available online to download. Because I used to work with these museums, that's my main platform. But I'd love to find a way now to bring them out directly to people as well, at home, in different formats. Yeah.

[00:13:07.439] Kent Bye: Nice. I'm curious to hear how you think of and define the term liminal, because it's called Liminal Lands. And so what do you think of when you use the term liminal?

[00:13:17.368] Jakob Kudsk Steensen: Yeah, so liminal spaces has been a key to most of my artworks. I try to like remove to rational a sense of space that you can easily define. And also like my story. So I try to make work that already exists between even like different timescales, like reanimate it. The piece you saw here in 2019 took, you know, like archival material and pictures of this landscape where the extinct bird used to live. Contrast them with memories of the present, using digitized fillers from 200 years ago and some of them from 50 years ago. So the actual materiality of the work exists across timescales, just literally how it's made. And with Liminal Lands, similarly, I really tried to give the feeling that a landscape is never static. It's always changing. It's always in the sense of continuity. and that's kind of a fact of life that these wetlands and ecosystems live by the reason we don't have a lot of wetlands today and a lot of like cities and countries are now struggling with their fresh water and they like a lot of programs to bring wetlands back. The theory that they're gone is that in the 1700s the idea of space became urban and it became rational in terms of like a forest you can walk in, or a forest that's a park, or a forest that you can convert into farmland. But a wetland you can't really walk in, you can't really enjoy, you can't really farm in it. So it's kind of this irrational space that doesn't really fit a modern logic. So all the kings of the 1700s and 1800s, they paved over and removed all the wetlands. So a lot of cities actually have roads named after rivers. And in some cities now they're starting to dig them back up. because it's becoming a huge issue that we don't have the wetlands. And so the way I look at it as an artist is like, oh, I can bring back a liminal perception of an ecosystem that doesn't have this complete rational identification. And I think VR is quite a powerful tool for that because you can just kind of like morph and make things change all the time. So that's my thinking on liminal.

[00:15:27.441] Kent Bye: Yeah. Yeah, I don't know if it was Rebecca Solnit who had a definition for liminal but there was someone that I read at some point was talking about how a liminal space is like a threshold space in architecture which is the space that's in between the space that you're going to and so like going through a train station or going through a hallway or a corridor or the place that's in between where you don't actually go to that place as a destination, it's more of like you're moving through the space to get to another space. And so a liminal space in VR would be kind of like the Matrix-like environments that you're waiting for the world to load into or the loading screen in VRChat. And it's kind of like this in-between space and like waiting for the plane to land at an airport is this like liminal time. So I could see how this in-betweenness that you're talking about where it doesn't fit into any nice categories, but that you're trying to evoke those spaces of being in between or taking people outside of their normal ways of thinking about things and put them in a space that within that liminality, they're able to kind of open up their mind in a new way.

[00:16:25.963] Jakob Kudsk Steensen: Yeah, I think that's very nicely said. I think also, like, what I learned working with Wetlands is that, like, the world is liminal and we try to construct it into these structures. But when you try to construct things too tight, things kind of fall apart. It's just not the logic of the universe. You know, it's like things aren't constant flux. They do evolve and change and transform all the time. And especially working with technology, like I was saying before, when I was, like, my background is, like, my interest was in video games when I grew up. But there was no way to study kind of a more, let's say, liminal approach to it. You could only study level design for this game, for this type, for this industry. It's all very rational. Like the liminal thing also is very important for just how I create things now. I really try to like flip it so that that's like, okay, that's the landscape, that's a certain theme. and I go just into that theme and place and then I start bring people in and then I build the world and then I bring people into that world and through that process at some point it will evolve and kind of have a more crystallized perspective of sorts but I don't know what it is before I start so I've completely abandoned the more conventional paradigm of 3D creation where it's like you write a story, you make a thing, you outsource it, people send stuff back I try to really now, intentionally, the way I structure creation processes and work with people, flip that. It's kind of difficult. And that's actually why I like working with museums and the art world, because that's a strong language and history of that approach. It's like, oh, you're not supposed to know what you're making before you start. And you can experiment with form ideas. And that's what brings you success, in a way, in the field or interest. So I like that language. But it's also difficult then to get the work out beyond the museum. So it's opportunity and trap too at the same time.

[00:18:14.222] Kent Bye: Yeah, really resonating with what you're saying both in my own process of creating my podcast which I feel like I'm in a like you said that the world is liminal and I feel like VR as a medium is in a liminal space and then as a Podcaster covering VR and the way that I do it is also in a very liminal space that it's something that's been between like I produce way too many podcasts and it's like I'm talking to people that a lot of these pieces that people don't see but That reflection that we live in a liminal world I think is also correct in the sense that from a philosophical perspective moving away from like substance metaphysics where you have these concrete static objects move to a more process relational metaphysics or ontology where everything's in dynamic flux and that if you try to boil the sense of reality down to its core fundamental parts, it's these processes that are unfolding and that are a constant dynamic flux of moving from one state to the next into these mere logically fractally nested context of this universe that's unfolding and then from down into the solar system into our world. But even our lives are an unfolding process. And so I feel like in some ways that there's a deeper truth of the liminality of the world, but we want to create those fixed concrete static ways of thinking about things. And VR as an industry has suffered from only looking at it through the lens of gaming. and not being open to some of the more liminal aspects and the whole infrastructure and the economic systems to be able to support those, I think you probably deal with, and all the artists here, of trying to explore the liminality of the space but also trying to be collapse-bound into like these curatorial decisions that are made at big major companies or economic models or concepts about what the medium is and what it can be with trying to sell it to a broader market while, yeah, it's sort of a chicken-and-egg problem of trying to like explore the lineality and then give the audience an experience of that but then create the market demand to be able to actually sustain it as a viable economic trajectory of evolving the entire industry.

[00:20:08.473] Jakob Kudsk Steensen: Yeah, I mean, economics is an interesting thing as well, where I think my work is only possible to be made because it's like I am, it's almost like performance to me. I've been doing it for so long. It's like improvisation and jamming. So by the nature of that, I can just instigate an idea and ask someone to work with me on something, or I can just go out and make it very organically, where if you don't have that capacity, very quickly it becomes, let's say financially, completely unfeasible to create a work like Liminal Lens. If you have a commercial studio making that, it's going to cost way too much than what a museum can do. So it's sort of like I've kind of found a space in a way to do that. But that's another thing I do, which is that I use very small teams over long periods of time with multiple iteration processes. So that also makes it more technically how to get funding and everything to make it. that makes it possible to create these projects. Because if you need a highly experienced professional team fast, the cost for that is this reproduction of the mainstream video game industry, of cinematics, of advertising and things, and these costs are just like... I mean, you know, a studio can cost several thousands just per day. to work with and it means that like if you do that as an artist like fast slow then you have to tell them like exactly what to make you can't make mistakes you have to really have to follow a rigid schematic because otherwise you go over budget. So it's kind of like I found this way of working like slower, longer, and people kind of pop in and out of a project to be rewarding for my ethos, but also just financially to be able to create. I mean, in the context of art, my work is quite like significant production. You know, in the world of commercial media, it's not. There's like these two worlds, right? So that's something I think a lot about too, of how to even make it happen and create a model for yourself that allows for a sense of experimentation. But I think it's a huge limitation that just the cost is so high that like as a creator, if you don't have that native knowledge, it will probably be impossible to create that kind of work. which I think culturally speaking is a huge shame in the innovation of the medium. Because as a result it means that it's like the same studios that already own softener are connected to existing industries that are reproducing a lot of ideas or ways to create things. I mean like visually, stylistically, yeah, I don't know, yeah.

[00:22:39.316] Kent Bye: Well, yeah, that's helpful context for both your process of how you got to this point and created this piece. But let's dive into the actual piece for a little bit here. So Liminal Lands, I have this image of you rolling around in the wetlands with the photogrammetry and taking photos. And like, is what I'm seeing in this piece, is it all to scale? Or are you taking small objects and blowing them up? Or did you capture these big environments? And so maybe just kind of fill me in if everything that I'm seeing in this piece is to scale.

[00:23:06.741] Jakob Kudsk Steensen: Yeah, they're not to scale, so it's... Like I saw a crystal might be an inch large, but then in Liminal Lens you encounter it as like a six meter large object. Or there's a scene where you sink into the ground, and it's actually like I went in December 2019 actually, and sunk my boot like into the ground, into this like black tar mud, and then came back and matched the same location, and in my boot print by then had grown these like green algae. So I scanned that boot print and enlarged it like, I don't even know how much, a thousand times. And that's part of the world where in this like black cavern and the whole background is actually the boot print. And then I use the color data of the algae to create these like pulsating effects. Oh, there are like little rocks covered by foam. It had this kind of crystallized texture on it where in the landscape, they were about like the size of my arm, but them liminal lines, they were like the size of a house. So that's kind of a play with scale and that's a progressive change for scale. So when the work begins, I'm going to put the headset on, you're in a one-to-one area. And then the next scene, let's say one to ten. And then the next scene, you're like one to a hundred. And then the next one is almost like a cosmic scale, where scale is completely vanishes. It's kind of like in the universe in a way. and then it progressively travels backwards again to a human scale. I think about also like liminal in the sense of like more like a ritual journey, you know, where it's like you start as yourself in a space, a ritual, a performance begins progressively, you enter like another realm. But you don't stay there, you're kind of like slowly weaved back into your own body on a human scale. So it also like goes from like human scale into almost like bacteria and something cosmic and then back out. But every single object and texture and sound is from the environment. There's nothing in the whole piece as the source material that is not from there. So it's also meant that like to make the artwork it's like I have to follow the rhythms of that landscape. I can't just buy an asset or something or outsource anything. It's like it must come from there. So it's kind of like a rule I made and it was also provoked by the lockdown. This was like hard lockdown in France. You couldn't go more than like a few kilometers away from your house. I was close to the landscape so I could go there all the time. But that strengthened this local foundation as well. And I thought, what's going to happen if you create an entire project over a year only from material right around you and nothing else? That's like a rule. You have to play with that. That also just made sense for me when I was making it. If I was making it now, I'd probably make it quite different.

[00:25:42.186] Kent Bye: Okay, yeah, I had this experience of, like, understanding the 1-to-1 scale, and then it ends in 1-to-1 scale, and then it gets into this, what I would characterize, at least from my experience, is a little bit more abstracted. Like, it didn't feel like it was 2-scale. That's why when I came out, I said, is this all real? And he said, yes, it is all real. And then I was like, what? What? This is, like, wild. But then I was like, wait, if you're rolling around the ground, you must be, like, capturing things that are small, so maybe it's a scale shift. So, yeah, I didn't pick up the scale was shifting in the experience. I wouldn't have been able to articulate that coming out of it.

[00:26:13.971] Jakob Kudsk Steensen: No, I think that's the choice I made. A lot of the work mainly was the rhythm, like the artistic work that goes into it is making this kind of continuous rhythm because you actually encounter a lot of material in it. There's actually a lot of 3D models and sounds and everything, but I wanted to make it feel, you know, fluid, kind of like if you go to an opera or something or like a long song or performance that's kind of like in a trance. You know, that's like a rational story, the actual models, you scale on all this. But in the experience, I don't really want you to think about that. So a lot of the work was really just fine tuning rhythms and scales and colors so that you go through a lot of transformations and actually different material. But I tried to make it feel like it was like a smooth movement, like one rhythm. So you're never too still. You are still sometimes, but never too still. It was just like balance to play with understanding space, not understanding space, using the textures. And another thing that I just learned in this landscape was like, I mean, it blew my mind that like these objects can exist. And I was not able to imagine that before I went there.

[00:27:21.176] Kent Bye: So these objects only exist in wetlands? Is that what you mean?

[00:27:23.798] Jakob Kudsk Steensen: Well, I mean, each object you encounter only exists in that particular wetland under particular environmental circumstances. So, like, one of them is, like, this is a completely white scene. It's just, like, little pieces of tree, but they're completely crystallized, but unlatch them, I mean, I don't even know, like, a thousand times, they become, like, the size of skyscrapers. And the sun is high, it's cold in the weather, but the sun is high, so it heats up the water. As a result, it evaporates and the salt then crawls up to the pieces of wood because they have more water in them and they coat them in crystal. And that can only happen under those specific circumstances in that environment. And it just kind of blew my mind that like I had never seen the world that way, you know?

[00:28:08.655] Kent Bye: So you captured that and then you have that in the experience then?

[00:28:12.108] Jakob Kudsk Steensen: Yeah, it's an experience. It's like this big white scene, but then there's another scene where there's more liquid and water and things appear a little less like crystal. And that's just because in that day it was more humid and less sun and there was a bit of rain. So the crystallization process wasn't as extreme as the other one. So what I'm saying is that it just kind of blew my mind that these formations exist in the world. And I was not able to imagine that before I went there. So I think this adds to the idea that it's like fantasy to people when they go through it. But if you're an expert on these formations, you can easily recognize each object because you've seen them before and you know they exist in the world. But most people don't know they exist. So I was also thinking a lot about like you have Elon Musk talking about like science fiction and life on other planets and everything but my approach is sort of like you have science fiction right here on earth and we need to take care of it and there are so many phenomenas on the planet and life forms that are like beyond anyone's imagination. There's so much life that we don't pay attention to or even able to imagine, you know, so it's kind of like encountering something unexpected for myself. And that also became my choice of what to include in the piece, you know, I included especially elements that I felt was like adding to this fantasy element for most, even though they're real, just by the nature of like how strange they were. So I mean, I did phonogrammetry of like more than 100 models. And I think at the final work, I have like 37 or something. So I also like discarded a lot of them. Or, in the beginning of the work, the models are more familiar, the 3D models. You can kind of recognize it in the end of the piece. But then, like, in the middle of it, these, like, crystallization processes are at their most extreme. And the most unique objects that I encountered are there. And that's probably when the work feels the most, like, removed from reality. But it's all real. It's just that we're not used to looking at things. Not many people really pay attention to the uniqueness of many natural formations. And my interest in that was really provoked by living in lockdown and working with digital art was often is like very referential to ideas and things in the web and everything. So it just kind of like enforced an interest in trying to remove myself from that and looking like beneath my feet, only using that. But it's quite an incredible experience. And when I created the work in the beginning, I worked with this biologist, Patrick Galar, and he spent 33 years of his life just working in that wetland. So, like, his eyes, you know, could see things that I couldn't. So, by going out with him multiple times, slowly he was personally opening up and he started showing me things beyond his professional point of view. Because when you go to a landscape with a biologist, they're going to be like, what do you want to know? Completely like, what do you want to see? And I'm like, I don't know. You tell me, but then you research, you have some initial ideas, you go out. And then like after a couple of weeks of going out into the landscape, he asked me like, how many pictures can your camera take? And I was like, oh, about 2000, because it's like a professional camera. And he's like, oh. And then in his pocket, he walks away a little bit and out takes his own camera. And he starts photographing stuff on his own. He's no longer just feeling he has to guide us to specific places, but he started looking at things that he's passionate about. And then I'm like, OK, this is what I'm going to follow. And then I kind of follow his pathway. He starts sharing his interesting formations. And then from there on, I focused a lot on going to try to find those things again on my own. And that's become like a guide for how I navigated the landscape making the piece. And now I really enjoy that way of working.

[00:31:46.096] Kent Bye: It's almost like this tracking of trying to find these things that may be very ephemeral or certain conditions and being able to see the signs but to notice what's happening in the world but then to be able to notice the differences of like what's new, what's different, what catches your eye and then tracking that. I want to talk a little bit about the interactivity because there's both as you move around there's a sound change but also like in some of the aspects like one particular moment near the end where I was moving close to the tree and it was almost like you were projecting a texture onto the tree in a way that felt like it was like a film recording of some black and white or something that felt organic within itself but also was being projected onto the shape. And so it felt like a projection map, but was dependent upon how my body was moving. So it was like very much my agency of my embodied movement was creating these projection maps of textures onto the environment to recontextualize that environment in some way. So I'd love to hear your process of doing that.

[00:32:41.082] Jakob Kudsk Steensen: Yeah, so every object in the whole world can do this. They have different sound layers and they can change the texture. And it's depending on the speed of your head movement and how close you are to the thing, mainly. So it's like kind of a camera-based system where you move your head, sound changes, texture changes. And that was also a way of getting away from more like hand controls, just because like, I'd love to have hands and all this stuff. But in a museum where a lot of people might never have done VR, people need to like easily go through the experience, just focused on something that became intuitive to people. So they have to move their whole body, you know, like their knees, their head, their back, everything is in interaction with the piece. And the kind of texture projection mapping you're referring to are like zoomed in images of textures of that specific tree, for example. focusing a lot on like little root canals, like little things that on an organic texture seems like a network or something. So they just like zoomed in of organic textures and then those like move around the object to create this effect of something that seems a little organic, but artificial, like in a smaller scale, but very like visceral in a way. So that's that effect. And then the sound also influences the color of those textures. So depending on how you move, that texture will also change. So that was like a tool that I kind of developed for the works. In my pieces more and more too, I try to think about a tool that can emphasize the key concept in the work. So this one is like the liminal space is that we used to think it of 3D. as like the epitome of realism. It's kind of an epitome of 3D. But you know, it's just like surfaces. It's just illusionary surfaces. So I was interested in contrasting that with sound that can have more volume and be more spatial to you so that like you can move up to an object and then slowly kind of like opens up. You can walk inside of it and you can listen to it or walk back out. So it's kind of like breaks illusion. But I was interested in doing it in a way where it felt also organic. So avoid this break of immersion while still being able to walk through things and just show that they're just surfaces. They're imprints in time as like a photography on that day where the sound keeps more like an all-encompassing volume. I think sound is really where I see the most technical innovation in VR or like spatial installations. I think sound is really, really, really interesting because you can track sound now in 3D. and its visual history in 3D has such a, I mean, the visual department has such a strong history, and the sound less so. So I'm really interested in sound, and I thought about using these objects as like perimeters. So usually a scene has like a certain architecture or something that might evoke a certain movement in certain directions. And as you do that, there are lots of sound moving around, but you know, they're invisible. So they're there and sensory to you, but you don't see them. but they're in space. So it's a way of inhabiting the invisible space between surfaces. So it's kind of like you're in a box, in a way, that provokes certain movement. And you think first, that's kind of what the focus is. It's on the objects, it's on the surfaces, but it's actually equally the sound. There's this invisible space around you. Because I think anything digital is, in essence, invisible to you. You cannot perceive the virtual. We've just invented certain devices that allow us to access it. Use these like a screen, and we can use our bodies, which is why I like VR. And sound has kind of become a new thing that I'm really passionate about, of utilizing.

[00:36:13.268] Kent Bye: Yeah there was a moment in the second scene where there was an object that's coming down from the top and then it starts to like collide with the space in a way that just goes through like you see in the virtual reality that doesn't have like colliders it just goes through the space and you did that and then later in a piece when it came out you said oh we were really hoping that you put your head through the wall or the tree And usually when I'm in a virtual reality experiences, I don't like it when things collide with each other because it starts to like break the immersion. I know I'm in VR, but once that stuff happens, it breaks the environmental presence in me in a way that it reinforces the fact that I'm in VR. And I especially do that when I'm in VR to deliberately not walk through walls because again, that usually breaks my immersion in a way. because this is a liminal piece in liminal lands and maybe you're playing with that in a way, I'm curious how either you've integrated that as a conceit for your piece and what happens when you walk through the wall and how you instruct people who may not be used to walking through objects in a museum context who've never used VR to discover some of these features.

[00:37:13.005] Jakob Kudsk Steensen: Yeah, so in a museum context, people have a much larger area per person. I mean like five times five meter area to walk in. So they have each person a larger area than the booth. And you know, it's up to facilitators as well, to like introduce people to the piece, say they can walk, stick their head into things, which in a museum context is kind of like how you do it. Because if I add text or narration or something that explains it, I think very quickly it's going to remove the power of this actual body sensation in the piece. I like to keep it very raw. Or like you asked me, why is there no credits in the end? But that's in a museum context. You also have an exhibition text and you have all the stuff on the wall and it allows you some freedom to just say this is what it is. It's a raw sensory experience.

[00:38:00.073] Kent Bye: That's also just to know when it's over, just to see, OK, it's done.

[00:38:03.876] Jakob Kudsk Steensen: Yeah, but exactly. But like in a museum, it's like an infinite loop of people walking in and out all the time. Yes. The wall, actually, I was playing Half-Life Alyx, which came out around when I was working on the piece. And I thought it's interesting, like they used occlusion in a good way, like you stick your head into something and things slowly becomes this kind of like strange orange color. So it added to this like alien sensibility of the game itself, that it is kind of artificial in its own form. You stick your head in it and it kind of becomes orange. So I was actually playing that game a lot when I was making the piece and it gave me a lot of ideas. Like you're sitting on the ground and it's this like dark scene where you're close to a kind of wall that's more encapsulates you. And then there are like really small sounds around you. And that's something that I did really well in Alyx, too, is using the sound, like the spatial sounds close to little objects that kind of like move around you. Yeah, I don't know what I'm talking about now. It's just because I have this background also in gaming and stuff, I try to learn a lot from, you know, it's like VR, the way I work, is an extension of the spatial dimensions of 3D. So that's my interest. And there are people who have been exploring that field for decades now. So you don't really have to reinvent the wheel in order to work spatially with VR. You can just look at what has been done in games already and the way they use senses and story in space. So I draw a lot of inspiration from different tools and ideas that I find really interesting, more like in big industry. But I take them out of guns and macho narratives of aliens and what not, like overcoming stuff as a hero and so on. I'm not interested in those narratives whatsoever, but I think the artistic tools and knowledge is phenomenal. I mean, it's amazing. It is a huge inspiration for me. You know, it's almost like operatic, spatial experiences, a lot of these games. I just don't like the guns and the dominating narratives there. There's a really interesting author, Linda Chang, in California, and she writes a lot about how a lot of the virtual worlds we build focus a lot on rational user interaction or a character moving through a world, but rarely does it actually bring diversity to the representation of ecosystems themselves. They're very generic in most virtual spaces. So that's kind of like, it's a paradigm I think a lot within, too, of bringing in a greater environmental sensibility and diversity to virtual space, so we don't forget that there's so much magic in the world already.

[00:40:31.909] Kent Bye: Awesome. And finally, what do you think the ultimate potential of virtual reality and environmental storytelling might be, and what it might be able to enable?

[00:40:41.614] Jakob Kudsk Steensen: Yeah, I think there are kind of two different genres I really like. One is the very documentary style approach. It's not kind of my approach, but I love it when someone is going somewhere, digitizing a place and just showing it kind of what it is and being able to share that with people. It's like spatial photography in a way. I think there's huge potential in that and also beyond VR and just like video games. I could imagine like environmental documentary video games start to come out where you like walk through a landscape and you hear these stories. For my own work, I'm mainly interested in the more like psychological aspect. So bringing back a sense of like ritual behavior for humans and technology. and being able to look at and share something that is a bit more ritualistic almost, like performative in a way. Like I think as humans, kind of my theory, any culture in the world are attracted to rituals to help them make sense of the world. But the way we do it is through like song, movement, colors, and kind of like animating the world around us. And then we build a kind of logic that makes sense to us. And we can make these powerful experiences, which can be a concert, it can be a little gathering of something, of people sharing some stories, or a dance or something. And that's kind of the language I think with and where I see a lot of potential in VR. For me, VR is not like this future where we all live in. in a virtual space, I think it's more like this short, powerful, psychological dimensions you can enter, almost like eating a pill or something. That's where I see a lot of interest.

[00:42:17.379] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the broader immersive community?

[00:42:21.804] Jakob Kudsk Steensen: No, I think what I really like about showing a place like this is that there's so many different approaches to the medium. And that's what I think is really healthy to keep and not like too easily let an existing industry or paradigm take over how something is supposed to be made. Yeah.

[00:42:44.038] Kent Bye: Awesome. Well, thank you so much for joining me today on the podcast and breaking down a lot of what you're doing there with liberal lands. And yeah, I really appreciated a lot of your perspective, not only how you're kind of walking the pathless path and blazing your own way of how to make it in the industry, but to push the edges of what the medium is and what it can do in a way that hopefully will start to bleed into other aspects of what 15, 20 years from now, we'll look back and see how to fuse all these things together. So yeah. Thanks again for joining me and helping to unpack it all.

[00:43:12.126] Jakob Kudsk Steensen: Yeah. Thank you, Kent, and keep up the amazing podcast and interviews.

[00:43:17.249] Kent Bye: So that was Jakob Gunnsteinsen. He's a VR artist who worked on a piece called Liminal Lands that was showing at South by Southwest in 2022. So the thing that really stuck with me in talking to Jakob is this concept of liminality, this kind of in-between space where he's focusing his work on liminal lands in the context of wetlands that don't necessarily fit within how we usually think of land or how land is used. It's a very crucial part of the ecosystem for providing freshwater But he sees not only the VR medium as a liminal medium that he's exploring these different aspects of liminality pushing the boundaries for what the medium can do and also that he says all of reality is liminal which I think is trying to have us hold a little bit more loosely the types of categories that we have and being able to make sense of the world and For me, I go back to Whitehead and the process philosophy where you're trying to see things in terms of these unfolding processes rather than these kind of static, fixed, concrete objects. And I think in that sense, we go back and forth between those mental models that has a little bit more of an object-oriented orientation because that's how we think about things, but we can have a little bit too strict of the types of categories that are there. So having this emphasis of the liminality of not only the medium of VR, but also the different types of experiences that he's creating here with Elemental Lens, Helps us get out of those existing categories of what we're expecting so I did experience quite a bit of awe and wonder as I was not quite sure as what I was seeing in these different experiences and he told me at the end that everything I saw was real but I didn't realize that the different things that I was saying were being blown up across different scales. You're taking a scalar journey into the different objects that he's finding within the wetlands and collaborating with different biologists and finding his own ways of seeing what strikes his attention as he's capturing these different objects, but then recontextualizing everything as you're in this immersive space. There's also an element as you're moving around, you're triggering different aspects, either sounds or these shader effects that are kind of projected, mapped onto the different objects that you're walking around. And so there's pretty interesting interactive aspects there. And also, as you see those textures that are being projected, then there'll be yet another amplified vision of this organic looking roots or imperceptible by your eye. But as you get up to these different objects, then you have these black and white textures that are projected onto them. Yeah, really an interesting project and always like to hear Jacob as an artist and larger discourses within the fine arts world and how to start to use VR in this artistic context. And yeah, just as collaboration to get these types of fellowships and working with these different museums to be able to not only spend time there making the different projects that are in the context of that geographic region, but also being able to distribute them as well. Anyway, really interesting guy, and always love to hear how he's thinking about things, especially the liminality across timescales as well. Like, that's a big part of the reanimated, of looking at this bird who had gone extinct, and to pull different artifacts and objects that are from many different timescales, and then also combining those together. So there's another aspects of liminality, which is not only in terms of the in-betweenness of the different objects he's creating, but across aspects of space and time as well. So, that's all I have for today, and I just wanted to thank you for listening to the Voices of VR podcast, and if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a less-than-supported podcast, and I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you could become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.

More from this show