I interviewed Break #11 director PK after Venice Immersive 2023. See more context in the rough transcript below.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. It's a podcast that looks at immersive storytelling, experiential design, and the future of spatial computing. You can support the podcast at patreon.com slash Voices of VR. So continuing on my series of looking at different experiences from Venice Immersive 2023, this is episode number 15 out of 35, and it's the second out of two of looking at the context of entertainment, hobbies, and sex. This is a piece called Break No. 11, We Move At Night. It's actually a VRChat event by a creator named Peter Krohn or better known within the VRChat community as PK. So PK has been doing a lot of these really amazing immersive music concerts for the last number of years and actually from the very beginnings of early days of all the different platforms of social VR. And he settled upon VRChat because they've been doing a lot of the most innovation when it comes to all the different features and social dynamics and just the most momentum when it comes to having these different shows. And so PK does a lot of this kind of kit bashing where you are taking different objects and he's got this independent music label that has gone through a number of different iterations. It's currently titled Break after these sets that he's been putting on within the context of VRChat. So These are anywhere from like 30 minutes to 45 minutes. This one happens to be around 45 minutes, but sometimes they get up to over an hour or two sometimes. And so he does a really amazing job of mixing and mashing all these different objects together in these different scenes, taking on these different spatial journeys, but also really playing with shaders in a way that I think is really quite provocative and interesting. all the different ways that these shaders can start to modulate your experiences within the context of these virtual worlds. It's almost like an augmented reality layer in the context of a virtual world because you're seeing the virtual objects, but then all the different ways that you can start to modify and modulate those at many different layers from the far field, the midfield, from the near field, even into your own avatar with these audio link enabled avatars to be able to have a whole other layer of the performance is the audience that's there and how their avatars are reacting to the music that's playing. So lots of really great innovation over the years from PK of pushing forward the boundaries of technology and starting to come up with these different grammars and language of the different immersive music types of experiences. A lot of these are actually recorded and you can go watch and there'll be a link in the description so you can go check out some of those existing recordings of sets. And I highly recommend going with some friends, especially if they have some audio link enabled avatars to be able to get a whole other aspect of this performance. He's going to also be showing a new set that's coming up here at Braindance and also a set that's already premiered as well. And so the overall contextual domain is looking at entertainment and music. The primary center of gravity of presence is very much into this environmental and embodied presence as you are in these scenes and seeing all the different ways that the environment's modulated. And it's cultivating this sense of emotional presence, both from the journey that you're taking on through the music, but also just all the different ways that the shaders are subtly influencing your mood and your vibe and how the music and the shaders and the VR technologies are really working in synchrony with each other. That's a big key for a lot of what he's doing is trying to tie these different things together to be able to use the power of the language of music that drives emotions, as well as the visuals to correspond with that and building and releasing of tension. That's really quite difficult to put language to. You just kind of have to go experience it for yourself. So that's what we're coming on today's episode of the Voices of Yara podcast. So this interview with PK happened on Friday, September 15th, 2023. So with that, let's go ahead and dive right in.
[00:03:46.586] PK: My name's Peter Krohn or PK for short, which most people know me as. I guess I'm kind of a director in VRChat. I'm not really a VR, I mean, I guess I'm a VR artist, but I don't model or texture myself. I collaborate with others. Sometimes I use art made by others like on Sketchfab, but I like creating, I have ideas and then I find people to help me make them. And most of it, at least now is focused inside of VRChat.
[00:04:14.657] Kent Bye: So maybe you could give a bit more context as to your background and your journey into VR.
[00:04:19.413] PK: Okay, so I was a professional tree planter here in Canada for many years. I did that every spring and summer, and that's like in Alberta, British Columbia, mostly. And then so I had the winters off. And so in the winters, I did art projects. Like I did a radio show. I started a label called Peppermill Records. Not sure where the name Peppermill came from. And so I'll just say something we did with Peppermill, just cause it kind of leads into the VR stuff. Towards the end of that first net label I did, I started something called the Peppermill Music Festival. And so basically what that was is, well, the idea was the first year I was going to try, we're going to, I was going to find like a, spot in the mountains, somewhere not in a park, because otherwise I didn't know if we could get permission, then invite everybody to hike up a mountain to this beautiful lake and just play music around a campfire for two nights. Well, one night I think at first, and then it was two nights. So I did that. It was really fun. So I invited some of my favorite musicians. Well, I invited a bunch, a few came, and a lot of them were Canadian, some were from the States. And so we hiked up, we played, it was all, nothing plugged in. a lot of folk music, some people who normally did electronic music and just played on guitar or whatever, whatever instruments they brought. And then I hired a helicopter to fly down all the garbage and stuff, leave no trace and all that. And then also, oh, and we flew up all the water and all the heavy stuff. So it was actually like a really pleasant hike, even though it was pretty far up there. And so that was really fun. The idea was so that my first label had a mix of electronic and acoustic, so that I wanted to do one festival up on a mountain. And then I thought the next year, let's go the other way and do more electronic stuff. So I want to do a virtual festival. And I looked all around, like Second Life, others, and there's some cool designs, but I think Wagner's site might've been one of the ones I used to try and find these. There's nothing that seemed like anything people would find cool. So then I started a new label called King Deluxe. That was more electronic. And we did a lot of music videos, animation. That's how we got into the visual stuff. But then the Oculus Kickstarter showed up and I got really excited. I thought, okay, this is finally how we do our virtual festival. I should mention the mountain festival, we ended up doing three summers at different locations. It was really fun. So it was just kind of like, I kind of gave up on the virtual idea for now, just because people enjoyed that so much, but it was always kind of scary because it could be fires. They wouldn't allow, if it was too dry, they wouldn't let us have fires or we could be rained out. And I wasn't making any money with it. So. Anyways, my idea was if you make the setting spectacular enough, people will come without any kind of fee, which they did, because I showed them pictures of these places. And that was my deal with the virtual as well. I wanted to make something that looked spectacular. Otherwise, I couldn't get people to make time for it. And there wasn't anything that existed. But the Oculus Kickstarter really, immediately, I thought this might be it. And so I waited until I got to the DK2 came out, and I bought it. And before that, I was already doing research. I was listening to Jeremy Bailenson's book. I was listening to Philip Rosedale's talks. I think Voices of Viaria I discovered towards the end of 2014, right around the time I got the headset. And so I was just trying to learn all I could about this technology. Because I mean, ever since The Matrix, I had kind of been hooked on the idea. And then I got the headset in late 2014, and I started looking around for... So early in 2014, as soon as the Kickstarter came, I started talking with this Turkish architect named Tarek Keskin, and we started talking about a realistic space station. I wanted to do a music festival inside a space station. I had no idea how this works, like with instances, any of that. I thought, okay, let's just have thousands of people inside this, Big O'Neill cylinder was one idea I had. And then, so I wanted to do this thing where you could see the world above us, like rotate it. I had, that's very unrealistic, at least for now. So late 2014, I started looking at social platforms to do this. Riftmatch might've been the first one I tried. Converge, Janus, all these came a bit later, but then beer chat in January, 2015, I think like one year after they started it is when I first tried it out. And I think they just finished, moved away from T-Pose avatars. So I got in after that, but it was a pretty small community. I kept trying other, other ones for the next 2015, 16. I kept trying others like Altspace, Neosome, High Fidelity, Sansar. Steam had a pretty promising thing too that they never pushed on anybody, but, um, Anyways, by the end of 2016, all of a sudden VRChat had this big update and it got way better. That's kind of when I thought, okay, this team has actually really worked hard on this. They hadn't had updates all year. I thought, okay, maybe this is dying, but no, it was very impressive by the end of that year. And then the next year I started That was 2016. In 2017, I started experimenting with some musical shows. Before that, I was working on this space station design without the festival because I realized how unrealistic it was. We came up with this idea for a realistic space station. We talked to people who work for NASA about how to make this plausible, get all the rotation right, and put a lot of work and thought into it. It never came to be. So then I started on, I learned enough about Unity through VRChat to attempt some early music shows in 2017. And we did two of them. Tofuku, or Tofi as she's known by now, she was the first performer. I think this was possibly the first club type show in VRChat. And people enjoyed them. They were kind of buggy and you have to keep switching instances as we still sometimes do, but they were pretty fun. It was just like, I didn't know that much. And I kind of gave up on the music stuff. After that, later in the year, I got in touch with Beeple, the artist, before he got super rich from NFTs, and I asked him if he'd be in to do it. He liked some of our music videos, so we'd chatted before. That was when I was the biggest fan of his work, when he was doing, I think, his best stuff, in my opinion. We asked a bunch of VRChat creators if they wanted to build worlds from his artwork, and I think about 20 or so made worlds, and you can still find them in VRChat. And so yeah, we brought some of his work to life. He chose his favorites, and the best one I bought a new Vive for, that was when the Vive just came out. That was fun. That was one of my first big projects. So I was doing that, and then the next year I was kind of working on ARGs. We did a hunt for Mr. Whiskers with me and Gunter and some others. and we kind of timed that to start this big hunt for the day the Ready Player One movie came out. That was really fun, actually. And then I did my own ARG called Cubes. It's more based on the kind of cult classic sci-fi Canadian movie, Cube, and also had Ready Player One vibes. And that was basically all of my 2019 working on this. And also working on another larger ARG that I never finished. I take on all these big projects and I can never finish them. But hopefully all that stuff will come out yet. The next year was the start of the pandemic. So 2019, 2020 was the pandemic. Me and my animator friend, Natalia Reiss, started IFF, the ISO Film Fest. And that's what I worked on most of that year. like an early pandemic, and we were trying to show what people were making at home in lockdown. So we tried to gather the best works that we could, and we did our own little film festival, and I built a world for it. And well, I got a bunch of people to make different stages in this water world. It was really cool. And I don't know how many people actually saw it, but I'm hoping that'll be a nice time capsule of the time in the future. But then, so by the end of 2020, I hadn't been working on the music stuff for a while. And then I checked out Ghost Club at the end of the year, and that just blew my mind. The design of it, the music, the music has gotten a lot closer to the stuff I listen to. And so then immediately after, I just asked a bunch of artists if they wanted to make cyberpunk clubs for me. And a few said yes. And so a few people made some, and I focused on The Core at first by Kamin. And basically I worked on that all year and it was a very complicated visual system I was attempting to build. So by the end of the year, I kind of had it figured out the first version. And so we started shows under the name Break at the end of the year. And then that was all of the next year. We did shows by 4L, Solid Dawn and, but some live shows. So because of Lund, Lund's help and the VCAT team, they had a tool called a sync tool. So we could, um, everything's built in unity and there was a timeline that hardly anybody uses, but it helps you build a sequence shows inside of unity. And with their tool, we could sync it all. So people could go back, play and use a slider to go back and forth in the show. And it also makes things like everybody sees it at the same time. It's just without something like that, when you do a live show, you have all these cool visuals, but you can't really make it. When you press a button, we have the system where you can trigger particle systems and 3D animations and shaders. But depending on where you are, it comes in at different times. But with this timeline sync, it's much more precise. And also because of LILU and the VRChat community, they built AudioLink, which is an amazing innovation. It syncs everything to the audio. How to explain that? I'm not great at explaining any of this.
[00:13:51.665] Kent Bye: Well, just to jump in, because my understanding of audio link is that there's different like frequency bands of music and that it sort of creates a quantized response for detecting when there's like a bass kicks, when there's a higher frequencies, mid frequencies, like two different ranges. And so audio link seems to be a way for both avatars and to have like more of audio reactive, responsive. shaders and whatnot, depending on the frequency of the music, then you have this dynamic reactions that happen. And so as I was watching through your latest couple of break shows, I saw a lot of audio link integrations that were making the visuals respond dynamically with the music. And I see that on a lot of. avatars that people have, audio link integrations, but also integrations within the context of the world itself to have as the music's playing, you almost have like the world's like vibrating in synchrony to different dimensions of the songs.
[00:14:47.035] PK: Yeah, it's mostly used by people with avatars, really cool avatar stuff they do. Some people use it to sync lights. There's a few different ways people use it. There's this cool orb world where they do it with particles. What really helped me was Luca made this shader called June. And basically, it's this really massive screen space shader that has all these modules and you can customize it so much that it looks completely different depending what you do. But then I asked if she could make it work with audio link and she just magically made every part of that work with audio link. And that's kind of when the break shows completely changed and I started relying really heavily on that. I mean, maybe I'll rely less heavy on it in the future, but it's a way to make, even in a live show, when you press a button and change the visuals, There's a bit of a delay, but at least once it's on, it'll react with the beat very precisely. And that to me feels, feels quite good. Although some of the stuff we're doing with the shaders might be a little bit much, much for people. So I think for the next show, I would like to make like a full effects layer and a simple layer. So people can still experience the show without having being so overwhelmed. If it's not something that they're at all used to.
[00:15:58.474] Kent Bye: I remember going to the core at some point, I don't know if it's 2020 or when, when was the first show of the core?
[00:16:05.018] PK: I think the first one you would have been at would be the first replayable show was the 4L Silent Dawn show. I'm not sure if that was the first one. The first one was in December 11th, 2021. Okay.
[00:16:17.016] Kent Bye: Yeah. So it's been a couple of years that have been coming to, you've been pinging me to different shows that you've been putting on with break. And I always enjoy going to them because you've created what is essentially like, I think there's been a couple of iterations of different core spherical, like in the round type of locations that then has the outside walls kind of disappear. And then you are taking us on these variety of different spatial journeys. And so both from Like the ways that you're using the timing of the shaders and to just kind of completely modulate what's happening at, I'd say like multiple different layers of like the far field of what's happening at the skybox layer, what's happening like at the mid range with like different objects. And then there's sometimes a near field thing that gets locked to your head. So there's some of the shaders that when you move your head, there's dynamic things that are happening, but that it feels like you're. taking this experience of like, when you go into a virtual world, you have a sense of what that world is. And these shaders are kind of like this augmented reality layer on top of this virtual world that is like completely modulating and doing some really interesting journeys of modulating from different types of shaders and creating these different levels of contrast. So as you're going through this history, maybe kind of elaborate on this aesthetic of mixing and remixing shaders. And if you're like pulling these from like shader toy, or like if you have shader artists that are doing this, or if this is just, you know, certain settings that you're able to tweak to start to create this completely augmented reality type of experience in the context of VR.
[00:17:55.380] PK: Yeah. So, um, Well, let me go back while I remember something a little bit. Like the early shows in 2017, the two I did, two plus I did actually two replayable shows and it's hard to remember how I made them replayable. I guess just a bunch of timers synced or timers that go after each other. So it was a little bit tricky, but so those shows, what I found out is that people, even though the people there wouldn't typically necessarily listen to the same music that I was playing on their own, I found that the people who come to VR early on are pretty adventurous in spirit. If you just give them a bunch of surprises, like you change the world around them, they really like it. When I saw Ghost Club, at first, those shows, I kept thinking if I ever built more stages, I wanted to do this big grand space. And when I went to Ghost Club, I realized the first version of Ghost Club, we were packed in this little room and just like, it felt so social. And this was during the pandemic. And there's also the club like Loner, which was great. And they kind of did the same thing. And what I realized is that it's not always best to have a huge space. So with the core, I actually thought it was kind of a smaller space, but it actually feels pretty big because of the round design. But at the same time, I wasn't necessarily, like, I really liked the idea of having a club to start in, but because of those early shows, I thought, well, you can have a show anywhere in VRChat and put these effects over. It's hard to move effects from one world to the other, but you can bring them in. And so then I thought, okay, let's try where we, kind of like I used to do, but a step up from then, but always start inside of a club. And sometimes people have the choice. The way I've set it up, the DJ can choose whether to remove parts of the club. And sometimes I set up entire new environments. And as you've seen with some of the shows, we just start in the club, but then we leave it for the rest of the time. Sometimes you just keep the dance floor. And then with the shaders, a lot of it's the June shader. And there's also material shaders that we use on the walls. But I use a lot of stuff that people that I find that people in the community make. Sometimes you buy it from Booth. Other times, yeah, most of it's not like other than June, nothing was really customized for me. Yeah, I'm always looking out for more though. So there's quite a few that you find in the community that people share and then you experiment with it.
[00:20:10.641] Kent Bye: So when you're like evaluating a shader, cause a lot of this is happening in the context of unity and then you're immersed in the VR. Are you able to like get a preview of some of these shaders in 2d and get a sense of whether or not it's going to work for a show?
[00:20:23.849] PK: Oh, yeah. Well, I typically in unity, I, I, I usually first experiment in a, just on the flat screen. I'm better now at guessing if it'll work or not, but you can never really tell unless you get it into VR. It's really difficult to sometimes judge the performance of it, so it's always a little worrisome if I find something cool, if it's going to work for others. So I try and invite other people in who have slower machines. Not that mine is the fastest imaginable, but it's, I don't want to make it completely inaccessible for people who don't have the latest GPUs. Oh, and also because I think because VRChat is so CPU bound often that these kinds of visuals are more GPU dependent. So it's actually, it doesn't feel like it puts that much extra load on the experience for a lot of people. Sorry, what was the question again?
[00:21:11.634] Kent Bye: It seems like a lot of your process is, you know, what they refer to as like kit bashing, where you're taking both assets and shaders and you're scouting for something and you're kind of remixing it in a way that I feel like you're able to, in a lot of ways, these break shows, you're taking us on like a spatial journey through the music. And sometimes the music is a bit of like, I hodgepodge of lots of different artists. And sometimes one of your break shows will be like one single artist that you kind of have a, for me, I feel like the latest one you did with, I think break 11, that was at Venice that I just, I watched before, but it kind of broke at the end. So I just watched it again. What I was noticing the second time was after watching Munch, which is, I think was like break 10. There's a lot of different type of artists and the music is very varied. Um, and that, that one is also just kind of like a trippy, like you're, you're using food as this can see to play with scale. And it's a pretty mundane object, but then when you start to like throw shaders on it, it suddenly becomes completely fascinating because you have no idea. what to expect. And it's amazing how you're able to take these really mundane scenes, but turn them into these amazing art pieces when you throw shaders on top of it. And they're consistently remixing between the mundane view and the altered view. But the kind of music that you have, um, maybe you could talk a little bit about your label and how you are, sometimes you're creating this compilation of different artists. And if these artists are artists that you're representing for your King Deluxe label, or if you're also, you know, going out to different artists and trying to get permission from them to see if you can create this spatial journey in these virtual worlds with like a series of their music tracks.
[00:22:51.939] PK: Yeah, I. Well, so the last show, Muta, that's my friend Cliff. I've released stuff of his on and off over the years. He hasn't actually put much stuff out. It's only recently he's become much more comfortable with his productions and he's extremely talented. That show was about half his stuff and half other songs he liked. And so the next show is going to be a release that I was going to do that as break five, but it's taken a while and we kind of took a break on it. I also had a lot of computer issues late last year and early this year. So I couldn't really get into VRChat, which is why I spent like three months working on a tribute to the group Faithless after Maxi Jazz passed away. And so for that show, it was like, it wasn't our music, but it was just something I, I didn't really listen to them in a very long time. And actually, no matter how much I like the music, by the time I'm finished the show, I've listened to it so much, I definitely need a break because I get really nasty earworms. I fall asleep to a song and I wake up and it's still playing. It's playing in the back of my mind for days on end. With The Faithless Show, for some reason, I was worried that their music wasn't going to hold up. But after three months of it, I still didn't mind listening to it. It's really quite timeless. And so, oh, so for the next show, it's for a release that came out late last year. That'll be all their own music. So the five artists that are on the release, they're all doing sets of their own music for it. So that's all original. And the label I'm just calling Break now instead of the old name King Deluxe, because it feels like a new thing. King Deluxe was very much about music videos and animation. That became kind of a thing, but you can't really The music scene isn't that interested in animated music videos these days, it seems so. And it's much more enjoyable. VRChat is the only platform where I can find an audience, and everybody's very enthusiastic, and they love experimenting, trying new things. They're very open. The producers, the musicians on the platform in the community are creating really high-end stuff, trying all sorts of new things. It's a really thriving club scene.
[00:24:47.850] Kent Bye: Well, do you see that these virtual experiences that you're creating, is that driving sales or is that pretty decoupled from the types of explorations you're doing in VR chat?
[00:24:57.156] PK: It's pretty decoupled. It's because there's not really, and this is something I'm really looking forward to in the future is if they can, once we find a way of like, like if you could in the show, just if there was a way to just say, okay, I want to follow this artist and it connected to their Spotify, for example, or if we could build something better than Spotify, that'd be great. But yeah, so right now it's like, I think maybe a few people have gone on and listened to our stuff elsewhere. I mean, people seem to like the music, but yeah, the system of connecting the two isn't great yet. But part of that might be on us to try and communicate it better, to build better systems for it.
[00:25:32.548] Kent Bye: Well, you're really on the frontier of trying to figure out, you know, how to create this immersive music experience. Cause there's experimentations that have been happening. You know, I'd say like the Fatboy Slim Eat Sleep VR Repeat, you know, that was probably one of the most polished from a big artist and Engage XR to take you on this special journey.
[00:25:55.115] PK: I work in that one.
[00:25:56.571] Kent Bye: Yeah, that, that was like over a year of, of like really creating a whole social experience. But, you know, I think for your pieces that you're doing in VR chat, there's a certain amount of like, sometimes you're just in the club and you have like a bunch of shaders and some of the shaders that remind me of the, what's the name of that experience? That's the tree house of the shade.
[00:26:16.539] PK: Yeah. 1001 hit me bad. Yeah. Because I use a lot of this. So they were the first ones to bring shader toys into VR chat. And it was tricky to do because I talked to them about it and they were trying to help me, but it was very complicated at the time I wanted to experiment with that. And a few others did it and it's a little bit easier now, but I still, I kind of rely on others to convert them. And they're just really fun. When I realized I could bring in shader toys and then combine it with other effects, it makes for some really wild outputs.
[00:26:49.015] Kent Bye: Yeah, especially with this latest break 11 that you showed at Venice. I had a chance to rewatch it again and really appreciating. It feels like you're taking us on a journey in certain ways. Like sometimes like break 10, you know, you'll be in the center of gravity of the club and then you'll be kind of mixing, matching different 3d objects, like different types of food and whatnot, and then putting shaders on top of it. But then sometimes you'll take away the club and then you have these different scenes that you can start to explore around in. The first time I think I saw that was probably with the, the L L L L show that you had done where it felt like you were at a certain point taking us on this like spatial journey from one location to the next. And it felt like the whole crowd was moving through this space that you were creating and it created this journey that you're taking us on. So I don't know if that was the first time that you'd done that, but I'd love to hear a little bit of your reflection on how to create this immersive music journey, both from the moments where you're more static, but leaning into the affordances of locomotion and VR and VR chat as you start to move through these spaces and create even more of an epic journey in that way.
[00:27:59.697] PK: Yeah. I mean, I switched environments a lot early on, but I didn't do the traveling through the scene. What I learned at some point was you never want to move people through the scene. You want to move the scene past them. It just makes for a more comfortable experience and you want to do it slowly. I mean, in the next show is kind of a more, a little bit more of a, it's also, it's more of a, like just over one hour and you're traveling through different scenes. It's even more of a, I don't know if it's a very strong narrative, but it has some narrative elements to it and some logical progression. And that one, I actually do move people a little bit, but it's just, there's a specific reason for it. So hopefully it doesn't make people uncomfortable. But like this show was more about exploring the sense of scale, like shrinking and then growing again. And when you're at the small scale, it was tricky to find anything at different levels of reality. So I kind of just improvised. That's where the shader toys came in. Like that's the only way I could find a show like bacteria. And then there is a part where you're feeling like it was interesting, like when you're growing. And when I was scaling the world, smaller to make us feel like we're growing into the clouds. It was weird because I thought it would just feel like you're growing up, but it feels like you're just flying sideways because that's just how it is. When you scale it down, it just feels like you're moving sideways. I don't know how better to explain that, but that was unexpected. But people seem to have fun with it. And then the next show is really the one I started last year, but never finished until now. That one's definitely about taking people through a space. Well, for most of it, a good chunk of it is traveling. and having things happen around you and more like some of it's more confined. So you're just kind of looking out then in the static parts, you can move around. So, I mean, it's, there's some similarities with this one, but on a bigger scale.
[00:29:44.792] Kent Bye: And from the first time that I saw one of the break shows, I think I took a ton of photos. And part of it was because like trying to make, put language to what was happening. If you know what I mean? Like trying to describe or understand the like in music, there's consonants and dissonance cycles. And then in the process of creating an experience with the kind of language of the shaders you have. different colors, you have different contrasts, you have different shapes. Do you have a sense of different types of categories or how do you have language for as you're creating one of these shows, like how you think of it or conceive of it, of what's happening visually and how that visual effect may be tying back either into the music or creating more of a a building and releasing of tension or contrasts that gives people some sort of like cohesion, a journey after they go through these kind of like seeing one type of shader and then you turn it off and turn on another one. As I was going through it, I was trying to think of different ways of describing what the effects are and what the either the emotion or the feeling or the visual effects are. But as you're creating it, I'm curious what your process is of how do you start to make sense of these kind of shader journeys that you're taking people on?
[00:31:01.124] PK: It makes sense in my head. There's one I called Halcyon, which was basically just my part of a live show, and I ended up liking what I made. So I ended up making it replayable. And that was based, so I made a mix of music of just music I liked. I was trying to hit a certain mood and also because of the other artists who were DJs who were playing. So I want to hit a certain mood with it. And so I made the music and then I tried to, and then I usually think of the environment changes and Durs made this fractal Mandelbulb environment for my one ARG I never finished. It was really cool, but then he ended up using it for his home world and spruced it up and did neat stuff with it. But then he asked if I could use it for this show, and he said, sure. And then I thought, this would be really neat to travel through, just fly through. And I just, by that time, had a pretty good sense that these different shapes were going to, like, the way I can have kind of an idea how the, I don't know exactly what the effects will do with these shapes, but I know it's going to be interesting. You can't always guess because different, so I don't always, like, a lot of it's just trying to find happy accidents. It's like you go through all these different shader modules and you just try different shaders and then you change different post-processing color schemes. and that changes it drastically. And you just kind of, I go through like, yeah, so I have the environment or environment changes. And sometimes I put in creatures and I do all that before I start adding the effects. And then, so I already have kind of a bit of an emotional journey with the music and everything in mind. And then I, when I add the effects, I also want to kind of make it cohesive in a way, but first it's just experimenting and seeing what looks interesting and I haven't really done before, hopefully for a lot of it. and just experiment through countless iterations of different effects, and then come up with some interesting ones, and then try and piece it together into something logical.
[00:32:59.785] Kent Bye: Yeah, there's all sorts of different, I don't know, like there's things like bloom effects and then there's having things completely washed out and you know, everything's just like one color. And you go through this experience of like sometimes the whole world is occluded and you don't know where you are. And then, then you sort of subtly bring things back in and you know, a lot of playing with depth and you know, have dots kind of like this matrix through the world and I don't know, I guess as I go through it, there's like a series of different techniques that you have probably about a dozen or a couple of dozen of different types of effects that there's like a pixelation effect. Sometimes there's audio reactive effects. Um, usually when I've seen your shows, I've seen it. with other people. And then when I just saw it now, I did it by myself. And I noticed that I was able to pay attention to other things like the music more, or, you know, sometimes there's like these little things that are flying through the environment that I miss. Cause part of the experience a lot of times is seeing how other people who have audio link shaders, how their appearance and their likeness show up. Yeah.
[00:34:04.090] PK: When I make a show, actually like the last layer of the show, after I do the effects and sometimes I'll add some more animations and particles in there to make it. or interesting, but then the last layer is always the people. I try and leave some sort of space. I count the avatars as the last part of the show, and that makes it unpredictable, because different avatars are going to look completely different. And Prowler, in one viewing of this new show, he has this crazy-looking dragon part of his avatar that he moves around the scene, and it just It was entrancing to just watch this moving dragon in the scene and how it reacted to the world. It's how the avatars will fill out the visual space with how they react in different ways, which is unpredictable sometimes. Then there's also just the social element of moving through the space and it's sometimes interesting like for the next show I want to like cram people in a very small space like it's open and then pretty open and then everybody's going to be I don't know if you can fit 50 people in this space but for the probably the most wild part of the show is just everybody in this I want I don't want to say what it is but uh they're going to be really packed tight before they're let go again for the final set so it's interesting to watch to play around with those dynamics as well
[00:35:21.121] Kent Bye: Yeah, I know that the last show that I saw with a group, there's a new feature that I don't know when it came into VR chat, but you can click a button to match your size to other people. And I've noticed that there were some people that were like super giant in their size and like kind of off in the distance, a number of like furry type avatars that were all dancing together with full body tracking and. the different types of avatars that are super customized and have all these audio link integrations that look like a total amazing light show within themselves. And so it's always nice to go to one of your shows and to see the full diversity of the different types of avatars. And, and it is a lot different to see it with a group. But I think the other thing that I'd say is that Well, the last time that I saw the latest break show, it, there was a bug at the end and then it didn't end. And so I was able to actually watch the full arc of the show. And then the part where it broke before it was like fixed in it, this whole, like, I feel like you really had this whole transcendent, like, you know, in space and seeing the galaxy. And it just was like, I'm so glad that I left myself enough time to finish it because. It just was a really satisfying ending, but it was different to see it by myself versus when I was there with a group and like the first time when it was break, there was all these people that were like falling through space and it became a whole other experience. And also with the VR chat, you have the ability to determine how you want to hear it, which I think is great because you can turn down all the other people talking if you don't want to hear them. And you can just listen to the music, which is what I usually end up doing to kind of hear. the songs that are being performed, but some people may do the opposite where they like turn the music way down and they're just there for the visual spectacle and to hang out.
[00:37:02.651] PK: That was a great innovation when they first brought that in. That was, it's really helpful.
[00:37:07.082] Kent Bye: So yeah, people can choose what kind of experience they want to have, whether it's a social one or whether they're listening to music or they want to block people out. And, and so, yeah, I just, there's a variety of different ways that I've experienced your shows and there's lots to be unpacked there. Now that you've done these types of shows and that you can record them, then people can go in and push the play button and go with their friends and have them available. And I don't know if you're going to be having some of these shows available for the longterm. Sometimes you have them for available for a short period of time and you take them down.
[00:37:36.417] PK: I just did that at first, but people convinced me that it was better just to keep them up. I guess at first I was thinking, okay, it'd be special to have exclusive times and then it's more special when it came back. But then I learned, no, it's probably best to just keep them up because everybody has different time zones and the friends can come at different times. It's just so helpful. At first, I wasn't sure how much work to put into these shows because I didn't have this way of making them replayable. Once we had that, then it kind of solved that problem. I mean, I would like to do multi-instance shows yet, but the way SlyFast manages it, and some others where they can sync visuals, but they use videos. So the way you sync avatars, like the performers and dancers and whoever you want to sync from one instance to the next in almost real time, that uses something called shader motion. which is some really clever math, using color bars at the edge of your screen by locks. And so it records these color bars, then replays it, and then you can control avatars with it. And the same video is also used for the visual. So the avatar sync and the VJ visuals are in the same file, and that's how you can do different instances if you put the work in. It's still a bit of work to set up. But with the visuals we do, you can't really do that. There's no way to sync all these crazy effects. And so the only way to probably do it is to have like, I was hoping maybe we could If people learned the visual effects system I have, then you could have people host their own instance of a party. We could stream in the avatar DJ and the music, and then everybody can do the VJ for their own party. So that's one possible way. The other way is just to make it replayable. Like the next show, I believe, will be replayable from the start. So we might host a few instances of it for a rain dance. And then so like have a few hosts, so you can just people can come in and have a few instances, but then we could still probably make it available. So anybody else could just start their own and play it. And, uh, that way you can see if we could like, so far, all our shows during the live shows or premieres, it's always just an instance of 50, 60 people, maybe 71 time.
[00:39:47.921] Kent Bye: Yeah. So this past week, Rain Dance had announced their lineup and there were 40 total experiences and 75% of them. So 30 of them are connected to a virtual world of some sort, 28 out of 30 are VR chat. And there's one Neos VR and one gauge experience, but pretty amazing that the Rain Dance selection is like 75% of these virtual worlds. And I noticed that you had like both an immersive music experience, but also like an immersive art experience. You're in two of the different categories. Maybe you could talk a bit about what you're planning on showing there at Raindance.
[00:40:20.838] PK: Yeah, so there's the God is a DJ show, which is the tribute to Maxi Jazz and the group Faithless. Maria from Raindance was the one who told me that Maxi Jazz passed away. So we were just going to go play some tunes of theirs in beer chat. But basically my machine wasn't working at that time. I had to upgrade some equipment. So I ended up just making a whole show around it. But they're going to show that, which is cool. It's almost two hours long, so it's a bit more of a commitment. I'm pretty proud of that one. And then the new one, Equilibrium, it will be a little bit over one hour. And as I mentioned, that's the release from last year, the music came out last year, five of the songs. So it's the five artists from the release and each will do 20 minute sets and it will progress one to the other. So there's no breaks. It's just like a journey that goes from one step to the next. Although in between, there's an artist I released years ago, Evie Jane. Well, it's Evie plus her musical partner, Jeremiah. They're a duo. So they're a very talented duo, although they're not making much music anymore, I don't think. My favorite Evie Jane song was a version of It was released on an album, but my favorite version was something that was never released. I wanted to do a VR music video. When I was starting in VRChat, I was also trying to get people to create Unreal Engine music videos, like VR music videos. That was an early thing. So I couldn't learn Unreal and Unity at the same time, I realized. But I was hoping others, I had a few other people make environments for Unreal, and we just didn't do anything. We still haven't done anything with them because they're very nice, but it's just, nobody knew how to turn them into VR scenes. So like with Ebi, we had her motion captured in the Vancouver studio, like full motion capture. We scanned her out, her photogrammetry scan, everything. And those also didn't turn into anything. But finally in this upcoming show, in between the sets of the, without giving away too much, there's a moment, a five minute moment or so where we're in between two sections and it's kind of calmer. And that's when we'll play one of those songs and she'll be in there as some sort of being. And Yeah, so we'll finally get to use the motion capture we did for that song, which is, I'm glad that was in 2016, like the start of that year. So it's been a while.
[00:42:38.912] Kent Bye: Yeah. Yeah. Always appreciate what you are able to put together with these break shows. And I know that I often run into Alun when I'm at these shows and I asked him the last time I was like, is there anybody out there in the VR chat realms that's doing anything quite like what PK is doing here with these break shows? And, you know, Alun's been doing some different stuff with the virtual market and VK.
[00:42:59.440] PK: Yeah. He's really on the cutting edge of this stuff.
[00:43:02.348] Kent Bye: But he said, no, not really. There hasn't been really a lot of other folks that have really been exploring the visual. There's the DJ and the VJ. And I'd say that you're kind of pushing the edge of the video DJ, or although it's more of like, I don't know, the shader DJ or the VR DJ might transcend what the VJ means, but this kind of visual component of. these songs, I feel like you're really pushing the edge for what's possible. I know The Wave with WaveXR, they were doing a lot of experimentation for a long time.
[00:43:30.084] PK: A lot of new particle stuff, Aaron Lemke and the rest. Oh yeah, Aaron Lemke was one of the first people I checked out when I got my DK2. The first was Fruxius and Sightline the Chair. Maybe I tried Tuscany first, but I was waiting to try Sightline and it did not disappoint. And I know Aaron Lemke had some of the earlier cool experiments, so did Khabibo. So the Wave was great and had so much promise, and they unfortunately stopped doing VR. Hopefully it'll come back. And then what else was it? Oh, there are a few like Cat Club, second in Japan, Cat Club. They did, it's more just a box room typically, but they were experimenting with, maybe I started slightly before, I'm not sure, but they, with the modern version of this, but they kind of independently did all these crazy shader things. So they do some neat stuff. And Unreal Journeys is starting to do some shows like this. And today, I found something that I quite liked that I just discovered today. Well, I'd heard of it, but I hadn't checked it out. Nico's Exoplanet Journey. So it's not quite the same, but it's like this. If you haven't checked it out, it's like you're in this sort of spaceship, and then you press buttons for different scenes that changes the environment around you. It's kind of a similar idea. The thing is, I've been wanting to do a show that's very meditative. Years ago, one of the replayable shows I did was kind of that idea where it's just like, I didn't have all these effects, but I had, Michelangelo does these great 3D animations. So I used them with this music and it was, and people told me they fell asleep in VR to this. And I was surprised. I didn't know people could fall asleep with a VR headset on at the time. So I've been wanting to do another show with these effects, but make it longer, like stretched out. But when I saw the Exoplanet today, I thought, it's very calming music. It has different environments. It's not quite the same thing as I planned, but now I'm going to have to rethink it because I really love what Nico's doing. And I have to think of, first of all, not to copy what they're doing, but also it's kind of inspiring.
[00:45:24.777] Kent Bye: Nice. Yeah. I know that like, when I go through one of these experiences, I feel like it's this combination of the visuals that really go beyond ways of being able to easily describe it. And so it ends up creating this texture visually that is mimicking different aspects of the emotional journey that the music takes me on. And so it ends up being this kind of vibe that is like almost sub-symbolic where it's like really difficult to describe. You almost have to like experience it yourself to really understand it. So.
[00:45:53.872] PK: It's really quite a unique, uh, yeah, I really like having the music and visuals blend in ways that are hard to describe. I mean, you would like what the music videos, it was like, um, my favorite music videos we did or even that others do is when you can't really tell what came first, the visuals of the music. And also they change each other. Like. the music will make you notice different things in the visuals and vice versa. And yeah, so that's part of the idea with the shows is to try and like for this, um, probably my personal favorite moment was this latest show. We move at night was maybe the trippiest moment when you're kind of like the smallest scale of reality and there's this creature and some of the effects, it's just like the music. I wasn't one of muted songs. That was, um, Ivy Lab, I think, and it's kind of this strange alien song with this alien creature. And the way the effects react with the shaders and the music, and it just feels like, it's hard to describe, but they kind of really blend together as one kind of unit there. That was like my favorite shader moment of that show.
[00:46:56.641] Kent Bye: Yeah, this latest show, We Move at Night, break number 11, I really loved the journey that I was taken on and lots of different moments of awe and wonder and really loved where you ended up to this kind of transcendent viewing of the galaxy, of these particle effects. It just, yeah, just a really amazing journey. And, um, Yeah, I guess as we start to reflect on other aspects of VRChat or anything else, I'd sort of give the floor to you. If there's any other topics that you wanted to bring up to discuss, we've been talking about your own creative journey or different experiences, but I know there's quite a lot of other things that are happening in the context of the world of VR and the VRChat community.
[00:47:34.251] PK: Let's see. I wrote down a few things that I thought I was looking forward to in VRChat as it develops. First of all, something that's going to change these shows and change everything is if we ever get much larger instances, like hundreds of people in the same space, I fully assume it's, it's inevitable one day. It's just, I know there's the RP1, the guys you interviewed. Yeah. I haven't actually, I tried to get into one of their instances and I had some issue trying to get in on a desktop, but that sounded really interesting. So that shows some promise. I don't know if it's coming anytime soon, but so that's something to really look forward to. One thing I really want is. For us to have enough processing power that we can more realistically model audio reflections, meaning we can take, one day, model any space and hear what it would actually sound like, like how materials sound like when interacting with it. For example, if you want to walk barefoot on the beach, the sound of your feet and sand will take you there a lot more than just with the visuals or like building a giant echoey cave for a party. Like if you find a real cave IRL, it would probably sound bad because of all these random reflections. But in VR, you can design the dimensions to amplify the sound in unique ways and really it would feel much more massive. What else? Voice recognition in many languages. That's something I've been looking forward to. It's not so much a show thing, but you can use it for fun, interactive worlds that help you learn a new language, for example, where you have to learn to pronounce words by engaging with NPC characters. More immediate use in VRC, I guess, would be using voice passwords to unlock parts of the world, where I'm sure people will create all sorts of spells that do who knows what. I'm looking forward to social historical experiences. ideally more accurate than film history, because they usually just loosely base it on history. Stuff like Pirate Queen, that's something I'm really looking forward to. You interviewed the Pirate Queen people, and I even learned about pirates, and I had never even heard of this woman before. And so, I mean, I'm not so set on, I'll probably enjoy the gamified part of it. That's not, me personally, I'm not so much of a gamer. I don't mind just being put through the experiences as more like a film, but either way, I'm quite looking forward to stuff like that. I think this is something we're engaged VR. I'm hoping goes to that direction where they, cause they're, they're a teaching platform and they're doing shows. So to them to have interactive historical experiences would be really fun. One more I had here. Something I'm also looking forward to that's not so far off is filmmaking in VRChat improving by leaps and bounds. As it stands now, people used to putting on a headset and immersing themselves in these environments. I think when they see video footage, looks a lot more impressive because they're painting over the scene in their own mind based on memories of these spaces, but outsiders can still look very uncanny. But with better techniques, using better methods, and these tools that were given, and the community tools, I think it'll improve a lot. And I suspect We'll get to the point where indie filmmakers are using VRChat or similar platforms to a central component in their process, and maybe mainstream filming as well, or at least a more virtual version of what Disney is doing now with Unreal Engine and the Mandalorian and other Star Wars series. So I think to do entire, like I have ideas for entire, there's a lot of kind of storytelling I want to do in VRChat one day, if I can get to that point and have a team of people and do original art. and maybe not even just film a movie in there, but have an interactive film where it's 360, it's all around you. So that's the kind of thing I want to do for shows yet too. And one other thing is I'm hoping like as there's so much potential in this in VRChat and others, and I just really hope that we can have people keeping them in check. I think journalism is very important. I hope people start doing a lot more independent journalism and that they're embraced and we can just build this healthy community. And yeah, I hope they don't sell it to Google or Facebook or anything, because it's such a great place. And after listening to Athena's retelling of what happened at the end of Vault Space, that was heartbreaking. I don't want us to go through that. So yeah. So I hope this sticks around for a while.
[00:51:51.903] Kent Bye: Yeah. I mean, I think VRChat still in the process of implementing their own monetization scheme, which I think is, you know, they have ways for people to become subscribers. And I don't know to what degree if the yearly subscribers or monthly subscribers, if that's offsetting all their costs, I'm imagining that they're still largely building off of their VC funding, but I don't know. It's hard for me to know as they move forward, the economy is going to be something that's going to not only make them sustainable, but also it's going to change the culture in a lot of ways. So, I mean, you're a creator and you are putting a lot of passion and energy into these experiences, but yet I don't imagine that you're getting any financial return on it.
[00:52:33.787] PK: So no, it's just purely a hobby. It would be like, I mean, I, not just for myself, like there's so many talented people that come in and try something and leave because there's no, can't afford the time. In a way, it would be great to have this greener economy. I think the big issue there is that Steam and Oculus and Apple, once they're in it, are all going to try and take a third out of every transaction. I don't know what they've worked out, but It'd be nice if we didn't have to go through those platforms. I mean, I use SteamVR, so maybe they deserve a tiny share of it, but it must be very difficult. I mean, I'm assuming that's why it's taken so many years for them to work anything out because it's... And I don't know how the beta version of this is going, but with these middlemen taking their cut, it's easier for people just to go on Patreon and keep most of their proceeds. But it would be really nice if the middlemen were cut out and then VRChat could take a nice small share and then build this. There's so much potential in what could be built out of this. Yeah, it'll change it a little bit because I like the free collaboration. VRC Fabs has this great community of helping each other out, them and others, but it might change with these financial incentives. But I don't know, I think we have some good years in front of us still.
[00:53:51.327] Kent Bye: Yeah. Yeah. And especially within the past week, unity announced their runtime fees, which feels like that's fine.
[00:53:57.951] PK: Yeah. I don't know if that affects VR chat, but I'm kind of scared to ask. I hope not.
[00:54:02.134] Kent Bye: Well, I mean, it says for any unity runtime, if you have over $200,000 of income and you have a certain amount of downloads.
[00:54:13.862] PK: Oh, so everybody logging into VR chat.
[00:54:16.133] Kent Bye: No, it's basically every time. So the runtime fee is every time somebody installs the application. So VR chats, basically a free to play type of experience.
[00:54:27.480] PK: Oh, cause unity is part of VR chat. Okay. So yeah, got it. Yeah.
[00:54:30.781] Kent Bye: VR chats running on unity. So that means that if there's a million people that install VR chat, that means that VR chat would be on the hook for paying $200,000 to unity. So basically the installs is disconnected from revenue. If it was just a pure revenue share, that'd be one thing, but. If you are a free to play game like VR chat and you have 20 million people that are installing that VR chat would have to pay unity $4 million.
[00:54:58.678] PK: Um, so this could like really kill the creator community part of it. Cause the only way for them not to pay unity is a fortune is to remain free to play or like not have that financial aspect to it. Wasn't it the way I thought it was something about if you also make more than $200,000, well, but they are right. But it's, if it's free to download, they're not making money off the sales. I thought that's okay.
[00:55:20.408] Kent Bye: Well, so that's the problem with the whole thing is that it's not tied to revenue. It's tied to downloads, which is totally okay. I thought it was both, but it's not a, it's not a pure revenue share. It's if it was a revenue share, it'd be easy. to calculate, but if you are a free to play game like VR chat, then it's basically been an unhinged new policy that could potentially completely disrupt the entire ecosystem of game development for lots of different shops.
[00:55:49.044] PK: Yeah, for now, I'm going to give Unity the benefit of the doubt. I know there's one guy that came in from EA, I think, that is probably spearheading this. Unity has some good people in it, and let's hope that they haven't just decided to cash out. Because, I mean, Unity has, I hope they see what we're doing here. And they do customize their engine for VRChat, and they have for the new version. So there's quite a relationship here. So I'm going to... until I hear otherwise, give them some credit that it's not going to destroy our scene, but I don't know.
[00:56:22.204] Kent Bye: It's, it's been very disruptive and you know, they may walk it back, but yeah, John Riccatello and the execs are not really backing down at all. But anyway, that, this is sort of like part of the larger ecosystem discussion about what's going to happen. It's sort of just dropped this week. And I think as time goes on, we'll get more information as to if it's going to directly impact games that both VR chat and rec room and a number of other VR experiences that are in the more free to play model are going to be potentially disproportionately impacted by stuff like this. So anyway, it's an open question as to where that's going to go. But yeah, like you said, the audio engine stuff, as you have like a physics engine within some of these game engines, I think eventually we're going to have more audio engines And there's been so much focus on the visuals, but the audio ends up only getting like five or 4% of the total CPU capacity often. And so I think with some experiences as we move forward, I'd love to see more emphasis on this type of audio reflections and real time audio stuff as you move forward. So yeah, lots of things that on your list of things to look forward to, I think that are going to really potentially really change the experiential dimension of some of these different music experiences.
[00:57:36.314] PK: And another thing is just like, and I've thought this for a long time and it's haven't really figured out but it just seems like a lot of it is up to the community to figure out like. like VRChat, they made this more powerful menu. It's kind of messy and hard for new people to figure out. And I still haven't really explored it as much as I should probably, but they try to make, do some things for discoverability. I don't know if I'm having an easier time discovering things. However, I think the committee will over time, hopefully figure out ways of curating good art. And like, I think that's just something I'm looking forward to how they I mean, there's a lot of things people are trying, but to try and share the stuff they like and just a lot more curation, because there's just so much out there when it comes to worlds. I mean, a lot more people are customizing avatars, but there are a lot of world creators too. So I'm just looking forward to seeing how people come up with better ideas than what I've had about how to share stuff, especially without something like Twitter being reliable for it.
[00:58:33.401] Kent Bye: Right. Right. Well, I guess as we start to wrap up, I'm curious what you think the ultimate potential of virtual reality and this type of exploration of music journeys and explorations might be and what it might be able to enable.
[00:58:49.290] PK: Well, I've talked a bit about some of the stuff I would like to do with the shows and I think they can get a lot more attractive, bigger scale. Like, I mean, it's hard to even start with that. I just think it's like, for example, something I hadn't, um, been thinking recently, but one more near near future thing is I would like it to get to the point where a lot of the artists, musicians, people like end up like they do these tours, and then you add a virtual, maybe a few virtual dates to the end of it, or it's scattered in between. So I think as we show how As it gets better and better, I think it'll be very desirable for people to throw in some virtual dates. And people will start realizing it's actually maybe cooler to do these shows. And it'll become a big part of the music scene, I hope. And then as that becomes a bigger deal and you get larger artists taking part in here, you'll have more ways of connecting the two and the scene with what we're doing here. And then as for the ultimate potential of VR, for me, it depends on the time scale. When I think of what could be the peak of this technology, might not rise or fall, rather evolve and mutate and eventually change enough to be called something else. For me, the ultimate potential isn't necessarily positive though. One possible scenario is An even more online connected planet with Zuckerberg's IOI overseeing it all were his children by that time. Zuck was willing to invest heavily early on, not because he truly understands people, but he's desperate to control the future markets. He like worships Augustus Caesar right down to his haircut. And so he's thinking ahead. He knows how to leverage power to steal ideas and water them down and make sure nobody else can compete. Sure, he's failed with the metaverse so far, but that's kind of encouraging. But others like him, they'll have access to all this biofeedback data at their disposal without good oversight and regulation. I can imagine it being somewhat similar to the America in Ready Player One, minus all the stacked homes. That is, if we even get to that future in our generation, seeing how unstable society is and will become when climate change really ramps up, It could have positive effects though, like letting us hang out with people from different parts of the planet, as is happening via chat on a daily basis. I think it'll always be messy, but I can see ways in which it is likely to help us understand each other better, especially if the kids grow up with this technology. And then if we're looking at positives, I think this will allow people to create new forms of art, kind of what I'm trying to do, but I think this is just like the very beginning. And like Dr. Morrow is just mind-blowing stuff. That's like my favorite VR artist of all. And that's a good example of just some of the things people can do in their free time already. And basically, they take what's in their head and for better or worse, often better, share it with their friends or the world. It will make it easier to learn new hobbies and new jobs, cut way down on travel. Eventually, most tourism might be virtual, although I expect Most that can afford will still want to see things in person, but maybe they'll do virtual hikes before they try it out for real. I don't think space tourism, I don't think it'll even take off much as if we can just make the same experiences in VR and feel like we're really there. Maybe we don't have to spend as much money putting people in space other than for some research, but Maybe the ultimate potential is a future where our minds are networked. We can share sights and sounds and other senses with each other remotely, which other people have talked about. We have very little internal privacy. We no longer call it VR instead. That would refer to the clunky glasses from our past. It's really hard to imagine the societal changes it would lead to, possibly centuries from now, although I suspect it's not a future people living today would be able to handle, as we haven't grown up having our social platforms data mine our memories. Although it's not clear, technology can never get to that point where they can make sense of our thoughts. Like for people with aphantasia like me, I don't have much for internal visuals, which is, I think, a reason VR has such a cool effect on me, because it kind of provides them for me. So I don't know, maybe I'd be safe from prying advertisers in the future. I'm not sure, because I don't know what could ever be gathered from my thoughts if we got to that point. If we make it that far, who knows, maybe we already did. Maybe we've already reached the ultimate potential. VR could be a thing that doesn't pass or just simulated for their sport. Although not sure if we could ever even know if that was a reality. I think like many who spend time in VR, I can imagine this as a possibility, but I'm wary of people like Musk who seem to believe it literally without any evidence whatsoever and treat the rest of us like NPCs now. So I think that's problematic, but I see it as It could be our reality. It's just, you can never know. I don't think so. Yeah. I'm not sure what the final form of this will be. There's so many paths it could take. My best guess is for the next few decades that we have some really interesting years ahead where creativity flourishes and then it gets squashed by algorithms and investors trying to monetize everything. But I'm just guessing based on history of tech so far. So I don't know, maybe we have a good decade.
[01:04:03.556] Kent Bye: Wow. You, you, uh, yeah, I have to say you came prepared for that question.
[01:04:08.913] PK: Yeah, I think about that question a lot.
[01:04:11.516] Kent Bye: I can tell.
[01:04:11.956] PK: It keeps changing. It got darker over time. My idea of the future of the technology used to be a little bit more utopian and not so much anymore, but I still see it. I think it could get, I mean, I find it kind of utopian now, like it's despite the whole pandemic, like we got to escape into this, like people in VRChat didn't have, I don't think had nearly the same level of despair during the lockdown as others did. on average, so we did well here for the most part. I mean, there's still a lot to deal with, but it's already kind of utopian in ways, and I think we saw what we saw, but until the climate destroys us, I think this could be some fun years.
[01:04:58.872] Kent Bye: Yeah. And just one quick follow up. I mean, you had so many different things that could be a whole other multiple hours of conversation, unpacking all those things. But one thing I did want to ask and follow up on is that you'd mentioned that you have like a Fantasia or what was that, that condition that you have?
[01:05:13.380] PK: Yeah. A Fantasia. That's, um, So I was listening to a podcast, Stuff to Blow Your Mind. It's a science podcast I've been listening to for years. And they were talking about aphantasia. And up until that point, I and a lot of their other listeners assumed that when anybody else said they saw something in their mind's eye, they were speaking metaphorically. Because when I look in my mind's eye, I kind of feel like I see colors and stuff, but when I zoom in, there's not really anything there. The idea of picturing something and drawing it, that's ludicrous to me. And when I read books, or listen to books mostly, it's hard to get through the first section of it because they're kind of laying out this scene. And it really doesn't stick. It's just like, it's hard to get through the beginning of most novels. And I'm not alone in this, I found out. But now it makes more sense that I know that other people literally do have pictures in their head. So they, I mean, it's a spectrum and I'm not, there's people who have even less than me probably, but I don't have much. So basically, I think I see things a little bit more in other ways. Like, so I would say with visuals, I have almost nothing in my head. which is one reason I think despite, well, I should say also, people in VR chat might not see me around often because since the beginning, I've been incredibly motion sensitive. I can only hang out for sometimes not even an hour before I motion sickness. So that's kind of kept me not from not really hanging out as nearly as much as I'd like. So, but that's something I deal with. But at the same time, I think what kept me, most people who have that level of motion sickness don't keep coming back early on. But for me, it's just such, Even 3D movies had a big impact on me. I loved 3D movies, unlike most people. And VR was much more pronounced. But yeah, without it, I don't have much for internal visuals. With audio, I think I'm kind of average, with something like sense of touch. is completely different. And I don't know why the brain develops this way. I think you start one way and then you lean on what you are good at and then the other stuff gets worse. I probably had better visuals when I was younger. But for example, if I imagine petting a cat, I can basically feel it. It's not the same as petting a real cat, but it's not entirely different. So the way my mind simulates touch is very realistic. And I think there's quite a few people using VR who have that as well. Phantom touch, they call it. But yeah, so I have very good model of touch in my mind, but visual is not so much.
[01:07:38.647] Kent Bye: Okay.
[01:07:38.807] PK: Well, that I guess helps explain maybe some of the different visual explorations that you're experimenting with, with the break, which is very, very hard to understand what, like, one thing that just really helped me is once I learned about aphantasia or this whole suite of senses that we all experienced different, it really started helping me understand that. different people have completely different ways of experiencing the world. So there's certain times when you just feel like, how on earth is this person not understanding what I'm saying? But it really helped me appreciate that it's not their fault. It's like, there's things that I'm just completely oblivious about. And yeah, so it's just, People might react in similar ways, but the internal workings might come to those actions in completely different ways. I just kind of see it now as we all have very unique software from each other that grows over time and evolves. It's just everybody's a snowflake, unique.
[01:08:35.682] Kent Bye: Nice. Nice. Very cool. Well, I guess as we start to wrap up and, you know, we answered the big question and I'm wondering if you have anything else that's left and said that you'd like to say to the broader immersive community.
[01:08:48.245] PK: Well, um, if anybody comes to the shows and likes the things we're doing, um, if they ever want to collaborate, let me know because I very much, I like collaborating with other people and, um, It's usually a team effort. And, yeah, so people who make shaders or any kind of visuals or DJs, yeah, just hit me up, I guess, in I don't know what the best way to do that is. The VRChat community uses Discord mostly. I'm pretty easy to find there.
[01:09:18.108] Kent Bye: You have a Discord for the break community, right?
[01:09:22.171] PK: I do. I guess you could just link it.
[01:09:24.633] Kent Bye: Yeah, I could put a link to, if you have a consistent invite, I can put it in the show.
[01:09:28.096] PK: They could go to, uh, break vrc.com. I'm going to, I was going to start that site in the next week anyway. So yeah, go to break vrc.com.
[01:09:36.421] Kent Bye: Okay. Awesome. Well, PK, I really appreciate you taking the time to help unpack some of your process and your journey into creating these really amazing musical journeys with lots of explorations of shaders and, and music. And like I said, it's, it's a kind of a new grammar that I find. really compelling experientially. It's difficult to put words to all the time, but just like one of those things you have to experience to kind of understand the new forms of how to modulate someone's experience as they're listening to the music, kind of the new frontiers of music video and using the spatial affordances of VR to kind of like take people on this whole sonic journey that has emotional tenor to it, that creates a whole, whole vibe that can be difficult to articulate exactly what it's like as you have a variety of different experiences now, and lots of folks that are experimenting with this. And, uh, yeah, a whole social dimension as well. You know, the eat, sleep, VR, repeat from that boy, slim had a whole social dimension. And there's also in your experiences, the other social aspect of it as well in VR chat and that context with the avatars and everything else. So it's a many different layers as we tried to unpack throughout the course of this conversation. And, uh, I really appreciated hearing a little bit more about your journey into making this from planting trees to tinkering around and starting your own music labels and, you know, creating these really epic adventures within the context of these VR chat experiences. So, yeah, thanks again for taking the time to help share a little bit more of your journey and your creative process. So.
[01:11:07.570] PK: It's my pleasure.
[01:11:09.308] Kent Bye: Thanks for listening to this interview from Fitness Immersive 2023. You can go check out the Critics' Roundtable in episode 1305 to get more breakdown in each of these different experiences. And I hope to be posting more information on my Patreon at some point. There's a lot to digest here. I'm going to be giving some presentations here over the next couple of months and tune into my Patreon at patreon.com slash voices of VR, since there's certainly a lot of digest about the structures and patterns of immersive storytelling, some of the different emerging grammar that we're starting to develop, as well as the underlying patterns of experiential design. So that's all I have for today, and thanks for listening to the Voices of VR podcast. And again, if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listen-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
