The Night Under Lights Presents: The Seasons at the Moon Pool (replayable VRChat link and video) shown during Raindance Immersive was one of the most immersive music experiences I’ve ever had in VR so far. It was so awe-inspiring and incredible to see 3 DJ / VJ sets using the SNR Labs particle-screen version of their holographic Apple Global Illumination shader that I covered previously in episode #1413 with Apple_Blossom and A://DDOSS. The real-time immersive lighting effects have to be seen in VR to fully appreciate the awe-inspiring nature of it. You can see a video capture of it below, but if you have PCVR, then I highly, highly recommend checking out the replayable version on VRChat.
VJs are able to input their 2D video livestream into the SNR Labs particle screen shader, which then translates the luminance values into height and enables a sort of volumetric translation. It has a DepthKit-like effect where you can see 3-dimensional shapes and objects with shadows or imagine if you were to push a cube through a sheet of fabric to get a feeling of objects from one-degree of fidelity, which is height from a fixed plane. VJ Silent and VJ namoron were instrumental in helping to develop this shader with SNR Labs, and TehArbitter’s Moon Pool location for Night Under Lights was also instrumental in helping to shape the development of it. VJ TehArbitter (aka Arbi) brought together two other VJs and three DJs to present three sets with the theme of the different seasons. The first set was DJ Ladybug + VJ Silent, the second set was DJ Yuna + VJ TehArbitter, and then the final set was DJ Terkoiz + VJ namoron.
I had a chance to catch up with half of the performers including VJ Silent and VJ namoron as well as with DJ Ladybug to talk more about the development of this specific project, but also to recount some of the history of the visual effects in the VRChat clubbing scene as well deeper trends of volumetric and holographic visual effects that VJs are exploring in VR. Also whether it merits a new terms as video jockey or visual jockey doesn’t always fully encompass the holographic, volumetric, spatial, immersive, or narrative elements that are being explored. Some of the VRChat clubs discussed include: Dimension, Virtual Base event, Shelter, Ghost Club, Concrete, Concrete: Pale Sands, PSHQ, Slyfest, Loner, Kaleidosky, Sanctum, Schism, Deviate, Atlantis, Mochii’s Island, Reflections, Redacted, Heist, Light Dimension, Simplicity, & Endless Serenity
Also worth noting is that there are three VRChat worlds that are being featured in the Venice Immersive Worlds Gallery that connected to the themes discussed in this episode, and they’re also three of the 20 VRChat worlds in the Venice selection that are not and likely will not have a public-facing world that’s easily accessible. The NUL Presents˸ The Seasons Replay ＠ moon_pool is not in the Venice Immersive worlds gallery, and does have replayable version. But Silent works at Sanctum, and their billboarded stereoscopic screen in the Reactor is being featured in Venice, and the SNR Labs: Test Facility will be demonstrating the more 3D version of their Holographic shader named Apple Global Illumination, and Silent also mentioned Jerk and May who have the Concrete: Pale Sands, which is also a part of Venice Immersive.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.458] Kent Bye: The Voices of VR podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. It's a podcast that looks at the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So in today's episode, we're going to be talking to two veteran VJs and a DJ within VRChat to not only talk about some of the history and evolution of visual effects within VRChat, but also one of the most amazing musical experiences that I've seen in VR since I've been in VR, which It's called Night on the Lights, the Seasons at the Moon Pool. And it was using a lot of the volumetric technology that was featured in the last couple of episodes with SNR Labs' Apple Blossom and Adidas and Namuron. They developed what's called the Apple Global Illumination System or the holographic technology or the particle screen. There's different iterations and versions of it. So in the previous episode with Solface Stuff, they were doing more of a 3D volumetric version where they're able to feed in through a video feed signals that then get kind of re-stitched together to sculpt a 3D object. So they're able to do video input and translate that into a 3D object at 30 frames per second. And in this specific experience, they were having VJs that were able to input their video screens for whatever they're doing on their computers and then have that translated into two separate screens. But rather than 3D objects, it's more of kind of a particle screen. So it's a flat plane and you're pushing up pixels based upon its luminance value. Then it also will translate that color into lights that's really beautiful and awe-inspiring light effects that are being emitted in the surrounding environments and avatars and everything else. And so it's just a super awe-inspiring effect and one of my favorite music experiences that I've had in VR so far. So Silent, Neymaron and Ladybug are also been involved in kind of the evolution, the history of these different types of visual effects over the last couple of years in VRChat. And so there's been this evolution for going towards realism and creating the lighting effects and then doing different screen based effects with VJs and then VRChat. doing shaders, so volumetric effects within the context of a virtual world, and then screen-based shaders, which are more like augmented reality filters that you're giving to each of the audience members so that as they look around the space, it's like they're kind of cracking through the matrix and seeing the virtual reality through a different lens. And then now moving more and more into like ways that you're able to use the volumetric and spatial affordances of virtual reality to be able to push the limits for what type of music visualizations or augmentations for musical experiences or even kind of journeys into spatial and immersive narratives that are happening as well. So lots of innovation that's happening in the context of VRChat. And there's been some pretty big breakthroughs in terms of some of the technologies that are enabling this type of move into more spatial, volumetric, and holographic VJing that we're seeing within VRChat. Either stereoscopic video that you're able to then have kind of like a spatialized view, so it's more of a screen-based 2D billboarded that the Sanctum has been experimenting with, or The concrete at Pale Sands has been doing VGA effects that then get the projection map effect on this huge, fast, expansive space. So that's a concrete at Pale Sands. It's also being featured in the VR Worlds Gallery at Venice Immersive. And then there's more of these volumetric effects that you're starting to see with these shaders that are able to take these video inputs and either translate them into 3d objects or to create more of a plane based particle screen effects that are also quite alluring when they're giving off all these lighting effects within the context of the environment and that's snr labs test facility that's also being featured Oh, and so the night and the lights, the seasons at the moon pool is actually available as a replayable VR chat world. And I highly, highly, highly recommend that you go check it out. At least there's a video down below to watch a 2d version, but it really doesn't translate as much to just how magical immersive it is to see these type of volumetric and lighting effects within the world itself. And so the world is named NUL presents the seasons replay at moon pool. And I'll put a link to it in the description down below. But you can also find it if you search for Apple underscore Blossom user. And then it's one of the worlds that she's released. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Silent, Neymaron, and Ladybug happened on Wednesday, July 3rd, 2024. So with that, let's go ahead and dive right in.
[00:04:26.782] Silent: I'm Silent. She and her pronouns. I'm a VJ and event producer and immersive artist in VR chat.
[00:04:35.342] Namoron: Hi, I'm Namo, he/him, or they, and I've been doing VR VJing for the last few years.
[00:04:46.325] Ladybug: Hi, my name's Ladybug. I use she, they pronouns. Yeah, I've been DJing for about a year in VRChat, and for half a year, I've been leading meditation classes.
[00:04:58.168] Kent Bye: Awesome, and maybe each of you could give a bit more context as to your background and your journey into VR.
[00:05:03.567] Silent: Yeah, sure. So I started to get into VR. Originally, I was working for a video game company. Me and a friend started a video game company, and we had a project fall through. And so we were looking into VR as our next area of interest to see what can be done in that space. So I got a headset to research VR. And then I realized that VR is not really a gaming platform. It's very, very strong as a social platform. And I found my way into the VRChat social scene and quickly into the club scene. And I was enamored by the production level of some of the events that were happening. This was back in 2020. And I've always worked in event production or audio production, being a touring artist and working on shows in the real world. And I had lost that outlet in my life. And then when I found it in the VR chat clubbing scene, I got pulled in by this one person, MissStabby, who reached out to me when they saw that I had some interest in working behind the scenes and more of the visual creative part of things. And asked if I wanted to do visuals for them. And I had never done visuals before. And she said that if I didn't do it, then she was just going to throw a bunch of GIFs on the screen. I was like, well, if she's going to do that, I think I could do better than GIFs. And that was four years ago. And I have fallen so deep down the VJ visual rabbit hole that I'm now making my own animation work and my own interactive real-time 3D experiences. And that's where I am now.
[00:06:42.114] Namoron: I first got into VR back in 2017, mostly for gaming, and then kind of dropped it for a while. I think all the games are very cool, but I see the social aspect as the main thing and interest for me getting in VR nowadays. I haven't really touched any games in a bit, VR side. And I originally just kind of got used to speaking at first. I was very shy, and VR really helped me open up. eventually I ran into Shaggy who was making a VR venue and showing off screens and that got me interested in DJing and For a while, I was just making prerecorded scenes for different people playing in the VR chat. And Modular Monday eventually took me on to try out my first live VJ gig. And shout out them. They really did help me open up to being a live VJ and trying to pursue this further. That's kind of where I am, I guess.
[00:07:55.256] Ladybug: yeah i think what drew me to vr mostly was i had seen the raves and stuff advertised on like YouTube and people were like oh check out this you can rave in vr kind of thing because i've been raving irl since like 2002 but i had stopped for the last 10 or so years i hadn't really gone to many raves and i was like wow this is a cool way to like kind of get back into it and enjoy some music. And it was right around the time a partner of mine had dragged me into VRChat. And as soon as I got there, I was like, this is really amazing as like a social place. And eventually I found the raves and everything. And there's this one place, not really a rave, it's more like audio visual experience called Light Dimension. And I had always really, really been into music, like since I was a kid, like so, so into music. I have a huge collection. And they were just like playing playlists. And I was like, well, I got some music to share that I've been, you know, holding on to for like decades. And so they let me play some music. And eventually, you know, a friend gifted me. They saw how good I was. So a friend gifted me a DJ controller. a ddj 400 and like after that I just got bitten by the dj bug and i that was last year last summer and since then I've done like something like 30 35 live sets i don't know i went off the deep end I've just really loved it since then you know
[00:09:32.637] Kent Bye: Awesome. Well, I know that just recently at Raindance, there was a couple of performances. The one that we're going to be mostly diving into is the Night Under Light seasons at the Moon Pool. And there's also Softly Stuffed Show, The Frictions of a Modulated Soul that I know, Nemo, you worked on that team as well. But I wanted to focus first before we start to dive into going from 2D to 3D. I'd love to get a little bit of a lay of the land of how you start to see the VRChat club scene because there's the venues that have a very specific look and feel. And the thing that I've always really been interested in is the different types of visual effects that the different clubs have. I've seen a lot of playing with lighting and recreating that strobing effects and And I think a lot of the work that VJs that both you Silent and Nemo have been doing is projected onto a 2D screen. And so I'd love to get just a little bit of a background of as you start to go into VRChat, where you saw the visual jockey or VJ, I don't know if it's a disc jockey, if it's visual jockey, if a video jockey, but when you start to look at the visual effects and the immersive quality of it, As you're entering into VR and starting to get a sense of the club scene, I'd love to just kind of take a look back at the evolution of the club scene and where it was when you started in, and then how you started to maybe push the edge for each of your journeys, pushing forward both the visual aspect, but also the music aspects as well. But I'd love to start with the VJ side of things.
[00:10:57.651] Silent: Yeah, I guess start here. So VJ visual jockey is one term people have attributed to it. I'm not exactly sure where VJ comes from, but VJ DJ video somewhere in there. So when I started in 2020 doing visuals, it didn't seem like there was a lot of VJs in the Western world. And we were really just getting our start again. I've heard stories that people have been doing it, but it seems like it was kind of revived in the Western world around 2020. I know in Japan, it was more established. And it started from what I saw as everything was pre-recorded. A lot of like, if a DJ was performing on their own, they had been doing things live. But when DJs started to bring on VJs around 2020, things had to be pre-recorded. Because it didn't seem like there was that many developed workflows for doing stuff live. And at first, it was just like rectangular screens behind the DJ and people making visuals for that. Midway into 2020, I saw this event called Virtual Bass that started to... take the video stream or the videos that were being loaded into the world and start placing them onto unique objects around the world where it would be like someone would take a portion of it and cut it into a circle and put it on flying robots around you. Slyfest was another one that picked it up into a different realm where the first iteration of Slyfest took the video screen and they plastered that video texture on all the textures around you. So there'd be trees and the leaves were made up of video screens and stuff like that. And it was very much in its adolescent period at that point in time. I think it was towards the end of 2020 when I was working with this person, Jazzy, who was creating VRCDN. And we brainstormed this way to do low latency streaming between a DJ to a VJ. So a DJ streams their music to the VJ and the VJ packages that up with live visuals that they're doing on the computer and then sending that to the world. That was a huge success. step up and improvement for the scene. And it opened up this new avenue of being like, oh, now like we could all do this together. We could get the latency down from like 40 seconds to like 12 seconds and kind of make a much more live experience where personally as a VJ, I would hear certain people in the club or these experiences talk about certain memes or words. And I was able to like quickly find that meme in a GIF and put it into the visuals and put it on the screen. And people were like, oh my goodness, wait, I was just talking about that. And I was like, yeah, I know. And I heard that and I put it in. So it started this bigger dialogue. And then 2021 for me was when I saw this huge shift with VRCDN had this event called the VRCDN Showcase. where their world designer, Pameth, recreated a stage from Ultra Music Festival, specifically this Ultra Music Festival stage in Japan, 2014, I believe it was, where the screens were intentionally sliced up in a way that a VJ, like a performance in the real world would have. And it gave VJs this kind of map where it's like, if you put visuals here, it'll appear on these screens. If you put the visuals here, it'll appear on those screens. And so it opened up this like, new chapter of approaching doing visuals in VR as if things were done in larger festivals, which is hard for a new VJ to get your hands on. If you're trying to get into the real world, you don't have the budget to make like a multimillion dollar stage, which is what the stage pretty much was emulating. then more people started to experiment with that. And they're like, OK, now we can put screens everywhere all over the club, and we can slice this up, and then we can start getting more VJs up to speed to be able to perform on these more complex ideas and put on much more grander visual shows. And then I saw towards the end of 2021 and middle of like 2022, the Japanese scene started to do crazier stuff with in-world particle effects, like worlds that come to mind. Cat Club was one experience. Project Cyan, I think it was called, which was another one where you had these... people controlling these visual systems in world with these in-world UIs and controllers and putting on these grander, more otherworldly experiences and starting to really branch out into like, okay, like we're in VR, like we could do much more than what you could do in reality. It took the Western scene a little bit longer to start to grab onto that. And it started a Western scene with doing these things called screen space shaders, where they could start to apply these glowing outlines to people's avatars or put on visual effects that just affect your whole video output that's going into the headset shelter was one of the places that did this with this this new turnout of virtual drugs is what people are calling it these little things that you could toggle on and it usually was very much had the iconography of doing drugs like there's like pills or like tabs of acid techno vinegar was a big one so yeah was techno vinegar was i forget who exactly created it but it was created in the Japanese scene
[00:16:30.837] Namoron: Might be from Ghost Club. I'm not 100% sure though.
[00:16:34.000] Silent: Yeah.
[00:16:34.740] Namoron: Obake.
[00:16:36.202] Silent: Yeah, right. I think it was Obake. So that was fun, but that was very much just like this one toggle baked thing and the experience was very much similar every time you would touch it. There was no person controlling that per se, besides the world creator who designed what effect that was in that moment. And so you just turn it on, a lot of fun, created this new otherworldly experience. And then towards the end of 2022, beginning of 2023, I started to notice people in the Western scene gravitate more towards, okay, let's start doing more things of like controlling visuals and world and creating these like new, you know, extra reality experiences where the one thing that comes to mind is someone by the name of Jerk with this group called Concrete. And he created this visual system called The Creature, which is this... floating, shape-shifting, glowing blob that kind of floats above the crowd. And he has full control onto like modulating the shape of it, splitting it up and fracturing it into all these different shards and putting on all these different textures and effects onto that creature itself. And then they started to implement putting a video player on that creature. So then VJs can now collaborate with the people that are controlling the thing called the creature. That's when I started to see this kind of like, whoa, like as a VJ, we could do much, much more. Now we have like control over like creating like not only controlling this thing within world effects with your hands, but injecting visuals that you're creating on a computer where you have much more freedom and then plastering it onto this little thing that's floating and modulating it in the air. And then I would say at the end of 2023, like around November, December, was when we started to see this huge performance art shift where we're now seeing these more artistic experiences. mainly fueled by the people that run Modular Monday and people like Apple Blossom and Adidas that are creating these new systems of... It's very, very hard to describe, but a VJ injecting 2D video can now influence much, much more. and then adidas and apple's project has come to as like a this particle screen where now vjs can create 3d holograms by just injecting two-dimensional video um and i think that's the most abridged version i could give of what's been happening i think that's a pretty good description of all
[00:19:21.246] Kent Bye: Yeah, that's a great overview that gives a lot more historical context to see the incremental innovations along the way. And we'll certainly be diving into the particle screen and the hologram that Apple Blossom and ADDOS have created. I'm actually going to be speaking to them directly after I finish speaking to each of you here to kind of dig into more of the technical details of all that. But Love to get any reflections, both from Ammo and Ladybug, just kind of reflecting on the clubbing scene and what you were observing both visually and also other vibes of the clubs from how you start to make sense of the scene from the music and genres and everything else.
[00:19:58.345] Ladybug: Yeah, so I don't have such a long historical context or background since I've only been around a year and a half now since early 2023, like January. And yeah, the VJing is... really been amazing and has had exponential growth. I like to say 2024 is the year of the VGA. And it's really cool to see because I have a background in 2D, 3D art. I've worked in the games industry for eight years or so, and I've been doing 2D illustration for longer than that. It's funny how I ended up in the music. Some of the first places I gravitated towards early on was this group called PSHQ. Because they had such amazing cohesive VJ and DJ shows, they would always have a VJ. The shows are very intentional. They had different events. One specifically that I really loved was around the full moon. So once a month around the full moon, they would have these like emotional, more like soft emotional sets with like you know, beautiful visuals. They'd always have a dedicated VJ. And I guess for myself, like joining the rave scene, it wasn't like that. It was more just like going to loud raves where the DJs weren't necessarily playing a story with their set. They weren't playing a narrative. They were just like playing tracks and it was fun and whatever. But then once I started going to these PSHQ events, I noticed that the sets had a message in them. like it was a story the set was telling, or there's some kind of narrative, there's a beginning, a middle, and an end. And that really resonated with me because I've been listening to music for a long time, a lot of sets, been going to a lot of raves, and I really liked that narrative. And also with that audio-visual experience, Light Dimension I mentioned earlier. I started gravitating towards more intentional, emotional style sets. People like Naku and Eulogy, Windward, Soxolotl, Medlish. They all had these beautiful emotional journeys, and that's what really inspired me to DJ in VRChat. And I gravitated towards that very quickly. And that's sort of where I see my strength right now is in eliciting emotions, bringing out like in the moon pool, the night under lights, rain dance set I did was silent. It was about, you know, eliciting emotions of summer and then a sort of shift towards fall at the end. Yeah. In general, I don't know if the DJ scene has changed all that much since I've been a part of it, but certainly the places that I've been gravitating towards is more like this intentional spaces that have like dedicated VJs that put on like a really good audio visual experience. And that's where I've been trying to get into. Yeah.
[00:23:11.212] Namoron: Yeah, I also got in around the same time as Silent, maybe a little bit later. But I think the first big show that I really saw was Slyfest. Silent described what was happening already, like where they put basically screens on all the world textures. And that already was very inspiring. But I really just kind of took in the club vibes for a while and just enjoyed my time before getting into VJing. After Slyfest, there was an afterparty at Loner, which really showed that there's a whole scene to me that I could probably go and check out these events every week. I believe they did have a screen at some point for a while, but they do more lighting stuff now, which is very cool still. And that eventually led to me going to Shelter, which was my main bi-weekly event that I would go to. And they had Kai doing live visuals. And that inspired me quite a bit because... I've always struggled trying to do musical stuff like DJing or making music. And I've always wanted to be a part of it in general, like performance or being able to make art for it. And this, I feel like, was something that really struck a chord with me. And I felt like I could actually make visuals and play them live. Eventually I ran into a friend, Consta, who asked me to make some visuals for their set. I don't remember exactly where it was, but actually I think it was Naku's Nurtured. That was back when I wasn't set up to do live stuff, so I had pre-recorded it. And it was nice getting the first reaction, seeing how people thought about my visuals from the audience without me really being up there on stage. And definitely, I guess, lightened my nerves and worrying about what people thought, because most people won't be saying bad things about the show. It's kind of crazy how big the DJ scene has become. Because I remember going to some meetup with at least pretty much all the ones that I knew of at the time. And now there's like way too many to even mention. Like I've seen so many people VJing at all kinds of events and it's amazing. I love seeing all these new creative people playing their very own visuals or loops combined in some sort of way to make it fit with the music. It's kind of a great medium for expressing yourself and trying to match the vibes of the DJ.
[00:26:06.604] Kent Bye: Yeah, it's interesting to think about different types of both the real-time reactivity that you need in order to have the VJ and the VJ working together. So it sounds like a pretty significant innovation from the VRC CDN to be able to send the audio to the VJ and then the VJ passes it along. So everything's synced up by the time it gets to all the rest of the audience that's able to watch it. And also Silent, as you were talking about the evolution of the scene, I've seen a lot of stuff that is on a 2D screen, the particle systems. I've seen a number of different shows from PK with the break where he will, like he does a lot of shader stuff where he will bring in the shaders, but then go into this whole like trip from world to world, take you on a spatial journey, walking through these different worlds that people have built together. And Candy Trip at Rain Dance had something that was kind of similar where you're having this giant cat that's chasing you and you go through different scenes where there's spatial objects. I feel like part of what the particle screen from Apple Blossom and Adidas, it creates this opportunity to have a little bit more of the abstract visuals that you may be seeing on a 2D screen, but to give it a spatialized dimension so that it also has... that space offset that gives you that sense of like you're experiencing these shapes and also are able to have the lighting effects that can really also start to shift the mood. This kind of real-time lighting dynamic of the moon pool with these particle screens I thought was super evocative. So I guess before we start to dive into The Night Under Light and how that system came about, I'm wondering if there's any other, like, as you start to think about the scene and the clubbing scene, if there's other points of inspiration, like I know Ghost Club is one that often gets mentioned. Shelter, I know there's Sanctum VR that I saw a video where you had a whole mesh-like screen within the context of a spatialized world. So it seems to be like the part of the club experience is not only the club that you've designed in that environment, but also the journey into that environment. And then sometimes like PK and some of his experiences, you're actually being taken on these spatial journeys. So love to hear some other shout outs of other events or experiences that really stuck out to you as something that really points to what's possible for what kind of music experiences you can have in VR.
[00:28:27.856] Namoron: I got a few of these. I similarly named, but I'm not connected at all. Eric's Knoll event where there's a very tall screen and live VRSL lighting. That was a big inspiration seeing like just working at a different angle. And having live lighting was very cool. I feel like that was one of the first times I saw a lighting system in VR, at least to that degree. And 10 EXEs events, she's done quite a few. She goes by Bliss now, I believe. She had many very unique worlds with unique UVs to work with that I believe she even had a 360 screen at some point. Sadly, I never got to see it. But working with a bunch of different UV maps is very nice and can create some new visuals just with the limitations of that map. and Windmill Sequence, which I'm not sure if it's still going on, but that map had six or so UVs to choose from for the VJ, and you could basically have a different set every time you play at the same venue or VJ set. That's the few that come to the top of my head.
[00:29:54.807] Silent: For me, what that gets me thinking a lot about is the different kinds of directions people take to do a sort of spatial storytelling. And one trend I saw for a while was a lot of people were trying to emulate real world spaces and create a sense of reality within VR, especially during COVID when there's a time when we couldn't leave our homes and we couldn't really get out and go to events. A lot of people are trying to recreate that and bring it into VR. And I also saw this other trend where it seems like there's a lot of people trying to create underground clubbing experiences that they thought that maybe they wouldn't be able to go to, whether that be where they live or the kinds of social connections that they have or just feasibility of doing events in certain places. I think a lot about there's a lot of people doing shows that would take place like in VRChat under like highway overpasses or like in tunnels of sorts, or I saw some people start to do events like in like almost like in sewage tunnels and these different like flood overflow tunnels that some cities' infrastructures would have. And people would just be wanting to bring these, they're like, what if we had an event here? What if we had an event in a warehouse? What if we had an event on top of a skyscraper and there was helicopters flying around and people could fly around on them? And there was a lot of people that were, for a long time, people were very much focused on how do we make this look like reality as much as possible? which I think aided the whole scene in growing and learning how to make VRChat worlds and make the lighting really good and make the textures really good. And it kind of let all these people hone in on those skills. And then as well as that, there was another side of the experience where people were trying to create things that you couldn't experience in real life. Things like that that come to mind are like Kaleidosky, like Rue's Kaleidosky, which is like super psychedelic world with a giant screen sphere that surrounds everyone that VJs can perform on. And then Ladybug, I'm struggling to remember who they collaborate with. What's Totsie's world?
[00:32:21.539] Ladybug: Deviate.
[00:32:22.539] Silent: Yeah, Deviate. So yeah, Deviate is, they take a lot of these Raymarked shaders to kind of create these psychedelic kind of experiences, like total enveloping, similar to Collider Sky, but they will also bring in these large 3D created sculptures that would hover above the stage that people can toggle through and then switch the experience around. Other places that come to mind is concrete. Concrete originally just took place in this club that was underground. It was just surrounded by these giant concrete pillars and structures. And at first, for a long time, it felt like a real space. But recently, they released a new version of their world that has this whole gigantic overworld, something like hundreds of kilometers in diameter. It takes like... 20 minutes to go across it with your joystick on NVR chat. It's really huge. And it's this like sand wasteland. It's just like this giant sand dunes covering this vast world. And it takes place like thousands of years in the future, I believe is part of their lore. And it's sprinkled with different artifacts from different clubs and their friends that they sprinkled everywhere. And they've brought their creature to expand to kilometers in the sky. It's very, very incredible to see. And then you mentioned the venue that I work at, Sanctum. We've created this giant cathedral. This is kind of like blending the two of real world feeling in an abstracted way where we now have this giant server room that lives underneath our venue. And it brings you to this giant, what we call the reactor, some kind of nuclear reactor looking thing that we put visuals in within the cylinder. Yeah. And then other places like that are Schism is another one that was created by Thrillseeker. It's this kind of abstracted sci-fi pseudo fantasy world. And you go on through this giant journey to get to the stage. It takes like a few minutes to get all the way to the stage. You're walking through these giant floating spheres that invoke very ominous kind of feelings as you venture through. And then I think the people that really brought together realism, storytelling and kind of surrealism would be Slyfest, the later editions of Slyfest, where the first world was very, very fantastical and very wild and not really grounded so much in reality. And then their next iterations were like an island cruise like experience where like a cruise ship. Yeah. Well, even before Atlantis, just going to Mochi's Island. I guess. Yeah, yeah. Leading up to the event, there was a bunch of events that happened on a cruise ship, and then the cruise ship eventually arrives at this island festival with multiple stages that were very much grounded in reality of sorts. And then that story, like the next years after that, yeah, it turned into Atlantis, where we were being brought to this totally fantastical place with these stages that took place in these large temples that belonged to the Azurians, these characters that they created. And then their last iteration of Slyfest... Definitely pumped that into like high gear where it was like each of the stages were very much not grounded in reality anymore. We were like breathing underwater and watching these music events take place. That's actually when I first saw Nam's visuals was in the underwater temple. I was freaking blown away, but... It's really awesome to see where the scene has been going, where we started finding our roots and finding out the best ways and practices to do things. And the prompt of reality, I think, was really good. And people got really, really good at that and making very tangible places and making places that you really feel immersed in. And now people are really spreading their wings, where it's like, OK, now we have these skills. Now we can start to really bring people to fantastical places and make them feel immersed and almost as if they were real. That's my luck spiel.
[00:36:33.413] Kent Bye: Nice. And Ladybug, I know you said you've been in a little bit less time with last year and a half plus that you've been going to a lot of different events and I'm sure you've been able to see quite a good sampling of all the different clubs seen in VRChat, but I'd love to hear some of your highlights of different spaces or experiences that really stick out for you.
[00:36:53.420] Ladybug: Yeah, definitely. A lot of places were already mentioned. A few that were not mentioned early on when I got in, there's a place called Reflections, which was really unique to me. Reflections was like a cube room. One side was a mirror and then two sides were video screens, like the entire wall. And they had these things called screen space shaders. So basically, it's a shader that's applied to all the characters and also everything you're seeing in your view. And it would draw an outline around people or make people change colors. Also, Redacted, a place called Redacted, I think they were one of the first clubs that did this, this kind of screen space shader thing. where it didn't matter where you looked, but everything was taken over by like a visual aesthetic, almost as if you were looking at things with cell shading or, you know, different colored patterns flashing on your vision or like the characters, all the other people, all the other avatars. Those were really cool, some of my first experiences. Deviate and Collide Sky, definitely classics. I know Collide Sky had screen space shaders. As Simon was mentioning, Deviate had these really cool sculptures that would change above you. And also, speaking of realism and stuff, there's this place called Heist where you're on a plane surrounded by gold bars because you just robbed something that had gold bars, I guess, and you're on a plane. Which is really cool and immersive. No screen, like no screen at all, but it was still so immersive and they'd have like fog and lights and stuff. So yeah, I would definitely agree on that interesting difference between like trying to be really realistic or just like, you know, super fantastical and just seeing like how far can we push this. This VR thing, as was mentioned, concrete, absolutely amazing with this creature thing that was basically a plane that had tons of animated transparencies on it. And then, yeah, you could put a... a video player on it to make all these cool patterns and stuff that could deform shape with blend shapes and all sorts of weird stuff to make unlimited number of combinations. And as I earlier mentioned, light dimension, which is a really cool space where you're basically sitting inside of a torus and you're all on the edge of the inside of this torus looking towards the center where the center column would be. And the creator of it, Les, programmed particle shaders into it that she either wrote or modified. And it would show different patterns of these particles. And there's like something like 40 different patterns as well as screen space shaders. I've had the pleasure of running and operating this particle system before, and it's a huge control panel. It's like tons of buttons. That was also an earlier experience of mine that was just so fantastical. I just love art installations like this. I live in Montreal and here we have the SAT, the Société d'Art et Technologie, where it's basically just like a giant dome. That's another one. Simplicity. They have a giant dome where it's projected onto the inside. But in IRL spaces, the set place, you lay down beanbags on the floor and you look up at projections of whatever they have curated. It's a show they would run several times a month. It's like a set curated show. I love these kind of art installation things that are very common here in Montreal. I'd be walking down the street on the weekend and you'd have like art being projected on the side of a building or there'd be like a little thing on the corner. So when I got into VRChat and I started seeing this stuff, I was like, this is it. And yeah, certainly there's been a progression over the last year and a half I've been in here as well as like an exponential growth of clubs and DJs. Kind of the joke is like the DJ pipeline, the VRChat DJ pipeline. Like you get into VRChat, it's just like, okay, when are you going to become a DJ? It's kind of the joke.
[00:41:19.827] Kent Bye: That's a really great historical context and review. One other thing that came to mind is I saw Jean-Michel Jarre's, he did a performance on New Year's Eve in Notre Dame, and there was a lot of like projection mapped effects on the internals of the church. And so kind of using this conceit of projection mapping, but doing it in a virtual space. So kind of having an augmented reality type of experience within the context of a virtual world. But I think that gives a really great overview of all the different types of trends that you're seeing in the scene. And so I'd love to go back to you, Silent, when you first came across Apple Blossoms and Adidas' hologram system that they're having, which is essentially the best I can describe it succinctly, was that they were trying to find a way of creating a cube-like system space where they're able to control the position of pixels and create fully 3d spatial objects, but to encode it all through the video. So to take the hue from HSV and to encode the positional information, having like six different views of that, and then to decode it and with a shader magically show these objects. And my understanding is that the screen particle effect is a little bit more of like a 2D plane that you are looking at the luminance values to control the height of those pixels. And so you have like a 480p resolution that you're able to then feed in a live VJ input to create spatial objects and lighting effects And so thinking back to the moment where you had the idea and what had already been created from Apple Blossom and ADDoS through this type of system that you wanted to do.
[00:42:58.957] Silent: Yeah. So this is one really, really interesting aspect of VRChat and the ways that we want to do things in that space. And that is that we are very limited into the types of information that we could stream into VRChat. We don't have any ways to stream raw data. We don't have any ways to plug in other outboard systems in order to control things. It's either you hard code it into the world And then we have these video players. There's also other ways to inject information. There's ways to do image loading and string loading, but both of those are very slow. And so the quickest way to get information into VRChat is through the video players, and that's streaming audio and video in. And so we're very limited in that way. And there was this particle system that showed up on this online marketplace called Booth, where a lot of people get avatars or shaders, or you could buy full worlds there. And it's a marketplace for people to buy things to use on their avatars or in their worlds. And there was this particle shader that came out that someone was selling. I don't remember exactly who it was. And what it is, is instead of having a flat video screen, you have this grid of particles representing each pixel on the screen or a summation of pixels on the screen. And the way it works is that when you pipe video through it, the brightness value of each pixel would make the particle leap out of the screen towards the audience. That was the original iteration of the system. Apple and Adidas grabbed onto this idea, and they saw how awesome it was. And that system itself had some limitations. I know Apple wanted to be able to put it on any mesh. They didn't want to be limited to just rectangular screen. They're like, well, what if I could put this on a cube? And so Apple and Adidas worked together to create their own particle system. They broke it down way more, made it much more granular. And they were able to throw it on all these different objects. And they started throwing this event called Modular Monday, which is this world where it has this big screen behind the DJ, all these big pillars around them, and a bunch of these almost like... Dungeons and Dragons dice, all these different platonic solids that people can pick up and interact with. And everything had this particle shader on it. So the pillars had the particles leaping out of the pillars. These cubes and pyramids on the ground that people could pick up and throw around had particles emanating from them as well. And they had real-time lights.
[00:45:36.062] Namoron: That's why you could pick up and throw them around and show that they're real-time.
[00:45:40.103] Silent: Yes. Yes.
[00:45:41.683] Namoron: That's a big part of why they were there, I believe.
[00:45:45.593] Silent: And I had caught wind of it. And Mondays is when they would throw the shows. I'd usually be dead from partying through the weekend. And I would show up, and I never really got much of a chance to put my visuals on them. But it was when they did a collaborative event with Arby the Arbiter, who was the person that put together the Neanderlites project that the three of us worked on. And Nam, you worked on that then as well, right? The one in December?
[00:46:14.210] Namoron: Yeah, I was on the first one for the particle moon pool.
[00:46:17.915] Silent: Yeah. And so it was Moonpool, and it was putting this particle screen into the bottom of what looked like or what was a swimming pool in VR. And what they did was they opened up the particle system to throw these particles much farther than anyone has been throwing them before. So the light values were now affecting them a great distance, and you could toss these particles like two meters into the air with your visuals. And I recognized, personally, I was like, oh, I can make a system where I could use a 3D program I use called Notch, and I could point a camera at objects, and I could put a light with an attenuation distance right next to the camera. And then any 3D objects that I have in front of this camera will, the closer it is, the camera will be brighter, and the farther away from the camera, it will be darker. And this will essentially let me play with these particles in a 3D way. And so I was able to make these particles create these kind of 3D shapes, almost as if you had a piece of fabric, like stretched fabric, and you took a cube on its point, and you just pushed it through the fabric. You could start to see the semblance of... the 3D nature of these objects. And I came up with this idea of like, oh, it would be really cool to create like a 3D hologram with it. But unbeknownst to me, Apple and Adidas had already been having this idea. Like the moon pool was their prototype for creating a 3D hologram system. And so months before I thought of it, they were already working on it. And so their concept was we can take this particle screen, and we can make a large cube that people are standing within. Or we could just have this cube where we put the particle system on each side of this cube. And then now we can really create 3D objects. In 3D software, you have six cameras doing this similar thing as to what I was doing, where the cameras can collect both depth and color data and convert that into two different channels of color because of the limited nature of the data we can stream into VR. And so the depth information was represented by the green channel. The hue was represented by the red channel. And then the brightness was represented by the blue channel. And then with the 3D software tuned properly, now you have this box of cameras. You pretty much have a box, all these cameras pointing in towards the box. And then now whatever you put within that box is going to show up. in VR as this floating hologram particle. And that has really skyrocketed us to a whole new realm of VJing. It's hard for me to call it VJing nowadays because, to me, a video jockey meshes video sequences together to create an experience on a screen. This is using video to create three holograms, and I'm not sure that's VJing anymore.
[00:49:25.591] Namoron: It's a completely different experience. I had to make a whole new project. and learn TouchDesigner as well because not all VJ software would even support the ability to do this stuff.
[00:49:38.988] Kent Bye: So it sounds like that in order to actually get the translation, you have to, because you're creating the 2D, you're using Notch as a visual effects system. And is it outputting a video that is then kind of like encoding directly the video? Or is there another middleware that is somehow properly encoding everything? Or is it just how you kind of have to set up your project in Notch is to kind of like... say, okay, this positional value of the height is gonna be written to the luminance value and then you are constructing this video version of it that's being fed out. Is that something that you had to create with a notch to be able to get the proper output and then feed directly into VRChat and then have it decoded?
[00:50:16.792] Silent: Yeah, correct. So in Notch, I was able to create each of these cameras create their own flat 2D video. And so I have these six videos. And this is the same with TouchDesigner. And then you take these six video feeds. And then within Notch itself as well, you stitch them together in the correct orientation. where it's just like if you look at a 16 by 9 rectangle, it's just three images on the top and three images on the bottom, all representing the different sides of the object that you're projecting into the system. And so yeah, we're just really not jankily, but kind of janky. It's like we're doing something that's really never had been a problem ever before, where no one's needed to come up with this kind of solution. But you're pretty much taking these six camera feeds, and then you're stitching them together in a way that appears on a flat rectangle that then we could stream that video into VRChat.
[00:51:13.732] Kent Bye: Yeah, I was talking to Apple Blossom and asking her, why don't you just use JSON data? Wouldn't that be more efficient than having this encoding? And she said, well, for one, you can't really get real-time JSON data into VRChat. And for two, it's actually way more efficient to encode it into the video stream because you have some information that's lost from the video compression, but essentially that it was able to more quickly get the data in and out than you would be able to if you were just doing it with the raw data itself, which I thought was really quite fascinating, that this is actually kind of an efficient way, but even though it doesn't sound like intuitive that you would use the video file to do that. But it sounds like at the end of the day, you're able to create a workflow so that you as a VJ can take a normal workflow, maybe have some special tweaks, but then all of a sudden have the output create these amazing spatial effects dimensional experiences that have not only the 3D space of this pixel grid, but also the lighting effects that start to have the impact, but also not only that, but have it from the bottom and the top. And so you have this kind of dual nature of you're being surrounded by these two spatial dances that are happening both above you and below you. So...
[00:52:27.706] Silent: Yes. I definitely went on the tangent of the hologram cube side of things. Going back to this kind of moon pool experience that we did, it was very analogous to normal VJ, where it's not this necessarily encoding stuff to HSV. where height means something. Like, height is on one channel, and the color is on another channel. But the moon pool is very much just the luminance of the pixel is how much it's going to pop out at you. And so that's a much more simpler way where, yeah, it's very analogous to what we do normally. So I don't have to use notch for the moon pool. Sorry, I definitely went on a tangent about the hologram cubics. That makes me so excited.
[00:53:09.748] Kent Bye: Just to clarify for listeners, because I had a big, long technical discussion. So the hologram, it has the six views that get decoded into a spatial object, whereas the particle screen is more just the luminance values of the video that then get translated into height, which is like a pixel grid. Correct.
[00:53:28.055] Silent: Yeah, and then that opens up with the hologram cube. It takes a lot of wrapping your head around different 3D softwares to create that kind of system, whereas this particle screen is very much like anyone who has video knowledge, like literally anyone.
[00:53:43.941] Namoron: You go, Nam. Any VJ. You could put in your current loops that you already have and have some stuff look cool. Not everything will look great on the screen. Faces get a little distorted. you could probably get someone who's been VJing for a while to just put their video stream in and have some cool effects. And I think once you hone in some of them to look better on the screen, you can pretty easily be ready to go on it.
[00:54:14.525] Kent Bye: I said, so name, it sounds like that you have a little bit different workflow. I saw that you're using magic music visuals and also touch designer, whereas silent sounds like she's using the notch. So maybe just talk a little bit more about you discovering the system and then use your own workflow to be able to design these DJ experiences that were shown at the moon pool.
[00:54:35.672] Namoron: Mm-hmm. At first, Silent mentioned the original asset that we first found was a particle screen on Booth that was pretty limited. I've been to a couple of events that used them, and I was asked to VGA at one. And that was the first time I had messed with that idea. And I realized that my current visuals, while they did work on it, I thought you could really improve them. And I find that simpler shapes really stand out, and gradients really make a nice, smooth texture, or I guess a nice, smooth 3D effect. And I believe Apple and EOS were there to see it and probably got a little interested and inspired by it. I don't want to speak for them, but it seems that way. And they messed around with that screen and then made the Moon Pool and the Modular Monday 2D or any mesh-capable screen, as Silent was mentioning. And Magic Music Visuals is more suited for 2D stuff. It has some 3D capabilities, but it's pretty limited in that way. It mostly just takes in a model, and you can do 2D effects to it or rotate it about. So I was much more limited in not being able to make a space like Silent did. At least I would have to have like drawn it in, I guess, or with a texture instead of with 3D objects.
[00:56:18.586] Kent Bye: Yeah, there was a different feeling of like, it felt like in Silent's experience, it was like, I felt like you had created these kind of like proto, almost like a pixel art, but in 3D where you're able to create like these scenes with like vegetation and... and kind of objects that were there. Whereas I think I am, I think yours was a little bit more abstract with more like big, broad sweeping shapes. And I was really noticing the colors and really reminded of the light and space movement, James Terrell, that I saw a piece called Baku at super blue Miami, just to see how much of the experience was the lighting effects of having these pixels create this whole mood and vibe to this kind of sweeping abstractions of color modulation and Yeah, it was just really transfixing to see that. But there's different styles that you each had that I think kind of speaks to more maybe the workflow of having the ability to have more control over those kind of spatial objects with Notch versus the more sweeping, algorithmically driven shapes and curves. And I know the language you kind of described, it's like more generative and mathematical in some ways.
[00:57:29.409] Namoron: Yeah. It's also a node-based program and it takes in a bunch of ISF shaders. And I usually make my base stuff in Illustrator. So I made the flowers with... It's just a 2D SVG that I converted to PNG. And I added gradients and other effects to it to make it look good on a screen while working in the world for multiple hours. You could get stuff looking good real quick, but it takes a while to really hone it in.
[00:58:06.061] Kent Bye: Yeah. And Ladybug, I wanted to bring you in because I know that you were designing kind of the musical set that was opening up the Night Under Lights, the seasons at the Moonpool project that we were talking about here. So as you're seeing what's possible with the different visuals, I'm wondering what the process was like for you to design the set. And then if there was any back and forth for how you wanted the different visuals to have this nature feel, or if you kind of just passed over the audio and let silent do all of her magic, but
[00:58:35.464] Ladybug: know just maybe talk about your introduction into this and your process of creating the first set yeah so it wasn't the first time i worked with silent uh we had done a previous night under lights before together so we had some experience working on projects so my the part of the set was summer into fall so a couple weeks before the event the first showing we had We just sat down in a call and had like a really long, a nice discussion about like, how do we feel about summer? What are the things we, you know, the memories that we have, we're both from New England, actually. So we had a lot of similar experiences and driving around as kids in Connecticut and, you know, driving to the ocean and how the summer felt to us. And as we were talking, Silent was just sketching out with like colored pencils, a lot of different ideas she was having. And on my side, I was like throwing her tracks in Discord and we were like listening to them together. I was like, I guess I was streaming my DJ software and we were just like, Oh yeah, this sounds great. And like this song reminds me of this feeling. And at this time in my life, when I was a kid in summer, So we had like a nice back and forth and then it was a couple of weeks before the event. And then we just kind of let those ideas sit and simmer and like stew. And I think it wasn't until a week or two just before the event that Silent got to hear the draft of my set. And it was like, yes, that's exactly what I was thinking, you know? And she took that and then built out all of her visuals from it that had been like kind of swirling around. I'm putting words in your mouth. I'm sorry, but this is sort of what we talked about already.
[01:00:29.827] Silent: Yeah, you're getting a direct read of me.
[01:00:31.088] Ladybug: You're good. Yeah, so it was easy for her to put all those into the program because we had already talked about it and she was kind of putting them technically together like over that time. And for me, over that roughly a month before the event when we talked about I started collecting more music, even more music than I showed silent. And, you know, the music that spoke to me, I kept and what didn't I discarded and made a beautiful journey out of it that I'm personally pretty proud of. And that can take that one. Actually, I was a bit inspired once I got all the tracks together. It doesn't take long to build the set itself, but getting the mediums together right once I have all my tools laid out, I could just be like, Oh, this, this, this, this, this. you know, boom, boom, boom. And it was only like two days and I just threw it together. I wanted to just do it fast. I didn't want to think about it too much. I just wanted to be like, okay, like what comes out of this without thinking about it too much? You know, I wanted it to be really like feeling driven instead of like, Well, I want this part and this part and this part. So it only took me about two days once I had all the songs I wanted together. And then I kind of refined it, you know, little tweaks here and there over the two weeks before the show.
[01:01:52.714] Kent Bye: And so I know that there's like the audio link real time, like audio reactive shader that's in VRChat that's looking at a number of different factors of listening to the music and doing some real time effects. And so I was actually surprised at the end of the show to hear that the VGA was actually happening live because typically with these different types of shows, a lot of stuff that I've seen that's on this scope and scale has all been prerecorded. So I was like a little bit shocked to hear like, oh, wow, this is all happening live. But in that liveness, is there any ability to feed into these VJ software programs like Notch or MMV that you're able to do like real-time audio reactive effects? Or is it more that you kind of have a set of things that you are doing that then you're trying to piece together to create an overall composite of the experience rather than have any information that's directly coming from the music?
[01:02:45.444] Namoron: For MMV, we do have a direct, like, we can take audio in from the DJ stream, which I had Turquoise, just as a shout out, Turquoise Music on Twitter. He has all his links. He did a great job. And he, or Magic Music Visuals. Takes an audio, and then I can set different shaders to react to it as I want. Or I can adjust it live, the amount that it even takes in. Sometimes I had it not using any of the audio, and I was doing everything manually. Other times, I had my shaders reacting directly to the base or something. Yeah. Silent.
[01:03:32.938] Silent: On the next side of things, it does have those capabilities of taking audio and injecting it so that the audio affects visuals. But as I've grown as a VJ, I've been developing a career doing real-life VJing. And an interesting component of doing visuals in real life is there's a lot of latency between the visualist and the screens behind a DJ. There's no real good way to do audio reactive in the real world. And so I've tried to stray away from that because I kind of use the VR space to practice for the real world stuff and I don't want to get into bad practices. And so personally, I don't do audio reactive visuals. I just do them by hand. And so like when there's the one scene that from in Ladybug set where I had this wheat field kind of grow and spawn and like is kind of just like static and then the beat kicks in and then i have like a slider i called it my energy slider and i just slide that up and now it's gonna just make the wheat kind of roll and toss like in the wind as the music increased and so there are ways for us vjs to do that in vr because we don't have to worry about latency so much because we're just combining the visuals and audio directly within the box in this very sterile environment that allows us to do so. I believe Arby did so as well. Arby had, who did their set with Yuna, had audio reactive visuals as well.
[01:05:05.974] Namoron: Arby also uses magic music visuals. This is a note.
[01:05:09.758] Kent Bye: Gotcha. Yeah, and just to kind of elaborate on that point is that because you're using the VRCDN, that is essentially sending the audio to the VJ, the VJ sending the synced version of both the video and audio. From the DJ's perspective, there might be a little bit more latency for what's happening in the world, but from a VJ perspective, it's actually a little bit more optimized to do that type of audio reactive experimentation that you can't do in the physical reality because of the latency. And because it is on video, you can record it and there can be recordings of these performances in a way that I think it'll be really exciting once the recording of the moon pool is made available for folks to go see the seasons show so that they can hear the music and see the visuals that went along with it. You know, cause there's a video that's online that RB had put up on YouTube, but it's a lot different than actually being there and seeing both the, the volumetric experience from both the bottom and the top. So looking forward for folks to be able to go actually go check it out. So. Yeah, I guess as we start to wrap up, I'd love to hear from each of you what you think the ultimate potential of virtual reality might be and what it might be able to enable.
[01:06:14.776] Silent: I will say a lot of the stuff I did not see coming, like this hologram cube and the particle pool. Every iteration of VR, every new big thing catches me off guard. And so what makes me excited about this medium is it's going to be more and more fantastical as time goes on. And we're already at such an awesome place that we are having these unreal experiences that are just so otherworldly. and then giving all these new talented artists so much control to craft these experiences. Some of the things I think about a lot is it seems like what we're doing here in VR is a lot of training for when things become more augmented reality. where you could put on some glasses or headset in a real-world club environment and have these kinds of experiences happening around you, but superimpose onto reality. That's where I see a lot of this direction going, is creating much more immersive experiences at physical venues without having to, like in Max Cooper's journey, suspending particles all over the ceiling and putting a lot of money into the... actual technical clustering LED screens everywhere, you know, having these grids of LEDs hanging from ceilings in the real world instead of having to throw so much money into that. Eventually, I think we'll get to a point where augmented reality is so in the norm that every place will start to kind of have all of these things. And I like to see sometimes or like to think that what we're doing here is really pioneering like what will become that kind of AR experiences. but nothing will necessarily beat the kind of amazing experiences that we could have in VR because not only are we able to do that right now, but we are able to do it with people all over the world and collaborate with artists that are not just based on our locality, but we could collaborate with artists literally everywhere. So I'm excited. Things are going to get really weird and really fun going forward.
[01:08:20.498] Kent Bye: Indeed.
[01:08:22.078] Namoron: I think one really big thing that already is happening is that it's a new venue for people that even get into VJing or DJing and that they probably wouldn't have been ever really had the chance to do it in real life. VR definitely opened that up to me. I probably would have never VJed if I didn't come check out VRChat. And I am not entirely sure what the future of the shows in VRChat will be, but I feel like after seeing the hologram and working on it with Apple, well, she did all the hard work. I am excited to see like the future of like shaders in worlds. I think that there's still a lot of potential that just someone needs to think up some sort of interesting new idea that hasn't really been done in your life. I don't know exactly what it would be, but one day I'm sure there's going to be some cool things.
[01:09:26.946] Ladybug: Yeah, it's so interesting. I talked about this with friends before, but we were just commenting on how like never before has there been so many creative people all together in the same place with like from all over the world with literal instantaneous access to just contact one another and collaborate and create new things. And yeah, seeing the 3D sort of voxel cube thing that was created from SNR Labs and Apple and Adidas. That really kind of stretched my mind and was like, wow, like the limit is really our imagination. And also the software though, VRChat using like an Udon modified version of Unity is certainly a limitation. And I feel like the future of VR will depend on the systems that are built that can be utilized stuff that's more flexible stuff that is more maybe audio visual experience focused or it could be more like silent was saying like more augmented reality where you're all in the same physical space but you all have headsets on that are showing you like visuals inside of a otherwise dark room or something like that But yeah, like Nan was saying, it's so accessible for anyone to come and create. And that was also what inspired me greatly to be able to share and contribute because I never saw myself teaching IRL. But like in VRChat, I mean, I can just stay at home and, you know, be able to play some music for people. It's so accessible for people.
[01:11:09.352] Kent Bye: So yeah, I'm wondering if there's anything else that's left unsaid that you'd like to say to the broader immersive community?
[01:11:15.628] Silent: uh i mean yeah i like we have the best trained artists in vr right now we're the best djs we have the best artists we are innovating like no other we're being able to create these crazy spaces and have access to them and i have seen some performances and i've been able to experience some things in the real world and nothing's really coming close incredibly so and i'm really grateful that you're talking to us today because I think that a brighter light needs to be shined on the kinds of things that people are doing in these spaces because it's truly incredible and a lot of it transfers to IRL experiences if people like us have are given a chance but I really think that we could be blowing away the entire immersive scene both in VR and out of VR very soon
[01:12:05.740] Ladybug: yeah it's so cool like vr is so cool and i've heard other people say this as silo was saying like it's all so new it's also experimental and like i've heard this vr chat being called the wild west you know of vr right now and other friends of mine have noted like this needs to be recorded like Someone needs to be documenting this, what's happening, this crazy, like, hurricane of creativity going on in right now. And I would agree with that. So I thank you, Kent, for doing stuff like this, interviewing some of us from the scene. And, you know, there's a lot of people doing interviews in VRChat on YouTube as well that are documenting this. I just wanted to shout out a few more artists that really inspired me, DJ specifically. I remember Endless Serenity was a world that I had first seen the particle screen. I think that had the original particle screen from Booth. And Turquoise, the DJ that went last for the Raindance Night Under Lights, was super inspirational for me, playing a lot of ambient and wave. Yuna, who played after me, between me and Turquoise, also extremely inspirational for me as a DJ. Juno, English Lad, Ru, Kitsune Mixes, Giga Strawberry, DJ Jigsaw and Anx also, as well as I had mentioned previously, all super, super informed me and formed me as a DJ. Ru was actually one of the first people to give me some frontline DJ lessons. Yeah. So I'm just super excited with where things are going. And like Ken mentioned, like the new software, what's going to come after VRChat or maybe like VRChat 2.0, like how they're going to be able to open up Unity more to devs to create more immersive audio visual experiences.
[01:14:09.420] Namoron: I just hope that the community stays as open and as welcoming as it has been to new people DJing. It stays that way, at least to a certain extent, and lets people open up and show off their creative side. I think it's already been doing a great job of that, and I hope it continues for many years to come.
[01:14:31.286] Kent Bye: Awesome. Well, SilentNam and Ladybug, thanks so much for joining me today to give a real deep history and context to what's been happening with the clubbing scene there in VRChat and to give a little bit more context, this particle screen effects and the evolution into the more 3D spatial effects of a VJ. And maybe there's a new word that needs to be invented to have like a hologram DJ or HJ. I don't know what the word's going to be. It's going to be... Particle jockey. Particle jockey.
[01:14:58.940] Silent: I've been trying to coin the term Pixelmancer. Yeah, I want that to pick up.
[01:15:06.126] Kent Bye: The PM. So yeah, just manipulating pixels through space and time. And yeah, I think there's kind of like a new genre of that, that I feel like the tools that I'm seeing here are starting to do something that's distinctly different than what I've seen before. So just really appreciated hearing a lot more context for each of your journeys into the space and what's happening in the clubbing scene and Yeah, after I saw the piece, I was just really blown away. It was one of the most amazing immersive experiences I've ever had in VR. And I'm really excited to see where other DJs and VJs are able to take this type of platform here in the future. So thanks so much for joining me to help break it all down. Thank you. Thank you so much, Kent.
[01:15:44.873] Silent: Yeah, thanks, Kent.
[01:15:45.993] Kent Bye: So that was Silent. She's a VJ, event producer, and immersive artist in VRChat for the past four years. And then Neymaran, who's been VJing in VRChat for the past couple of years, but also first got in back in 2017. And then Ladybug, who's been DJing in VRChat for the past year. So of a number of different takeaways about this interview is that, first of all, Well, the Night Under the Lights, the seasons at the Moon Pool is one of the most immersive music experiences that I've had in VRChat. And I just think it's completely magical what they're able to do. Not only because they're able to have these kind of audio reactive effects, which for me personally is the things that I really resonate the most with because you kind of see this multimodal representation of visuals that are very much in sync with the music. But even if it's not completely audio reactive like what Silent is doing, she's been able to go on completely other different tracks some of the stuff that silent's doing is actually more like 3d pixel art in a way that is able to kind of have the low res pixels of these objects that are very subtle especially at the very beginning of her set that she was playing with ladybug and so it basically can take any type of video input and then it gets translated into these different shapes and as they recount it does take a little bit of like getting used to But in the Night Under the Lights, the season's replay at the moon pool, there is an ability for you to actually put in like a video feed and for you to actually use that world to kind of experiment around. So highly recommend for anybody who's an aspiring VJ or just to kind of like see what this video or this image looks like. It turns out that you can have very simple letters on a screen and the colors will start to dictate these different shapes. So yeah, there's actually a number of different sets that are recorded and available for you to rewatch there. So highly recommend going to check that out just to get a little bit more of an embodied experience of what we're talking about here. Because I do think there's actually been a bit of a turn and some of the stuff that I've been personally interested in, in terms of like going to the clubbing scene and just to see like, what are the different things that you can do in VR that you can't do in physical reality? And that's a lot of what Silent was saying is that. vr is really at the bleeding edge of pushing the types of visual effects that maybe at some day they'll have like augmented reality analogs for some of these different types of experiences but some of the stuff you could only really do within the context of vr at least at this point because you know you're able to do this kind of holographic you know it's such a huge scale of these you know concrete at pale sands which i did actually have a chance to go check out after i did this interview because it's going to be featured at then it's immersive and so I had a chance to go meet May and Jerk and to see this really vast sand dunes. In that space, they have the sand dunes that have a mesh that's attached to it, so they're able to either directly put screen-based content onto the sand dunes, more like what it would look like a projection map onto the sand dunes. But there's also this effect of, you know, having a 2D plane that's floating in this kind of holographic cube. And then from there, the lighting effects are having an impact on the sand dunes. And so you have this real sense of presence that this is actual hologram that's floating in the sky and that the lighting effects from that hologram are impacting the world around it. And it's at such a huge scale, you're overlooking this like many kilometers wide of space of these sand dunes it just gives you the sense of vastness and awe and wonder that was again one of the more transfixing experiences that i've had in vr concrete at pale sands it's going to be featured at venice immersive as well as the sanctum which was a place that silent also took me on a tour of to see a little bit more of a 2d billboarded effect so doing like stereoscopic 3d effects within the context of this volumetric space so there's the cylinder that's this reactor that has again the lighting effects are impacting the world around it so it gives you this sense of environmental presence and there's also other lighting effects that are happening that the vj is able to control and so this combination of lighting effects with holographic and 3d effects and then the snr labs test facility which adidas and apple blossom took me on a tour of which again is more of the 3d spatial holograms that they're able to create these floating hologram objects that is really transfixing And so, yeah, it was just really fascinating to hear a little bit more about the history and evolution, starting with this kind of photorealism and recreating it, focusing a lot on the lighting effects. There's a whole other lighting effects systems that are out there that we really didn't dive into. That's a whole other conversation just to talk about the evolution of different lighting systems, but focusing more on the kind of VJs and the visual effects that we're seeing within the context of these projects. different platforms, starting with screen-based effects, putting those screen-based effects onto objects, and then volumetric shaders that are happening within the context of these rooms, and then having screen-based shaders that are impacting, you know, kind of like augmented reality filters that people are seeing. And then even taking people on these kind of more immersive stories where they're able to create a whole virtual immersive adventure where you're going from different world to world and scene to scene. And so it becomes more of a a story that's being told as you're listening to the music and a lot of stuff that PK has been doing with the break series that have covered previously here on the voices of your podcast. But yeah, just a lot of shout outs to different spaces that are out there from dimension and virtual base event and shelter and ghost club and concrete with both their, normal location but also with concrete pale sands pshq there's sly fest loner kaleidosky sanctum schism deviate atlantis mochi's island reflections redacted heist light dimensions simplicity and endless serenity were some of the different clubs that were mentioned here throughout the course of this conversation so you can definitely go check out some of those There's actually a website called VRC.TL. That's the timeline of all these different events. And there's actually a listing of like 369 clubs. I think when I last counted a couple of days ago, they'll have like links to the discord and the Twitter and basically get more information for, you know, how you can get a part of the VRC groups that are there. So when there's group instances, you can join in or just keep track of what's happening on the VRC.TL just to see some of the different events that are coming up. Yeah, just so you can start to see some of these different performances of what's happening in the VRChat clubbing scene. So yeah, just really a great opportunity to catch up with Silent, Neymaron, and Ladybug and to hear a little bit more about what's been happening in the scene and to really highlight what I think is one of the more exciting things is kind of turn towards this holographic effects just because it's like more real time. A lot of the stuff that we've been talking about up to this point has been more like you have to do it ahead of time, you have to pre-code it, and it's just a lot of work ahead of the actual performance stage. That's not to say that the VJs are not doing a lot of prep work to create their sets and everything, but it's just the fact that they could do a live stream of a video that have their own existing processes where they're able to really focus on their pipeline for either using MMV, which is the Magic Music Visuals. There's Notch that silently uses, and then there's TouchDesigner, which in my conversation with Apple Blossom and Adidas, they were talking about how they wanted to make it so that people could use the free version of TouchDesigner and that you wouldn't have to be forced to buy the full version. So you can actually download TouchDesigner, get a OpenStream Labs stream going, and there's somehow to take a URL and put it into VRChat and start to live stream into VRChat and kind of experiment with some of these different systems. So anyway, that's all I have for today. And I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoyed the podcast, then please do spread the word, tell your friends and consider becoming a member of the Patreon. This is a list of support podcast. And so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voices of VR. Thanks for listening.
