I talk with the founder of the FIVARS VR & AR story festival Keram Malicki-Sánchez & WebXR developer James Baicoianu to talk about how the repurposed the open source JanusXR code in order to create a platform to deliver 360 video. It’s a pioneering effort to push the technology forward this way, and they share details about the struggles of their journey and why they see it as import to create a sovereign platform. The festival itself has an independent, avant-garde spirit, and that’s reflected in how they’re also trying to push the technology forward to show folks what’s possible with WebXR.
You can read more thoughts that I have on the FIVARS festival in this Twitter thread:
The @fivarstories 360-degree video fest runs until November 2nd.
It's built using #WebXR & @janusxr, and you can watch videos using Chrome + PCVR.
The curator @kmalickisanchez wrote up an overview & argument for why indie fests like FIVARS are important: https://t.co/E40Kd0IhTy pic.twitter.com/vPWzDMtx0H— Kent Bye (Voices of VR) (@kentbye) October 27, 2020
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So the 5Rs Festival of International Augmented and Virtual Reality Stories started way back in 2015 by Karim Lutzky-Sanchez. And he created this in order to create this kind of avant-garde indie alternative platform to be able to feature what was happening in both the 360 video medium, but as well as these interactive 6DOF experiences. Kind of like this real independent spirit of curation of seeing what kind of immersive storytelling was happening within the medium. So the festival's been going for six years now, and this year with the pandemic, Karim actually brought in James Bequiano, formerly the principal engineer of Janus VR, which shut down last year. Now it's an open source project. And so he was taking this framework and expanding it into this very specific use case of having a WebXR festival. So you can go into this through a web browser, go into this space that has a portal into all these 360 videos. And so you can be in the VR, like I did this week to just go in for many hours at a time, go into the single VR space, open up a 360 video, watch it and go back out into the space and watch another one. Which I think is a big part of what the Museum of Other Realities started to do this year with trying to recreate that festival experience and now we're moving into the web browser to be able to see what you can do at least with 360 video for now. So in this conversation, we spent about the first 20 minutes or so just catching up on 5Rs and the curatorial vision that Karim has. And then for the last 50 minutes, we go into all a different journey of what it means to be able to try to use WebXR to be able to launch something like this and what they were able to achieve and what still needs to be done in terms of the overall making a streamlined experience for folks to be able to create experiences within WebXR. So there's certainly a lot of really interesting innovations just from the festival itself, but the larger point is that they're trying to create this sovereign platform using open standards and to create something that's a different alternative to the closed walled garden ecosystems that we have either with Unity or Unreal Engine, or whether it's through the Oculus Store or Steam or whatever else, you know, what are the ways to be able to get content out to the people without having to have some sort of gatekeeper? And then even when you try to do that, there's still ways in which the browser vendors can be a sort of gatekeeper in that sense as well. So we'll be talking about all those dimensions as well. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Karim and James happened on Wednesday, October 28th, 2020. So with that, let's go ahead and dive right in.
[00:02:42.156] James Baicoianu: So I'm James Bekuyanu. I'm a VR developer, mainly focused on WebVR, WebXR. I was the principal engineer at GenesVR for about three years until the company closed down last year. Since then, I've taken stewardship of all the projects we released as open source. I'm kind of keeping that all running. Yeah.
[00:03:05.177] Keram Malicki-Sánchez: I'm Karam Malitsky-Sanchez. I am the founder and executive director of the Five Hours Festival of International Virtual and Augmented Reality Stories and also of the VRTO World Conference and Expo.
[00:03:21.580] Kent Bye: Cool. So Karim, let's maybe start with a bit of the backstory for 5Rs and just give it a bit more context as your journey as to starting this festival for augmented and virtual reality stories that it's based in Canada. So maybe you could just give a bit of a context, the history and where we're at today with 5Rs.
[00:03:39.861] Keram Malicki-Sánchez: Sure. I started it in 2015. I started it the same week that the Toronto International Film Festival happened. It grew from a VR meetup that I'd launched in Toronto that summer. and I did not know what to call it at first. Eventually we came down to this perfect acronym because it sort of sounds like five, you know, and they were kind of about these shorts that were starting to come together. At that time things like the IVRPA were out there for panoramic photographers and people were starting to do video that was stitched together with these cameras, and it was a very expensive and laborious process, mainly for enthusiasts. But I had been attending the Toronto Film Festival since I was a kid, and I really modeled a lot of the core ideas around that, which was a festival that started as a grassroots group to look at international cinema. Also, I was reviewing games with my blog, IndieGameReviewer.com. I had five or six writers, and I was going to the IndieCade Festival every year, which was this small group of indie game developers who were telling subversive, transgressive stories through this unchecked medium. And those two things were conspiring together for me to address something that was emerging. It was a new underground channel for discourse, for telling stories that were outside the mainstream. I was no new person to interactive transmedia stuff. I'd been working in that media for two decades. In the mid-90s, my band released a CD that had a sort of missed style world that you could travel around in. And all the data was placed on the zero index of the CD. So unless you put it into a PC, you wouldn't know that it had an interactive CD-ROM game on it. But I loved playing with these ideas and kind of secret codes that existed in hidden places. And 5Rs was decidedly about story, narrative. It wasn't storytelling, because storytelling implied that there was some sort of voice that guided you through it, but rather that a spatialized, embodied media had its own sort of narrative that we needed to discover. And so that very first year, we launched it, and it took place a block away from the Toronto International Film Festival's box office. That at that point was the biggest film festival in the world. They would close off six city blocks. There was red carpets and every celebrity you could imagine would walk that carpet and it was all about the parties and the champagne and the limos and the deals and the whole thing was getting bigger and bigger. Kind of very far away from what it used to be. in the old days when you would sit in these back alleys waiting to grab a waitlist ticket, you know, and get miracled into some sort of amazing Thai or Finnish or Colombian film. So we wanted to be kind of the avant festival happening right underneath it, just like Slamdance was to Sundance, which itself was once a small, independent festival. And that first year, we did not know what to charge. We charged $10 a ticket. There was something like 30 experiences. You could come in and play interactive things or watch 360 films. And we had pieces from Felix and Paul. We had a piece from Kamchatka. We had a piece from Adam Kosko, his debut 360 piece. And it was this hodgepodge of ideas, and people did not know what to make of it. And yet, at the same time, they were completely enthralled. And that was the beginning. That was the beginning of Five Hours. I even named our website 5Rs.net. You know, people don't usually go for the .net, but it was because there was tiff.net, and I really imagined that one day we would become a part of TIFF. That was my big dream. And I will note that we did talk to the principals of TIFF twice in person. They knew very well who we were. In fact, one year they launched something called POP. which was cool. It was cool to have it, but I said, you know, we know what we're doing here. Let us take care of this. This is an extra thing for you. And in the end, they decided that they were not going to do VR. In fact, about a year ago, they said, we love what you're doing. We think it's tremendous. We're not going in that direction. I said, wow, all the other major film festivals are, and you, who are the biggest, are not going to go this way. And at first I was really bummed out because I had set everything up for this. And then I realized, all right, then we are that festival that needs to be there. And so unlike the VR part of these festivals that is usually attached to an existing film festival, we stand sort of on our own as a VR, AR festival on its own right. There's nothing attached to it. It lives and dies by its content.
[00:09:13.889] Kent Bye: Yeah. So that's the, I guess the, the seeds and the origin back in 2015 and that you've been consistently doing this each year and it's 2020 now. So this is your sixth festival that you've put on now. Yeah. And I'd say that as I go to different conferences, there's a bit of a cycle that usually begins in January with Sundance and then. South by Southwest or March and then Tribeca and then Venice. And then, you know, for me, I would sometimes go to other festivals, but not always make it to all the other festivals that are happening. So I've never been able to make it to five hours, but I'm so glad I get a chance to see some of these different festivals and experiences. I mean, obviously the pandemic's a huge pain in the ass for everybody in the world. Yeah. But the upside, at least for the virtual space, is that I could watch all of this in the comfort of my home. And so I was able to see almost all the program that you curated. And I have to say that your curatorial vision is distinctly different than anything else that's out there. I mean, it's probably the closest to if a doc lab in terms of the experimental avant-garde, that tends to be a little bit more of the location based entertainment, documentary style, you know, things like the collider or immersive experiences on top of documentary, some 360 video, but for this festival, I think is distinct because there is so much 360 video, which almost in every other festival, it's almost like the forgotten stepchild, or, you know, it's just something that is not a medium that has really been cultivated in a way that I think your festival is trying to really take a much stronger curatorial vision for the 360 degree video. I mean, you have some interactive pieces as well, but I think the thing that sticks out to me at least as I go through your program is the diversity and the breadth of different types of 360 video. So maybe you could catch me up over your vision over the last six years that you've been doing this for what you were trying to do in the 360 video space.
[00:11:06.107] Keram Malicki-Sánchez: Sure. I think one of the most interesting parts about running 5Rs was that I always felt like I was paying for the education of the public. Every time I would hire a publicist and they wanted to charge me these enormous fees that would take up the majority of my budget, I would try to explain to them, I'm going to be paying you to learn about how to talk about this media and then you're going to be teaching the journalists how to talk about this media and I'm paying for all of this and this was the feeling that I had that I was paying to educate and create the bridge that did not exist between the public and this as an entertainment storytelling, narrative, documentary, newsreel, medium. And the second part was that we had to eventually be careful about calling it a VR festival because people thought it was a tech show, or they thought it was like an enthusiast thing, or that they needed special hardware, or that they had to do something physical. And so we really needed to turn it around and express that this was like a passport to experiences. And the third thing is that we have to constantly remind people that no, it's not the 360 festival. Although I appreciate the fact that we are kind of one of the last festivals that are maybe championing that side of it. And I'll get into that in a second because I think it's really actually important. But we're not a 360 film festival. We're not the 360 video guys. Because last year we had sculptures that were mailed to us that we had to assemble so that you could walk through a volumetrically captured MRI of the artist. Or we've had full installation pieces in the basements of abandoned Masonic buildings so you could experience like a walkthrough horror event. And we've always been really mostly interested in what is outside of the medium that's taking us in a new direction. I mean, I don't need to go through that. You've probably done dozens of interviews with festival curators that will tell you about the virtues of those things. And this is going to link to what my relationship to James is. As much as Five Hours was about curating the content, it was also about being a UX show. It's always been an experiment in UX. Like, how do you exhibit immersive media, and why should you do it that way? And when festivals at first approached it, they kind of treated it like a festival, whereas we would try to distribute the way that it was showcased in different ways, and every single year the show happened in a different venue, usually one that had been left behind, to kind of reinvigorate that little pocket of the city and sort of show people a new ethnographic, psycho-geographic space, and to rekindle your relationship and notions with space and architecture and communities, and remind you that that's really what this is. It's an emergent grammar about space. So when we talk about the 360 program, it's about knowing that the world is really different in the present than we assume it to be from the geography books and the travel blogs that we have. And you can get this sort of lightning in a bottle that comes from another territory, another community, that will blow your mind, that will completely surprise you. And I think that that's really, really important. So we've had pieces from Lebanon that were shot in Qatar this year, like the Fashion Institute in Qatar sent us a piece. We have a piece from the Republic of Korea. We have pieces from Australia, Spain, Brazil, the North of Brazil. And how am I gonna ignore that? These are immersive, powerful pieces. I don't call spherical video VR, by the way. I'm a bit of a semantic purist in that sense. I think that VR is a PC, CG-driven, interactive format. And I think that immersive media is a sort of larger catch-all term. I don't really like the term XR at all. I think it's far, far too broad to even be needed as a definition. There are so many sub-estuaries to that that need to be explored, and sometimes things like that can squash it, but not to get too controversial. So my interest in 360 is its ability to show us a version of something in the world that we would probably never be able to experience otherwise. And you said it in, I think, a tweet today, precisely what I feel, which is like, how are we not constantly running these cameras? How are we not constantly running these 360 capture mechanisms that could look at the world as it is today before there are driverless cars and before the drones come into the picture or before robot dogs patrol the streets. This is something that should always be going on. So it's not that I'm here to be like a preservationist of the six-year history of spherical video, but I think that it has something that we should not be ignoring, which is the world as it is today and the way that people in the world see it and themselves.
[00:16:50.317] Kent Bye: Yeah, I wanted to bring James in here, but I also wanted to just share one quick experience that I had from watching the program, which was that watching all these documentaries of going into these different places, like these cultural rituals that were happening around the world, And as we're going through this hell of a year of 2020, I'm recognizing that some of these rituals, they may or may not be happening for certainly not this year or many years to come. It was almost like this realization that this footage I was watching could be like some of the last iterations of an annual ritual that may have go back for decades or centuries, perhaps even when some of the cases, I don't know the full history of some of this, but just felt like this culture that was very well developed. And yet there's videos are capturing these ethnographic slices. And yeah, just that, that as I'm watching it, it's just like a year or two ago, but because 2020 has been such this huge shift, it feels like a lifetime ago when we used to gather together and be co-located in the same space. And to just watch the footage of that, it was so surreal and weird in that those types of anthropological lenses of capturing different aspects of cultures from around the world is something that I always have loved about the medium. And it was something that's just striking that I saw
[00:18:11.620] Keram Malicki-Sánchez: It's definitely a big point. I mean, I have to point out that I grew up, my mother worked on all sorts of boards that were designed to look at culture and diversity and culture, like growing up, you know, from the 70s onward. She was a female anchorwoman for the Spanish news in Canada. And I'm a hyphenate, Pole, Ecuadorian. And it was always the conversation in our house is like how to look at multiculturalism and the Canadian mosaic. Like in Canada, they call it a mosaic, not a melting pot. You know, in a mosaic, everything retains its original integrity and color and shape, but it all works together to form a larger pastiche. And that always really impressed me is to allow things to come as they are and to remain as they want to be. And so, yeah, I'm looking at the catalog, and we have Believe by Maude Clavier, which is a French-African expert. shooting about the different communities in Lebanon and how they get together despite their religious differences, or the echoing of Fire and Life, which is about an old Christian parade that happens in Spain, or about the Arbaeen Walk, which is about 25 million Muslims who march every year, the pantheon of queer mythology, even a piece that was a bit contested about going in at first, called Walking Past Abandoned Houses, I Think of Eric. And it's about how this neighborhood in Portsmouth, Ohio is falling apart. There's opioid overdose, buildings are falling apart. And I thought, that's a portrait of America right now. So we've got 24 hours in base camp where you're hanging out with the Sherpas at the base of Everest. And the other piece about the woman who started her own airline. And they're done precisely to do that, to represent the whole of the world and to see how rich and diverse it is as it wants to come at us without trying to do so. These are two of the places that they're derived from like everyday life.
[00:20:25.177] Kent Bye: Yeah. And so maybe we could go to the actual WebEx R portion of five hours. Cause I think that's like a whole other area on top of the curatorial selection that you made. There's this immersive experience to get access to the 360 videos that you were talking about. Is there's about, what is it? They're like 20 or 23 different experiences.
[00:20:44.871] Keram Malicki-Sánchez: There's a, yeah, I think. I mean, overall, I think we have 35 experiences and around eight of those or nine of those are the CG VR pieces and AR. So yeah, about 28 or 29. And so here's the thing, 5Rs was something that would have a rating system. Every time somebody would stand up from their film or their experience, we would ask, our docent would ask them, hey, what would you give that out of five? It was a casual question, but over the course of the day, we started to get a real series of of numbers that would say, hey, this one's kind of polling better than the other. Why is that? And we would sometimes ask those people that question. And we would take that back to the content creators. And sometimes they would iterate on the piece in the middle of the festival or try to improve certain problems of the logic or the sensitivity, motion sensitivity, whatever the case may be. In fact, this year, we have a piece that came back fully improved based on that. So I wanted to maintain that ability to look at a piece, to hear what people think about it, rate it, and then that factors into our People's Choice Awards. So that was the first thing. The second thing was that the UX, as I said, is half of 5Rs for me, like learning what people's experience is, how to come in, make it as simple as possible for them to democratize the process, to remove the edge. And whereas VRTO is an industry-facing show, 5Rs is a public-facing show, and we don't assume that the people who come have ever touched VR, they're just curious about international experiences and interesting new art stuff. So what is the point? The point is that I have a relationship with Janice that goes back to 2015 as well. I was at a meetup in downtown LA. I met this person who was showing off how he was building a Christmas tree that had a bunch of Christmas ball ornaments that were all video spheres and that he could fly into them in real time. And I was so blown away by the performance of whatever process he was using. And he was like, yeah, and we could stick this whole thing on a thumb drive and just build like a little local mesh net and just stick these things in brick walls and have like our own private web of spaces. And it was built with Janus. So I went completely crazy about this. I was like, oh my god, it's the future of everything. It's the metaverse. And we always had them front and center at VRTO. And I really got to know that community. The story of Janus is the story of Janus. And it never fully was able to meet that commercial connection with its audience for a variety of reasons. And it was a personal project for me to be able to help go back to that engine and say, I think we could take on this challenge and create a far more usable front end using this use case. And so I was lucky enough to have James do a big update for VRTO in June. And James added some absolutely killer features like video chat and better VoIP and some onboarding UI that I knew we could exploit for 5Rs, which we wanted to be that social intermingling discussion of the films. But also we had to figure out how to show this So I'm gonna hand it over to James in a second here, but I knew that there was things like Museum of Other Realities. I knew that there was kind of like video players out there, but my hope was we don't need to do what's already out there. If 5Rs has any role is it's to try something else again, try something different, see what else could work, what else could like resonate. And so I knew that Janice was the fastest engine for playing spherical video. And it just so happened that in the summer, we got a grant from Amazon Web Services for VRTO to try out new online virtual conferencing stuff. And so I said, we have a bunch of headroom here where we can play with their VOD transcoding services, because Amazon powers Netflix and Hulu and everything else. So I thought, what would happen if you put Janus together with AWS? What could you do with video? And James quickly demonstrated through a lot of tinkering that we could get 4K spherical video going in real time with no buffer, no downloads in the browser. And that was the beginning. And that was the testament to the fact that these two tech stacks going together could do something kind of unprecedented and amazing for a custom build. Like you didn't have to go to YouTube for that, James.
[00:25:39.702] James Baicoianu: Yeah. I mean, I think there's, uh, like you said earlier about kind of the challenge of how to present this stuff in a way that is digestible to people who aren't necessarily familiar with that. And I think that echoes a lot of my journey with the web-based, like I said, WebVR, WebXR. And even before that, you know, I started doing WebGL back in 2010 and it was kind of just struck by this idea that it unlocks so much power in a website. But the main challenge for us has always been, People land on this interactive 3D experience, but they don't expect that. So they kind of like land on a website, expecting to use it like a website. And like you said, Karim, it was almost on us to kind of like guide them through it using the right UX to say, Hey, this is an interactive piece. You can walk over here. You can explore this whole 3D space. And so that was kind of a great opportunity working with Karim on 5Rs and VRTO before that. client, more so than the native client. Because I think with the native client, you install it, you have that expectation that you're going into a place using this 3D software. But for websites, it's been hard to convince people that that's, A, possible, B, something they want to do. And it's been a great opportunity working with Karim, because he has this vision of how to present all that stuff in a way that users can jump in. And because it is a website, We went for kind of like a hybrid of kind of traditional website. So, you know, you drop onto the website, you just start scrolling, and you interact with it as if it were a regular website. And then it takes you through this onboarding process. And then, all right, now you're in the world. Now you can watch these videos. And we did a lot of work on accessibility. were using screen readers to navigate the experience, people who want to use keyboards to tab through the experience, you can do that. So we kind of took cues from both the 2D web UX best building immersive experiences. And so, you know, what we ended up with, I think, is something that kind of smoothly transitions someone from using a 2D website into being in this immersive 3D world. And because we used, you know, WebVR, WebXR, WebXR now is what we're using. You know, we're able to target people on any devices. I mean, we support desktop and mobile 2D screens. We support Cardboard, Oculus Rift, Vive. index, the quest, all of these using the same code base. So we didn't have to kind of build totally new experiences for each of these platforms. We didn't have to do a bunch of work to port because, well, I'd say the browsers have implemented the standard. But that's a whole other discussion there. Well, it's not easy.
[00:28:42.565] Keram Malicki-Sánchez: I mean, in theory, it would be great. But the problem is that there's been strange evolutions, let's say, and some companies have pulled back on certain tech or made weird updates that kind of break everything. And James has to do just as much patching and working around stuff and hacking stuff to make this work. So there's a bit of brownboxing going on here where we're kind of saying, it's like an organ grinder. You're like, hey, look, this thing works over here and this will happen over there. And imagine if this worked over here because WebGL, WebXR is harder than it seems right now. And the small community that is fostering it is finally now really coming together again and coming together as a larger community. And it's putting all kinds of powerful injections back into this technology, which again was itself kind of left behind by the VR world. But the reason I think the web is important, and I have to credit my friend Jin, who's such an evangelist. I mean, he is just doggedly chasing the true independent, sovereign, open source, interoperable metaverse. And, you know, he constantly inspires me. But, you know, it's really what I want is to be able to give it to any person anywhere on any device without the barriers, without the heavy duty downloads, without the walled gardens that can suddenly spring up around these things and the slow patching mechanism. Like here, you're able to look at what's going on, create an update, push that, get a pull request, push that to the live thing, and it's already patched. In real time, you can do that. So we have this real interactive dynamic with that audience, well, of fish. We'll get to that in a second. and the experience. I love the dynamism that the web affords us. I love the power that WebGL affords us, drawing on the GPU to draw that thing in real time for you. And I think it's also important to note that JavaScript is a different creature than it was just a few years ago. It used to be kind of like this web interactive form-based thing. But you can kind of get rid of things like jQuery and everything else now because JavaScript and its libraries have become so powerful and so robust that we're also kind of looking at the pioneering of what is this new thing that's just emerged with 3JS and Babylon and all of these new systems, you can create deep, powerful, rich, interactive 3D spaces natively in your browser. So 5Rs is also exploring, like, how do you do that in a way that is enjoyable, easy to understand by the public, and yet plays to its own nature? You know, that doesn't just say, hey, it's like a movie theater, but in the web. But like, what is it in a Nun2 itself? and what can it feel like unto itself. So rather than have humanoids with feet bobbling around with bad physics, I thought, why don't we just be fish and then we could swim around? Because you could be an octopus, you could be an amoeba, whatever, but something that feels right in this world, in this physical metaphor. And it's funny because You know, we asked ourselves, should we be animals? Should we be plants? What could it be? Fish just felt like that scene from Monty Python, that skit where they all wake up and they've got these little faces on their fish bodies and they're like, good morning, good morning. And I thought, you know, with these little video chat faces on our avatars, it feels just like that. And when people come in as a fish, and I mean from any country in the world, from any tax bracket, from any cultural upbringing, they immediately are fine with being a fish. There's no question as to why or how. They just do it, and they love it. And so we have aquarium side chats, and nobody questions the metaphor. It's just amazing how quickly they can transform into that. In their web browser, swimming around a theater, in between rooms, I mean, even the speed at which you can switch from one room to the next. Janus has these portals. And when you go through that portal, it's not like, please stand by while we download the new zone. It's instantaneous. It's so fast. James, maybe you can explain why it's so fast and what's different about it than other systems.
[00:33:19.642] James Baicoianu: I mean, a lot of that has to do with the fact that I mean, one of the things that makes developing for the web difficult is that you do have to do a lot of work to optimize this stuff. There's a lot of work optimizing assets. When you download something like Museum of the Reality or any game, you're talking tens, 60 gigs. I think Microsoft Flight Simulator is like 100 gigs of download before you even can start using the experience. And that's because they're using uncompressed textures, uncompressed audio, be as high quality as possible and it's not really necessary. The amount of quality you can get out of like a JPEG or a PNG versus a TIFF is like your average user does not see that. So because we're able to use these kind of technologies that were kind of made for the web, basis universal texture support, we're not using it too much in this experience, built to make web experiences load well, we're able to use those with our 3D engine. And going back to what you said earlier about this whole process where JavaScript has improved over the past, ever since I started using it, it's been improving every year. But it feels like in the past, 10 years, it's just been kind of like this exponential growth. And one of the things that always struck me, like back in 2010, I started developing WebGL, and it was, it was so rough around the edges. There was so much stuff that like, you could get something to render, and then it would crash, your whole system would crash, or you would get like, textures from your operating system would be rendering in your 3d world. And it's like, that shouldn't happen. But Anytime I ran into any of these obstacles, it's like I could file a bug with Chrome, file a bug with Firefox, and I could walk away from that. I could go work on something else for a while. And it's like there was this team, like I was working alone on this stuff, but there were hundreds of other people who were working alone on it, and people working in these large organizations who, like you would show them something, you say, And they're like, wait, you're doing what? You're doing what with our browser? We didn't intend that, but it shouldn't crash. So we're going to look into it and figure out how can we make this not crash. And then, you know, WebGL started being taken a little bit more seriously. And then WebVR and then WebXR, you know, now I think for a while at both Mozilla and Google, there were very small teams working on these things. And I think it feels like they've started getting a lot more serious about it. I think we've collectively, as a community, proven that you can build pretty amazing things with this, but we've always been kind of dependent on the browsers kind of to set the pace. And that's been by far the most frustrating part is that, you know, WebVR came out in 2014, and I kind of very quickly added support for that to the engine that kind of underlies Janus Web Client. And You know, it's been kind of this evolutionary process to get to where it is. But, you know, it's it's I think finally paying off. I think people are seeing, especially, again, with the pandemic and people wanting to move more things online into these virtual experiences. You know, it's clear that this unlocks a whole new type of experience that, again, it's hard to say because it's what we're doing isn't 100 percent like nobody's ever done this before. This is stuff that people have done before in the form of like Second Life and other platforms. Even before Second Life, there were things like Active Worlds, Worlds Chat, but there were always these kind of downloaded apps you had to download and install. And now we're able to do that all in a web browser because of this massive evolution of the web technology that's happened in the past 10 years.
[00:37:18.942] Keram Malicki-Sánchez: And the other thing is going to be that you can build portals to other WebGL ecosystems, right? I mean, I'd love to see this 5R stack like link out to crypto voxels or what have you, and just have these interoperable worlds. And of course, like Averse, like Metachromium would allow things to be superimposed on top of each other. But by working in this open source ecosystem where everybody's contributing, I mean, part of my buy-in for James was, hey, if you work on this, which you'll work on way more hours than I could ever afford to pay you for, the benefit of the work will fold back into the greater community and its ecosystem and its code base. So take this as an important pet project to benefit the advancement of the larger cause that you've been working on. And then there's that recursion between those communities that they're all sort of supporting each other instead of working harder and harder on proprietary systems that can eventually become pixel-rotty and crash and burn when somebody's upper thing leaves or it gets taken over. So that's the other thing I really want to foster, is that open source, punk rock, DIY spirit, which is not to be mistaken for low quality, cheap, and crappy. It's just that it's sovereign, and it's not beholden to anybody. It doesn't have to play by anybody else's rules. It's by the people, for the people, for the artists, and so on.
[00:38:45.649] James Baicoianu: Yeah, and I think that is definitely something that we've had to struggle with. People doing web-based stuff had an uphill battle to convince artists that this was possible. So for the longest time, you basically had programmers who were kind of defining the aesthetic of these experiences. And, you know, you have people like Mr. Doob, the author of 3JS, he comes from like the demo scene background, and some of the other people working on 3JS, like Core they're working a lot with like generative geometry and shaders and things like that. Convincing texture artists and 3D artists, 3D model makers, environmental artists that they could take their trade and kind of move it over to the web, you know, export their massive worlds that they made for Unity or for Unreal and, you know, do a little bit of work to optimize them for kind of download time and performance, runtime performance within a web browser. but you could deliver those same worlds within a web browser and it's just one click and you're there. And again, like you said, the fact that it's built on the standard technology means that if I have a link to, you mentioned crypto voxels or, you know, an A-frame experience, Babylon JS experience, I can link to them without really having to, there's nothing special going on there. It's just like, It's a link. It's it's just like a website normally click. Yeah. And because of the standards, they kind of figured out these ways that like, OK, well, if you click this link while you're in, VR will emit this certain event when you pop out the other side that says, hey, this guy was in VR already. He's allowed to kind of pop right into VR on the other side. And so kind of like it's like this little handoff. that the A-frame people may not know that I'm even linking to their experience. They don't need to know. It just works because we're all using the same standard. So yeah, I think it's really powerful.
[00:40:51.739] Kent Bye: Yeah, I wanted to talk a little bit about my experience of both the opening night party and what I was able to achieve last night by watching all the videos. So on the October 14th, there was the open night party of 5R's where you had folks come together and kind of move around. There's this 3D space that when you're looking at a 2D version, you kind of like move your mouse around and it allows you to kind of navigate around. And then the VR at that point wasn't working yet. It was working later in the week, but I was like just looking at a 2d screen, but I had my webcam here and was able to locomote around as a fish. I honestly couldn't tell that I was a fish once I was a fish. Cause I didn't have a mirror. The only way that I could tell I was efficient was that everybody else was a fish. So if there's no one around, I didn't really know what my embodiment was. There's no mirrors or anything for me to really get grounded into that embodiment. because I'm a first-person perspective as a fish moving around. But you kind of took us around on this guided tour, both you, James, and Karim, showing us around the world that you had created. And there was spatialized audio in the sense that what was weird is that I could see everybody on the webcam, but yet the audio that I was hearing was dependent upon how close I was to somebody within the virtualized space. And so there's a bit of a matching where I know like gather.town, they have like dynamic webcams that will only pop up if you're within a certain geographic distance from somebody else. And I think something like that might help to create a level of persistence when it comes to, as you're moving in around a space, allowing for organic clusters of people to come together and start chatting. Because I think that's going to be actually a really powerful way for people that don't even have VR equipment, but still using this virtual world to be able to use it as a social space to be able to come in and talk to each other. And I think actually the onboarding that you were able to put together was really streamlined in the sense that you're able to set up your audio and your video, choose your embodiment, your name, and then kind of go through these different levels of granting permissions and then making decisions as to whether or not you want to go 2D or in VR, whether or not you want to use your microphone or not use your microphone. So a lot of just like basic setup that decisions that you need to make and just making it clear. And then you're in the world after that. Now, when I tried to actually go in and watch the experiences when it first launched, there were some things that weren't working yet. Like the VR wasn't turning on consistently and the frame rate was like really stuttering when I did get it to work. So I waited and you push some new updates. And I was just amazed from over the course of that a week or 10 days where you're able to basically fix all of that. And I was able to go in yesterday, which was the 27th of October. And essentially spend, and it was probably around six or seven hours just going into the main virtual space and watching each of the pieces. There was a weird bug where I did figure out a way to get rid of this little white box that shows up. And what I would do is I would go into a world I would click the menu button to be able to see the desktop version of Steam. I'd be able to use my pointer to click on the reload button. I would reload it, and it would reload, get rid of the white box that was not supposed to be there. I'd go there, hit on enter into VR, and then when I enter back into VR, then I would click the play button, and then I was in the video. So there's some ways in which that, you know, obviously there's bugs like that that can be fixed, but I was able to, with those workarounds, be able to sit in one spot for like six or seven hours and essentially watch the entire show of all the different pieces that were there, including a piece that was like an hour and 45 minutes. So there's lots of content that I was able to just binge watch. And I absolutely love being able to do that. Just being able to be within VR, go into an experience, come out of an experience and go into another experience. The interactive portion was a lot more thrashing where you have to download stuff. And I don't have that kind of seamless museum of other realities type of experience where I just download a big giant singular download on Steam and be able to do that same type of going in and out of interactive experiences because linking the web into these Unity projects or Unreal projects. And that's just something that hasn't really necessarily been figured out how to launch binaries or how to download binaries. I mean, it's like everybody is trying to figure that out. But what you were able to at least achieve with the 360 video and the web space, I think is like you were saying, like the 360 video as a medium is a perfect medium because you're authoring it outside of the web and you're able to just use the web as a display medium and create curatorial experiences like you have with 5Rs where you can go and see these like 27, 29 different experiences just by going into this virtual space and kind of popping in and out of stuff. So I think that it's a usage pattern that I think is super compelling, especially for going to a festival to a number of different pieces and to go into a world that serves as a portal into these other experiences. Eventually those portals will be into other worlds, but this type of thing, I think is actually going to be a template for what we're going to see a lot more of moving forward, which is world builders that are essentially curators into trying to guide people into specific experiences that they want the audience to have.
[00:46:00.433] James Baicoianu: All right. Yeah, by the way, the bug you were mentioning there is now fixed. Again, as Karen mentioned, the fact that we can kind of push these updates on our own schedule rather than like, oh, we made a fix. Now we've got to submit it to Oculus and have them go through their approval process. And maybe it'll take a week. I don't know. We're able to iterate on it much more quickly. I want to tell you a little bit about where that bug came from. It's kind of like one of the pains in the butt of working for the web. is that you don't know what device someone's going to come in on. You either have to say, this only works on desktop, this only works on mobile, or this only works on Rift, or Vive, whatever. With web, you got to target them all, because you don't know what they're coming in on. So that floating white box that you saw was actually a sprite that represents as you mouse over something, your pointer turns into a hand to indicate something is clickable. In VR, that doesn't make any sense. So it's like, ah, we're not updating that image, but that mechanism was still active. So it's kind of like we had to build, like, the upside of building for the web is that you can target all these different devices. The downside of targeting the web is that you have to sit and use each and every one of them and see, ah, well, this in the way for VR users. Some change we made for mobile users made the experience worse for desktop users. And so there's definitely a lot of back and forth on that, like Kara mentioned, like the organ grinder, where you change something and everything else breaks. And it's the same thing with the VR as well, was kind of the story of Janus Web. Again, we started with Web was a spec in 2014 that Mozilla championed. Google had an implementation in Chrome, but it was always kind of locked away behind these flags. So it was kind of like this constant battle. We needed them to unlock it so that other people, like the masses, could use it. We're still waiting, really. So WebVR went for a while, but Chrome said, no, we're not going to enable. We're never going to enable WebVR in our browser. We're going to work on WebXR, which is kind of like, version 2 of that and it kind of allows you to do AR and VR. But it's still been kind of like this arduous process for them to kind of get that working because they need to work with the runtimes that Oculus provides and that Vive provides and that Windows Mixed Reality provides. As an independent developer, it's great because I do have the freedom to like, I'm not locked into any system. I can develop what I want without having to go through gatekeepers. but there are still gatekeepers. We're still reliant on the browser manufacturers to actually turn these features on, to have bug-free implementations, and then they're reliant again on companies like Facebook and Valve to have compliant... Basically, everyone's now standardizing on OpenXR. So, Facebook has started releasing OpenXR-compatible builds, but they have their own set of bugs things specifically for oculus that in a month hopefully they'll have a fix for that which will then we'll have to go back and update our code to undo our fix so that we work with their fixed code so it's it's this ongoing kind of struggle to kind of keep everything running across all these devices dealing in the shadow of the titans
[00:49:41.726] Keram Malicki-Sánchez: You know, James is extraordinarily disciplined, though, in always looking at the macro view of the code base. And he doesn't chase it with these little post-it note tack-on fixes that are going to kill us later. And I really feel comfortable and confident with James's discipline in terms of maintaining that at the highest level so that we don't get into these conciliatory messes just to make it work just in time. And, you know, even at the very beginning of the show, we were ready to launch, we were ready to go. And then we find out that Chrome has suddenly disabled 360 in Android Chrome browsers and that you have to now it says to play this back, you have to use our YouTube app. So for some reason, they suddenly pulled support of spherical video in the Chrome Android browser and made it proprietary to their app. So you can maybe explain that better than I can.
[00:50:36.968] James Baicoianu: There were two different issues there. One was I think we were looking at some UI thing and we're like, okay You just want to launch into it like you're using the 2d website But you want to launch into the video in VR because some of those are stereoscopic They have like ambisonic audio. And so yeah, you can watch them on the 2d screen. They're great but when you watch those in particular on a VR headset you in your right ear, you turn and you look at it and it's right where you expect it should be. Anyway, we were looking at how to make that seamless. And so we're like, well, YouTube does it. What does Google do? And so we're like, okay, let's check it out. Let's see what UI they use for that. And we go and check it out. And the UI is, we're sorry, we don't support 360 video on desktops. You have to basically use it on your mobile phone using the YouTube app. So it's like, and it's funny because this is something they used to support, but now it's, you know, What we're showing it on, it clearly is, you know, but it's just a matter of like different teams within those companies may not be keeping up as well with the WebEx. Our team is separate from YouTube.
[00:51:52.240] Keram Malicki-Sánchez: So the implication of that change is enormous because they control the phone, the OS, the platform, the browser. So when they make that shift, it impacts everything downstream from it.
[00:52:04.582] James Baicoianu: Yeah, absolutely. And then it becomes this question. It's like, did they remove that because they actually think it's not technically possible? Did they remove it because they couldn't get it to work? Or did they remove it because they would prefer you use their native app? It's like, what are the incentives for them to make it work on the web when so many people are pushing people through Well, if you want to get the best experience, you've got to have an Android phone with our Android app.
[00:52:41.828] Keram Malicki-Sánchez: It's hard to say how much of that is actually... Yeah, we don't have to go down the conspiracy pipeline, but it still shows you what happens when everything's kind of reliant on a company, right?
[00:52:52.585] Kent Bye: Yeah, well, the market share of the Chrome and Chromium is so huge that it's like so many other browsers downstream, Microsoft Edge, and you've got the Oculus browser, of course, Metachromium and other, lots of different other companies that are relying upon that. But I think it also speaks to just like the difference between say Unity builds where mostly Unity developers will pick a version of Unity and they won't update it for like years because it'll break everything. So you can't really do that on the web because you can't just pin your release to one version of Chrome because it's constantly automatically updating. you're kind of getting into this dark pattern of when you go to like reddit.com on your browser and you're trying to like look at a page and it's like, oh, sorry, you can't see this unless you install the app. Or it says every single time you go to the page, are you sure you want to use this website? You know, you couldn't use the native app. And so they have this whole like native app mindset where they just want you to like download the app and just spend hundreds of hours on their app when you just want to watch like one thing. And then maybe go back and do whatever you were doing before. It's like easier to like capture your attention when they do that. So I can totally see why there could be like a lack of support of things like 360 video that because they have so much control over the browser that they are breaking things to sort of send people off into their native app. So hopefully that's not a trend that's actually happening because if it is then It's going to be kind of a dark future for WebXR when you have that type of anti-competitive type of behavior happening. Thankfully, there's a lot of hearings that are happening currently right now where that type of actions are really being scrutinized by the U.S. government, but it's still yet to be seen to what degree any sort of enforcement or change will be implemented. Anyway, that's a lot of the big tech flowing down to just something like WebXR and your experiences of what it means for an indie film festival to try to build stuff on the web, which I have to say, congratulations for getting for what I think is probably one of the more compelling WebXR experiences, having spent seven or eight hours within it. of course, going into these immersive videos, but the point being that I was using this as a portal to have an experience for that long. And I think that we need to just have more experiences like that, that are trying to find real compelling use cases to really stress test the technology. Because until there's the artist and that need to express yourself and to use the technology to create something to have the technology that has its capabilities, the artists come in and make something, and you have the distribution platforms, which in this case is WebXR and the web, and then you have the audience that is coming to see it. So as you have each of those as an iterative loop where you're able to make something, put it out into the world, getting into people's hands and for them to be able to actually experience it, then you get that feedback and it just helps catalyze the evolution and development of the medium itself. and you get more experimentation and more forms. But if that cycle is broken, which it has been broken with the distribution aspect of just WebXR being such a nightmare of Google waiting until it's perfect and waiting until the latest version of the spec. And you know, it's just a lot of issues that were going in that I've just been waiting for the big like go sign to be like, okay, let's go. We had Mozilla hubs that came in, but it hasn't been a lot of other things. Well, this is probably one of the things culturally at least, where you start to see an experience within WebXR that's like, for me at least, it's like, this is a really compelling use case. And it actually is doing a lot to push forward what's possible. And you're likely running into as many problems as you're solving because you're such on the bleeding edge with all this stuff. And I think it needs to be recognized and acknowledged and for people to just check it out just for the sake of creating that loop of getting that feedback and just making it more better. Because if you make all of this and no one shows up to give you all that feedback, then you still don't have that loop.
[00:56:47.925] Keram Malicki-Sánchez: Every person that walks through this little valley improves it. We watch it. We can see sort of anonymous logs of like, when did you come in? What device did you come on? What did you preview for how long? What did you go in and out of? Did you throw errors using what controller? And all of it is just constantly helping us to understand better. Where do they go first? How does it work? How does it move? And so on. And also, we have to pay tribute to all of the community that has contributed to Janus over the years, because they really, really did this as a hardcore labor of love. And a lot of people, I mean, James is an extraordinary, extraordinary person and programmer working basically solo with me on this. to get it into position. So every time there's like a new feature idea or hey could you fix this or fix that, it's James and his time and how much time he can dedicate to squashing the bug. There's so many things we would of course love to do, but we're effectively building a platform here and a theater and a festival all at the same time between like two or three people on this team, right? So we know that it can't be all things to all people on all platforms in all ways at all times and all resolutions. It's just simply impossible. And it might not be a great idea to do so, but it's a really, really good experiment for now to take it in a new direction and see what else this means, as opposed to just folding in with all of the other wonderful solutions that are being used by the other festivals out there. I just want to see what else can happen and take some really wacky, weird chances as well as implementing something that can be folded back responsibly into the code base. But this is also like, again, we're building on a lot of people's blood, sweat and tears that have contributed to this code base up until this point. And now there's a big chunk of it that's behind a private Git, which is like 5R specific, that is a break off chunk that we're developing hardcore and testing out new possibilities. And it's nice to have a little bit of this bandwidth with AWS to see what happens when you match that up and throw that power behind it for transcoding, for serving up video on demand. You know, we're throwing this thing like eight to 16 gigabyte files And it's got to transcode them for all different sizes until we find the exact perfect balance of which film is the highest resolution but still performant. And then we have different audio codecs that get thrown at us, spatialized, that have to then be parsed into the system that James created to sort of pseudo that up in the browser and give you that spatialization. And then you've got over, under, 3D, side-by-side 3D. And it has to be able to figure out, what are you watching this from? Let's give you that back. at the highest possible resolution in the browser with no download time, with no buffering. It's an enormous feat just to get that tuned correctly. And just one flag in that system of flags could make the text on the screen look artifact-y, alias-y, look at bad compression. lose frames. I mean, it was a whole recipe that eventually had to be found to make that as smooth and high res as it is. And the first time I saw that playback full screen 360 with ambisonic audio, I was just like, oh my God, what just happened? I hit play and it started playing. That's crazy to me.
[01:00:07.531] James Baicoianu: Yeah, we were definitely pushing the limits of kind of what these companies thought the technology would be used for. Like, you know, again, leaving aside all the WebXR stuff, the Amazon, they have the Transcode service. We kind of upload videos and they can run these rules to generate the different sizes and resolutions, bit rates for kind of this online streaming. And we were pushing the limits of what Amazon was configured to do. You know, they max out at which means 3840 by 2160. But because we're working with a stereoscopic 360, if you're doing over under, that's 4K by 4K. So we ran those videos through there and said, hey, we want a 4K by 4K version of this. And they're like, sorry, no one would ever need a video that big. We don't support that. It's like, what do you mean you don't? It's just a few more pixels you got to process. But in their mind, you know, they're working with the Netflix's and Hulu's of the world. of what's possible, online streaming of 4K, ooh, wow. But we're going even beyond that. So it was definitely a struggle to coax the quality. Technically, the 4K by 4K videos, we had to downscale, like I had to do manually, throw a bunch of resolutions at AWS until I got a number that it didn't complain about. So it's like 3172 by 3172. maximum that they would handle.
[01:01:40.438] Keram Malicki-Sánchez: Yeah, we squeezed in every last pixel we possibly could.
[01:01:45.920] Kent Bye: Well, when I first tried it on October 14th at the open night party, it kind of matched my expectations. It was like, is this going to work? And it didn't quite yet. The VR wasn't launching in and I just wasn't able to get it. But just a week later, it sort of exceeded my expectations in terms of just being able to like have the experience. And I think that at least for me, at least is a great stepping stone in the evolution of what's going to be possible. And like you said, you're getting all these other big companies to push the limits and maybe fix the bugs that No one else has been having them do that. So being on the bleeding edge and the pioneers in that sense, you have to run into these things, but you're paving the road for people in the future to be able to do this without so much thrashing.
[01:02:26.726] James Baicoianu: So you have no idea how much it means to me that you were able to sit there and go for however many hours you said in that experience. Cause like you said, 10 days ago, that would not have been possible. You know, it's been a slog to get VR to that point, just because of kind of all of the browser. platform inconsistencies, I'd launch into it and like, all right, I got it working just right. And then I'd try another one. I was like, Oh man, now my, I've just got like this huge headache from whatever bulk it's on this other platform. It's like, it was, it's been an animal. Yeah.
[01:03:02.153] Kent Bye: Yeah. Well, uh, just to kind of wrap up things here, I'm just curious what you each think the ultimate potential of VR might be and what am I be able to enable?
[01:03:15.147] James Baicoianu: I'll jump in first, I guess. I'll give you kind of the same response. We did an interview, what, three, four years ago now, and you asked me that question. And my response then was, I feel like it's the future of remote work. It sounded a little crazy back then, I think, but I think it sounds a lot more reasonable these days. you could jump into a VR headset and you could be controlling a robot. They have people controlling robots to stock shelves in Japan using VR headsets. You can kind of take that to its conclusion. We could be exploring the universe, sending out robots to Mars, and you kind of jump into a VR headset and you got like a fleet of robots you're controlling. You can kind of just embody one of these bots and kind of like necessarily 100% real time, just because of time of flight and things like that, but being able to use this technology to embody things that are remote. for VR like mechanic simulator and job simulator vacation simulator that's like a little too abstract but like mechanic simulator where you go in there and you're like diagnosing problems on a car and this is entirely virtual you know it's a game but I can see a future not too far from now, where you need work done on your car, you go into like a robotic service center. And the robots can handle most situations. They can change your oil, they can diagnose a timing belt issue, whatever. But maybe you need every once in a while a human to kind of come and say, huh, all right, I need to look this over. That's the kind of thing that, you know, someone sitting at home with their VR headset, they can put, you know, bing bing, you know, incoming service requests they put on their headset. And now they're wherever they are, they're servicing a car that might be on the other side of the world from them. They're able to go in there and say, OK, well, I see what's going on here. I can kind of give instructions to the remote robot type things. And so I think that's where a lot of this will be heading. And I think that right now we're looking at VR as this kind of platform for gaming. But the ultimate potential of that is way beyond gaming. I think it's going to be transformative as far as, you know, how we interact with technology, which, you know, with everything becoming technology, you know, becoming technology, becoming part of everything. I think going to be a lot of different uses are going to show up from that beyond just the gaming uses.
[01:05:47.019] Keram Malicki-Sánchez: And I think I definitely agree with you, James. And I've talked to Kent before about that question. And I said, yeah, being able to pilot a three-story mech down the street to pick up debris and floating cars after the wreckage from climate change hits us will be an important thing to do from our basements. And also to add another idea, I think that VR is a way to give ourselves that mental scaffolding to go into places that would be very difficult to do otherwise and to sort of test certain limits, cognitive limits. anthropocentric limits, ontological ones, and explore new ideas and embodiments and relationships and cultural input and allow ourselves to be free in a different way and in a different relationship to those things. And all of these different experiments that we're taking on are ways to make that a more robust process and a richer one, I hope. I hope that it creates a broader, more wonderful canvas for us to play on and not a panopticon, you know? And so I think it's really, really essential for us to constantly keep open hearts and minds about the people inside of this process and allow those people to be fully expressed in every possible way that we can and beyond being people, you know, that the people part is something that we can almost leave behind and see what is left over afterwards.
[01:07:22.471] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[01:07:29.137] James Baicoianu: Keep up the fight. I think a lot of people involved in WebXR and immersive technology, they've been fighting an uphill battle for a long time. And I see a lot of people that are kind of burnt out on it. Myself included, to be honest. It's been such a slog to get to where we are now. And it's to the point where I'm like, Do I give up or do I keep fighting? It feels like we're right on the edge of this kind of becoming, like I said, Chrome releasing this as an out-of-the-box. You don't have to do any special command line parameters or install special versions. It's so close to being unlocked for everyone. And so I think we've been in this, the valley of despair, chasm of despair, whatever they call that. Of disillusionment. The bog of sorrow. The bog of sorrow. These all work. We've been in that for so long, but it seems like there is a light at the end of the tunnel here. And I hope that we can get these experiences out and in front of more people. That's why I kind of, when JanusVR as a company folded, I kind of took it on as JanusXR.org to kind of keep things alive, because I still feel like in a year, maybe two, hopefully sooner, but we'll see, more and more people are going to come around to this idea that A, it's possible, and B, it's something they want to do. And so much experimentation was done by the community, not by us, not by Janus the company, but by the community. We gave them this tool where they could build these worlds. And the things that they built with them were amazing, but so few people actually got to experience them because you had to download the native app, you had to be a VR enthusiast, which at the time was such a small community, relatively speaking, that I wanted to preserve that stuff in a way that as it becomes more accessible to the mainstream users, I didn't want that all to just disappear, to fall off the face of the earth.
[01:09:37.967] Keram Malicki-Sánchez: I would say that if you think about what Burning Man is now, at the very beginning, before there was a Burning Man, it was like thousands and thousands and hundreds of thousands of artists living in their small little lofts, doing weird stuff, welding things together, putting sparkly lights on stuff, lighting things on fire. And then eventually they all came together, and eventually they said, wow, look it, there's a whole thing going on here, and what else can we do together? And it became this sort of larger and larger ritualistic thing and then a force, a true force. And I think of the Janus community, like all of those experiments out in lofts where you would never find those things on your own. Why would you? What I would say to the community is remember that the world is vast and big and far beyond the dome in which you live. And I'm constantly surprised with 5Rs about these unsolicited submissions that come in from remote, from countries, from towns I've never even heard of that are doing extraordinary work and they're speaking on their terms and they have incredible tenacity to finish the project. They're resourceful and technical skills and it's amazing to me what is going on all over the world. So remember to embrace the entire planet and allow in and create the opportunities for those things to come in and teach us something that we probably will want to learn and benefit from.
[01:11:08.299] Kent Bye: Yeah, well, just my final thought is that 5Rs as an entity is a whole thing where you've been curating for six years now and have a whole vision that is trying to bring in this more experimental avant-garde perspective, which I personally find as a relief to see not only the newness and the experimentation and what works and what doesn't work, but it's Either way, it's giving me new insights into the medium itself. And also, as we're all quarantined or sheltered in place to whatever degree that we all are, that this served to me to be like an escape into these other parts of the world and these cultures that I just don't have access to. No one has access to them because no one's gathering like they were in the past. And so for anybody who has previously completely written off the 360 video medium, maybe take a look at it again, because there's a lot of stuff that has been documented and happening that has a new historical lens in terms of things that have been happening. And one of the thoughts I had in my Twitter stream was that people in the future who are anthropologists are going to be like studying this time period from like 2014 to 2020 to see all this like 360 video that was happening to see perhaps some cultural practices that may not continue after this, that may suffer and die off and that maybe this will be the last record of that. So just to be part of that curatorial process of trying to put forth as a medium and you know, all the other experimentation and arts and the interactive sixed off stuff as well as a part of it. And just nice to see that kind of Indiecade, Indie vibe spirit that you have within 5Rs. And then the whole technological stuff that you've done with the platform is super impressive, especially to see it evolve and grow across multiple iterations from VRTO until now, and to what you're able to pull off in this sprint to be able to make it through this month-long version of the 5Rs. For anybody that is interested in the WebXR space, I think it's just important for them to check out to see what's happening. And so yeah, with that, just thank you both James and Karim for doing all you've done to create this and for joining me today on the podcast to unpack it all and make sense of it. So thank you.
[01:13:15.018] Keram Malicki-Sánchez: Kent, let me just say thank you and a shout out to my co-producer, Stephanie Greenall, who is so, so hardworking and fundamental to me being able to pull this off. I just, I couldn't do this without her. And so thank you to Stephanie as well, who's extremely savvy and works her ass off on this show.
[01:13:32.682] James Baicoianu: Yeah, and thanks for having us. It's been a great opportunity to talk about this stuff and share it with the world.
[01:13:39.383] Keram Malicki-Sánchez: And you can. Thank you for the time.
[01:13:42.744] Kent Bye: So that was James Bikwianu. He's a web developer focused on WebXR and formerly the principal engineer at JanusVR, as well as Karen Malitsky-Sanchez, the founder and executive director of both the 5Rs Festival as well as VRTO. So I have a number of different takeaways about this interview is that, first of all, Well, first, with the Five Hours Festival itself, I've seen a lot of different content from different festivals. And I'd say there's a distinct independent spirit that's in this selection of pieces. And I think the thing that Karam does that's probably different than other festival curators is that there's some stuff that even Karam sees that there's not perfect, but yet there's something that he learns from it about the medium itself. And so trying to find the different underlying affordances of what the medium is good for and what it's not, and then sometimes There's experiments that don't always quite work, but you can also still learn from those experiences as well. So in that spirit, it's like this avant-garde experimentation, pushing the envelope. And so in order to really advance the medium, then you do have to push that envelope and take some of those risks. That's not going to be the perfect story in all those cases. That's not to say that there aren't some amazing stories that are in this, there certainly are, but I think what makes it distinct is that there's more of that kind of experimentation that I see, that avant-garde, independent spirit, that there's lots of different festivals that happen over the course of a year and I tend to go and see a lot of the stuff and there's just stuff that Karim's able to curate that has likely been submitting to these other festivals, but for whatever reason that didn't match the curatorial vision of these other festivals. And so I think it's just important to have a diversity of different types of curators and different perspectives to try to see what's interesting or novel about the medium. And also there's quite a lot of different selections that had this anthropological and ethnographic survey of these different cultural rituals that have been happening and In the context of 2020, it's just actually really nice to see how people used to come together to do practices and to build culture with each other, especially now that a lot of those different practices have likely been disrupted and may have created this break in a long continuity of decades or centuries of ways in which these cultural practices were transmitted. And it's just an interesting historical document. Even from just looking at footage last year, it feels like a lifetime ago. So a big point that I see here is about this sovereignty and control to go out and to try to launch a whole conference and experience within WebXR with all the different things that you have to deal with and all the problems and challenges. I mean, there's a reason why not a lot of other people have tried it yet, because it's a huge pain in the ass and there's just a lot of stuff that isn't quite fully baked and you have to deal with things breaking across different platforms. for changes being pushed to break things in the middle of your conference. And so there's all this stuff that they have to deal with. But I think it's important. I think it's actually a pretty strong political action. Lewis Gordon is a philosopher that I interviewed at the American Philosophical Association. And one of the things he told me is that there's a difference between like a moral action and a political action. A moral action is very much connected to you as an individual. a political action is completely anonymous you're taking actions and you're doing stuff that people in the future are going to have no idea that you did that like the fact that james is like filing all these bugs with webgl and like you know finding these things with amazon and you know how they don't have Support for be able to do the type of stereoscopic 3d transcoding that they need for this festival to be able to run things in the browser You know, they have to actually expand amazon's tech specs and then from there that becomes like a political action that allows other people in the future to be able to do stuff that you know when they go there and it just works then they don't have to like go through all the thrashing that people that are on the bleeding edge and pioneering the medium and the platforms like what Karen and James are doing here with five hours. So I just want to reflect that that that's like a highly political action that they're doing to try to create these viable alternatives to these closed walled garden ecosystems. And even though they're able to kind of bypass the gatekeepers when it comes to like these stores, either through Oculus or five port or steam, you know, there's obviously other Independent distribution with HIO and whatnot, but for the big major platforms I think the web is going to be one of the bigger ones as we move forward because it's just gonna be easier for people to just publish stuff and For it to be able to get into the hands of the audience Now the thing that they say is that you still have the gatekeepers because you still have all the different web browser vendors you still have like Apple who at this point really hasn't even a done much when it comes to a lot of the WebXR implementations, who knows if they ever will, if they like force people to use all their apps and their proprietary approach that they have their whole tech stack that they have. So as we move forward, we're in this realm where it's a choice even within Google to preference their native app over getting strong support for all this within their web browser. So There's these different trade-offs that even though you're using the open web platform, you still have to deal with some of the business and political decisions within these companies that are either supporting things or not supporting things based upon what's in their best business interest. You would hope that something like the web would be able to create that, but there's been a bit of consolidation with all these other web browser vendors that are kind of consolidated under Chromium. Samsung and oculus browser and Microsoft Edge So there's like, you know Not a lot of diversity when it comes to the core engines of a lot of these different browsers that are out there so that's just another thing to deal with and as Firefox and Mozilla have been you know going through a lot of different financial struggles and whatnot. There's a some aspects of Mozilla hubs that's still surviving, but a lot of the core WebXR team that was at Mozilla is no longer working on supporting a lot of this stuff in the future. So it just sort of like puts the Firefox browser and a little bit of a question as to if there's going to be like one major vendor, which is Google and Chromium. implementing all these different web specs. And if there's decisions that get made that doesn't have a market competition for other people with other interests, then there's just a risk of not using that kind of inherent market dynamics to be able to build something that works for a huge, broad, diverse range of different applications. Hopefully within Chromium itself, they're able to maybe recreate that kind of market dynamic. So that's yet to be seen. But I think the big point that I take away is that even though we're trying to create this alternative and get away from the gatekeepers, they're still building upon these tech platforms that can have a certain amount of gatekeepers. keeping. So yeah, just the final thought is just that it's worth checking out some of these experiences, not only just to see what they've been able to create with the Janus XR resurrection into this new context and their private development of what they've been able to do. And you can go around and maybe bring some friends and see some of the different social interactions, but just to go into a space and to pop into a 360 video for it to just play, like Karim said, it does have a bit of this magical feeling when it works because it's like, wow, I'm in a browser and I'm able to have an experience that I used to only have to have within a native app. And usually within streaming of video, that's also come a long way since the beginning and early days of the buffering and everything else. It's pretty seamless of being able to watch it on my PC with an Ethernet cord. I was able to see all the stuff without any buffering at all, which is kind of a miracle to think about where we were just five or six years ago. That was certainly not possible, certainly not even within the browser. And then in the actual experiences, there's just a lot of interesting stuff to be able to check out. And I think it starts to open your mind a little bit about some of the affordances of the VR medium. So it's just refreshing to me to see this curatorial vision that has a little bit more independent spirit embedded within it. So that's all I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
