Janus VR has been implementing an IPFS distributed file system option for creators for the past couple of years. I talk to “Taco Dog” who was the catalyst behind helping to architect the protocol foundations of the decentralized metaverse.
He has also been experimenting with building mesh network hardware like a fashionable necklace that contains a WiFi router, SD Card, and the mechanism to be able to enter into a shared virtual world together. This mesh network node was inspired by dead drops from the information security world where there would be a physical way to transfer digital information in such a way that couldn’t be tracked online.
This type of physically-constrained network could provide the basis for exclusive VIP virtual and augmented reality worlds that could only be accessible by having access to the physical location where it’s located. Or these types of distributed, P2P mesh networks could also provide the basis for more secure back-channel communications within these virtual worlds.
I had a chance to talk with “Taco Dog” about his passions of helping to architect the decentralized metaverse, information security, and the archival of physical event spaces using photogrammetry + a depth-sensor camera on a Tango mobile phone.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye and welcome to the Voices of VR podcast. So one of the most interesting people that I had a chance to talk to at the Decentralized Web Summit was somebody who was going by the name of Taco Dog. This was a self-sovereign identity identifier that he was deciding to go with. And he's somebody who's been involved with and working in tracking the metaverse. He's got a blog tracking his evolution of these different experiments that he's doing. It's a bit of a creative coder and mad genius trying to put all these different things together. So he was somebody who really brought the idea of the interplanetary file system, this IPFS. It's a decentralized file system and he got it into the radar of Janus VR and they were the ones who I think were the first virtual reality system that had really implemented IPFS so that you can potentially host your entire virtual world within a distributed file system using IPFS and Janus VR. And Taco Dog was one of the big catalysts of that. But he also was carrying around, it looked like a USB stick around his neck on a necklace, but it was really like a Wi-Fi router where you could put an SD card in it and you could basically broadcast this virtual world. And so you have this concept of a decentralized mesh network where you can start to go into these virtual worlds, but only if you are in physical proximity of something like this Wi-Fi router or this mesh network node that he's carrying around his neck. So we'll be talking about like the decentralization efforts on the metaverse and mesh networks and decentralization in general on today's episode of the Voices of VR podcast. So this interview with Taco Dog happened on Thursday, August 2nd, 2018 at the Decentralized Web Summit in San Francisco, California. So with that, let's go ahead and dive right in.
[00:01:55.871] Taco Dog: Well, I'm basically just a mad scientist of protocols and ways that we can make virtual reality, especially when it applies to the immersive spatial web, seamless and also a fun kind of experimentation zone. And so I published a blog, Metaverse Lab, in which I've documented 3D scanning, neural networks, peer-to-peer mesh networks, ways that people can essentially take what's out there on the open web and build a new internet. And I've been using Janus a lot for that because it's the fastest way to experiment and prototype.
[00:02:38.677] Kent Bye: Great. So maybe you could talk a bit about when did you first have the idea of this decentralized metaverse?
[00:02:45.295] Taco Dog: Well, I think really when it comes to just the metaverse that we envision in Snow Crash and whatnot, I was always interested in like, okay, what's it going to be built on? And I don't want to learn Unity or Unreal when I got my DK1 and what was there provided for a person like me, which is... comes from like computer security or just wants to express myself. What really came to me was Janus VR which I saw a screenshot of a couple of people in the Tuscany experience and that was one of the earliest DK1 experiences where They were through a portal in this space, and I realized this could disrupt the whole App Store kind of model, these walled gardens. And this guy, James McCrae, is building essentially the internet, but spatially from scratch for Linux, which is my favorite operating system. I started experimenting a lot with that and just building out websites and he made it super simple to get my first worlds going and it was really a bad world it was just like a 360 photo but I made something and I could share it and that's what got me really interested because it's really just quick path to doing that but What really got me hooked was also learning about IPFS and I started looking into that around 2015 and digging more into building out prototypes later in 2016. Seeing that it just works really got me thinking about how you can scale essentially kind of like a seed from if someone wanted to publish their experience but didn't want to pay for server or hosting costs and whatnot. like they can share that story and the community decides and like just like how BitTorrent works the more it gets like spread the faster it loads and then more and more you can put into it and so when we're trying to scale it out like not only does it make it more redundant and resilient but it also makes it more performant and You can also just see how this can be better when three years, when the fidelity is so high that we need to have gigabits going down the network. How are you going to do that with everybody's devices? And if I'm right next to you and I want to share my world, how do we do that without going through a person in the middle? And IPFS just solves these problems that really make sense. If I have this file and I want to share it with you then we can just grab it locally peer-to-peer.
[00:05:10.173] Kent Bye: So with the IPFS, I know that talking to some of the developers, they have a JavaScript client that they are able to serve out files from a browser. And so you basically have a URL or an identifier for where it's located. And then it can find that hash and then go out into the network. And I guess, to some extent, you can think of it as a BitTorrent, where it finds all the different people that have seeds. And then it can try to load it from multiple locations. So I guess the question is whether or not Janus is able to actually, once it's downloaded, then become a node within the IPFS network of distributed files, and what control you have to be able to say, oh, actually, I don't want to be broadcasting this out and kind of seeding it, or if that's sort of happening automatically, or if it's even possible to serve it out from Janus.
[00:05:56.852] Taco Dog: Yeah, what I noticed while working with it is how potentially, I mean this is very experimental of a protocol and how we want to take steps so that if you want to publish something and suddenly you don't want it but it's so distributed now, how do you deal with that? Or like something like where a version is unsafe and you publish it and people are still like ceding a vulnerable version of your creation. Well, IPFS has a solution called IPNS, which is their means of basically pinning a hash to something that is immutable. And that kind of solves some of those issues.
[00:06:36.563] Kent Bye: So it's like an intermediary step where you could have maybe a human readable or some sort of pointer, and then you can change it and update it? Is that the idea? Like you could say, go to this pointer, and then if you want to update it, it will sort of take whatever the latest version is, and you would update that hash?
[00:06:53.613] Taco Dog: Yeah, essentially that. I mean, instead of having a mutable hash, because it's all content-based, and so if you change one thing, then you suddenly have a totally different hash, and it gets really unmanaged very quickly, and a lot of links die, because it's constantly changing. But, yeah, IPNS really does solve this problem. One immutable string that you're known to, and that can be your portal on the street, for example. And so, if someone wanted to visit your world, then they know that it's just like, just go through this portal. And this, another thing with the whole spatial web is, you know, you've got this kind of geolocative understanding of where things are, and so you can traverse it just by Like, oh, it's kind of like around Main Street and down the street from the Black Sun or something, you know? And so that could be a way that we can remember, identify where a certain continent is, even so, like, we don't have to think about hashes at all. Just, like, where on a map it could be.
[00:07:53.770] Kent Bye: So we're here at the Decentralized Web Summit, and I know that you've been taking some time capsule recordings of this space. Maybe you could talk a bit about your process of documenting spaces and making 3D capsules that you can put into JanusVR.
[00:08:07.982] Taco Dog: I love doing this at any kind of physical event that I go to, and I've been experimenting with 3D scanning Tango and 123D Catch for since the Tango dev kit came out. Essentially just going around the conference, taking 3D scans of Setups and areas and they have a lot of gaps and and whatnot But that's fine because like after a memory or an event like all we're left are these kinds of pieces Recollections of our memories and that forms like a time capsule that for one event I went to a hacker conference and I took a bunch of the pictures and I ran them through a glitch algorithm and and then I applied some seamless filters to make it so that I can wrap it around a sphere and then fade those images together so that it's like a cyberdelic sky of digital memories that are pictures I scraped from Twitter hashtags and anything that was related to that and so you got kind of like a time capsule crystal ball that you can kind of return to afterwards and it becomes a meeting place so the content Yeah, I can have some recordings and stuff connected through portals, but you can create a space for the community to have a place to come back to, but that hasn't happened yet.
[00:09:26.419] Kent Bye: And you mentioned decentralized and distributed mesh networks. And I know that you have a little thing around your neck that would allow you to broadcast a world without having to connect to an internet. You'd be able to have Wi-Fi and be able to jump into a virtual world just by being physically co-located with someone. So maybe you could talk a bit about where you see this technology of what you've been able to create with the mesh networks, but where it's sort of going as well.
[00:09:53.565] Taco Dog: Yeah, that's one of the things that I've been interested in even before I got into virtual reality. And I realized this one night, these two things like just kind of work. So when it comes to this note that I'm wearing, it's just an open word router. And you can't always rely on Wi Fi or, you know, when you're going into web VR worlds, like you're downloading sometimes 100 megs if somebody has a video playing and it's not on YouTube, which T-Mobile won't count as data anymore, but who cares about all that like what I see is this note is like I use it for time capsules because this one in particular is cheap enough that it can be built probably less than 20 bucks now, but that'll go down in price but When you're in a world, going into a device and opening up a browser and going to that URL are points of friction. And I saw this as a way of reducing friction by just connecting to a Wi-Fi and opening the browser and you're in the world. That way it also has a magic feeling when you're connected in that world with other people simultaneously that It can only be there. So this is content that's kind of tied to a location because it was inspired by like dead drops. It's interesting in a sense that this data, when you're in VR and holding that thing that is kind of like carrying you around, it's a trippy feeling. And I think that's what got me really excited about it because people think of this web, it's a really abstract kind of thing and it's become so big that it's escaped our comprehension. And this way we see and we kind of understand like it's coming from here, just devices. This is a computer and I'm holding it in my hand and talking to that and it's a way for people to kind of understand just how we can build a new internet.
[00:11:50.748] Kent Bye: So you have this node around your neck that is broadcasting Wi-Fi that has available on this node of WebVR world so that somebody could jump into a virtual world while being around you physically. And that reminds me of something like Ready Player One when they're going down in the private basement or someone. just having ability to go and hang out. But this is also connecting a virtual world to a physical location, which would mean like if there's some sort of like place that you'd have to get into physically, then once you get into that physical place, then you have opened up to you these whole other virtual worlds that could somehow either be an augmented layer on top of that world or a virtual world that is otherwise inaccessible.
[00:12:32.233] Taco Dog: Right, yeah, I've been thinking about it in a sense of AR as well, and one of my favorite books comes from the Blue Ant trilogy, William Gibson's, and there's some cool locative art in there, where on Tuesday night, the Decentralized Web Summit hosted a silent disco party. This is just a silent disco, but for AR and VR.
[00:12:56.354] Kent Bye: I see. The silent disco where you're basically having a dance party, but you have headphones. And if you don't have the headphones, you can't hear it. So it's like this ability to sort of go into something that is very sort of ephemeral and context-based. And what do you think the potential of that is, or what could be possible, or why people would want to do that?
[00:13:13.473] Taco Dog: I think there's two really big main inventions for the content to provide for virtual reality is live streaming and internet browsing. And this kind of combines both worlds. When it comes to live streaming, We're going to have the infrastructure to provide really low latency bandwidth in the future once that infrastructure is up. But essentially the potential, it's hard to imagine because I'm just trying to investigate the infrastructure that we can create freely and frictionlessly anytime. feel like god-like in this virtual world and be able to express ourselves however we want and kind of bring out new forms of expression that I'm just trying to plant seeds for.
[00:14:01.935] Kent Bye: So in terms of the protocols for the metaverse, there's the WebXR specification, there's self-sovereign identity, there's cryptocurrencies out there for currencies, and then there's GLTF, there's a number of different open standards. What are some of the things that you see are maybe the gaps or the biggest things that need to be figured out when it comes to the protocols for the metaverse?
[00:14:26.292] Taco Dog: So I believe that all these protocols are really great. And one of the things that I believe that the early internet pioneers failed on, and what's happening again in this cycle, is that these groups aren't really collaborating close enough with each other. It's not enough that you're part of this group and that this initiative sounds good to you. We need to have a channel that I think WebVR provides a really neutral common grounds for. to meet, share, present, teach, and be able to collaborate even. And even if you're working on a completely separate framework, there's no type of collaboration or software platform that beats physical places. So if we can take What's unique about physical places and bring that into the web VR space then we can solve that distance where it's Silicon Valley people kind of determining protocols but we can decentralize the whole location and distance problem and get people working closer to each other to achieve some kind of collective ideal outcome.
[00:15:32.058] Kent Bye: Well, there's a lot of things that are hurdles when it comes to being a programmer and being able to create things, but also get things to work in a decentralized architecture. So why should people bother when the user experience is so much better and centralized? Why is it important for people to go through the struggle of figuring this out?
[00:15:49.725] Taco Dog: So the question is, why is it worth the challenges? I mean, we're solving friction early on. We're the pioneers of basically making it so that we can tackle these challenges. It gets easier over time once we solve how we can make certain things better, like how we can make our meetings more effective, or how we can use this to brainstorm, or plan, or pre-visualize. Our area at the Decentralized Web Summit 3D modeled beforehand, which gave us a lot of reassurance in like, yeah, we could bring this and this would look really cool in there. And you know, what kind of things we can do. And it really helps to plan out. And also for afterwards, we can then give that out and share it. And so it becomes something of like a time capsule from the planning process itself. And that can scale out to anything.
[00:16:46.510] Kent Bye: Great. So what are some of the either biggest open questions that you're trying to answer or problems that you're trying to solve?
[00:16:54.212] Taco Dog: I think a lot of it kind of drives down to human issues. But when you're working on a decentralized virtual web, and this thing's been brought up to me from the person who is building out the WebVR side, WebGL side of Janus. you've got this thing and it's really cool and you're excited about the new futures and all the possibilities of the thing that you're working on that you don't want to pay attention to the other things people are working on and so like that's the difference I see with people are kind of doing on Babylon and A-Frame and all these others we're kind of building out these sand castles and we're not really collaborating on like I've taken steps to make traversal from Janus into other platforms really seamless and that's one of the things that is going to be necessary for the open decentralized metaverse. So I think Virtual Reality Blockchain Alliance is a really good place in terms of their initiatives to make object and avatar seamless traversal between worlds and also a bill of rights for avatars and ethical advertising models. The browser wars, I see as another thing in terms of an issue for adoption. I'm really fond of Brave Browser nowadays because they're solving a lot of issues when it comes to the web, like tracking actually, it eats up your browser memory and you're downloading all this extra stuff that just, it's not gonna be good if you walk into a world, if you were to visualize just like how many trackers are tracking you in a website, like Lightbeam does and Mozilla Firefox add-on, it's scary. I don't want that to persist going into the virtual world where literally like every biometric data is just being like siphoned off me into like a DataCorp USA. I want transparency in that and I think that visualizing that data is going to help people see and also kind of scare them into like we're going to need to rethink the internet and like our advertising models which is going to be a big challenge, because right now, the top three are just dominating the internet. And we're going to have to find ways for the UX and decentralized apps and organizations to compete against that.
[00:19:09.708] Kent Bye: So what type of experiences do you want to have in VR?
[00:19:13.889] Taco Dog: I want to have experiences where we can come up with an idea and prototype it on the spot. I've always been interested in the maker movement and how these community organized spaces, these hacker spaces are have become like a new kind of English coffeehouse that predated the Enlightenment period. And I think like when it comes to virtual reality, I've always been interested in a non-gaming potential of it and how people can use it to work and collaborate better. So I'm interested in ways that we can bring smart people into a world and not just be there to socialize, but to be productive and do that in a frictionless manner so that it's just on and off. And then from that step, we can create more fun things, like we can plan an event, or we can create a game. So like people are working on blockchain gaming or something, maybe they can collaborate with their remote workers or something.
[00:20:11.532] Kent Bye: You have so many different projects that you've done and kind of tinkered around in virtual reality. I'm just curious if you have any other favorite ones or ones that you think are kind of interesting in terms of technological achievements that you've done or experiences that you've had that you've been able to put together within VR.
[00:20:27.981] Taco Dog: Yeah, I think one of the ones that I really haven't mentioned is just like how I began really playing around with style transfer and neural nets and how that really blends in this infinite creation like you can 3D scan and then you can get your neural net to turn your scans into a painting and then And that really just gives you this feeling of unlimited power. I can turn this into macaroni world and then instant spiders the next. So I'm really excited about that potential. And with TensorFlow.js, the possibilities are endless. DUSTIN KIRKLAND, JR.:
[00:21:00.605] Kent Bye: : Have you started to do any sort of neural network or artificial intelligence within a virtual world dynamically?
[00:21:08.213] Taco Dog: Not so much as, well, I'm on my computer remotely in a virtual world, and then I can just open a Linux terminal and then do stuff from there and create the world from the inside out. But in terms of neural net, it's not really tied into that. Just through my Linux terminal from there, it is.
[00:21:28.670] Kent Bye: Great. And finally, what do you think is the ultimate potential of virtual reality, and what am I able to enable?
[00:21:37.072] Taco Dog: I think like the ultimate potential is really just how we can evolve as humans and build just a kind of a higher level of existence and I think it's gonna be a challenge but In doing so, we're creating really awesome use cases for peer-to-peer technology and getting people excited about building a new internet that we can code our values into. And in this way, if we save the internet and keep it open and free, we're going to ensure that we keep an open and free society.
[00:22:13.321] Kent Bye: And because we're here at the Decentralized Web Summit, I'm just curious to hear your thoughts on this intersection between VR and these decentralized technologies.
[00:22:24.070] Taco Dog: Yeah, so cryptocurrency is something that incentivizes open source developers to work on open source projects, and that's always been the problem with open source software, which I see for the past 25 years, there's been an interesting parallel between virtual reality, the worldwide web, and Linux. And that's the story that's still continuing today. And now the whole world, in terms of web infrastructure, is running off Linux. It won in the end. And so I think, what is going to be the Linux of VR? That's what we're trying to figure out here.
[00:23:00.305] Kent Bye: Awesome. Great. Anything else that's left unsaid that you'd like to say to the VR or decentralized community?
[00:23:06.630] Taco Dog: Consider checking out the VRBA and Janus. They've always been about the open metaverse and figuring it out as a group. And we can solve these problems together. You don't have to take it on as if we're alone.
[00:23:21.740] Kent Bye: Awesome. Great. Well, thank you so much for joining me today. So thank you. Thank you. So that was Taco Dog. He's got a blog about the Metaverse Lab where he explores a number of these different experiments that he's doing, both with creating time capsules and the decentralized Metaverse with the Interplanetary File System, the IPFS, as well as creating mesh networks. So I have a number of different takeaways about this interview is that, first of all, So the decentralized metaverse is something that I think is at this point, it's an unknown as to whether or not there's going to be enough of the technological things to come together. My sense is that there's enough movement and initiative with enough people that are thinking about this that really want to try something different. It's still an open question as to whether or not it's still going to work. But At this point, we have somebody like TacoDog who is able to essentially get JanusVR to do an implementation of the IPFS so that right now you could start to create something on JanusVR and have it hosted out there on the distributed web. Now, the big thing that came up over and over again about IPFS is that it's kind of like BitTorrent, which means that if it's popular and you get the community to really get into it, then it'll survive for a little bit. up until everybody that had downloaded it decides to no longer seed it or it'll just eventually go offline because, you know, it's just a matter of time before it kind of slowly dies off. And so there's this larger issue of having these different archive institutions, somewhat like the archive.org that is kind of the cedar of last resort. And that's kind of what we really need. But at this point, if you're going to move away from the centralization, then you have to deal with some of these issues with content being kind of inconsistent and not there forever. It's not permanent. Bruce McHale said that an average age of a website is around 100 days, and I think that you can kind of expect the same thing with the metaverse, although at this point it can be a little bit worse just because the volume is so low that people who are actually downloading and propagating some of these different content is not really at the scale to be able to make it viable. But the concept is that, you know, there's people who don't have good internet connections and that something like IPFS could start to bring these types of immersive experiences to environments that don't have a very solid internet backbone. And also this concept of these, you know, mesh networks and these little Wi-Fi routers that you have around your neck, I just made me think about something like an immersive theater experience where you get into this kind of secret access to this physical location. And in that location, you have this ability to go into a virtual world. And so you have this ability to overlay a virtual reality and augmented reality on top of physical reality. And if you have something like the concept of a dead drop or something where you're kind of having a level of information security to be able to transfer information back and forth to each other, then you have this ability to kind of create these, you know, I guess like secret societies or these exclusive types of experiences. I would imagine that if people don't want to have all of their movements tracked and recorded on a centralized server, and then they want to just have some privacy for whatever reason, they want to just, you know, not be tracked or overheard or having their privacy invaded, then these types of mesh networks could start to then really catalyze that. Now, I think that IPFS has talked about Filecoin and these larger incentive structures to be able to actually make some of these decentralized mesh networks a little bit more viable. Because right now, in order to get the internet, most people use the ISP, which is a centralized choke point. But if you really want to have the ability to go from your computer to anybody else's computer in the world, then you really need to have this robust mesh network to be able to actually do that. So it's still just like really early days before any of this that I've seen like a viable deployment of something like a robust mesh network. I'm sure there's been examples, but within the context of this conference of the decentralized web summit, it's still like super early days. But the fact that taco dog was, you know, carrying this thing around his neck and And just talking about this theoretically, and also, you know, that he's able to actually kind of host these different worlds and that if you did have a virtual reality headset, then you could potentially log into that wifi and start to go to a specific, you know, location that way. There wasn't a working demo of that. So I didn't get a chance to actually have an embodied experience of that, but theoretically all that is possible. And just the other thing that Taco Dog was doing was just taking these captures of the surrounding environment with his Tango phone and being able to do these depth captures and to be able to create these photogrammetry type of scans so that he could start to capture these memories of these memory palaces of these different events and kind of archiving 3D physical spaces. And so that was just another thing that he was kind of hacking on and working on and showing off while he was there at the Decentralized Web Summit. So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast, and if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listeners-supported podcast, and so I do rely upon your donations in order to continue to bring you this coverage. So, you can donate today at patreon.com slash voicesofvr. Thanks for listening.
