Graham Gaylor & Jesse Joudrey have been working on VR Chat for well over a year now, and they’ve been consistently providing social spaces for some early experimentation with Social VR experiences. They talk about some of the latest features that they’ve been implementing in their platform, and what they’re doing to help make their dreams of the metaverse a reality.
Jesse says that social interaction within a VR experience a feature, not the final destination of a VR environment. More and more people will want to be doing or seeing something specific within the VR experience, and so that’s part of the reason why VR Chat has created an SDK with a set of unity scripts to help easily add social experiences to your VR experience.
They’re also interesting in being able to link virtual experiences together as well and helping to do some early experiments in building out the metaverse using tools like Unity that are capable of creating highly-performant and interactive immersive environments.
One of the requests that people have had with VR Chat was to be able to know when their friends were online, and so Jesse and Graham have been implementing user accounts so that people can have friends lists. They also have a whole system for being able to create rooms within VR Chat, and whomever creates the room has a set of moderator privileges in terms of whether it’s hidden, open, and access only given to a white list of people or friends of the moderator. They’re starting to implement the technological foundation in order to have the type of chat room environments that were described in Ready Player One novel. I think that this is going to be really, really powerful to be able to invite people over to your personalized VR chat environment, much like you might invite people to come hang out at your home.
Some of the other features that VR Chat has implemented was being able to have different types of games and interactions while hanging out socially. For example, Jesse created a race track where you can ride around in a car with four other people while racing other people. But these types of physical proximity constraints start to replicate taking a road trip with friends, but in a way that’s super silly and a bit absurd. Jesse says that one of the biggest lessons and reactions of creating experiences like this is that people tend to laugh a lot more together.
In the future, I foresee that Virtual Reality has the capability to allow us to play with a way that we don’t tend to do in real life. It’s going to break us out of socialized patterns for how we normally connect and relate to each other, and I think there’s a lot of rich opportunities for finding new and fun ways for people to play and spend time together whether they live across the country or if it’s just something that people decide to do within a room-scale VR environment or VR arcade.
For more information, be sure to check out VRChat and drop by one of their upcoming events.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: the Voices of VR podcast.
[00:00:12.114] Graham Gaylor: I'm Graham Gaylor, one of the founders of VRChat.
[00:00:14.934] Jesse Joudrey: And I'm Jesse Jodry, one of the other founders of VRChat. We are the developers of VRChat. We're here at SVVR showing off what we've been up to over the last year since SVVR last year.
[00:00:26.957] Kent Bye: So maybe you could tell me a bit about what's new in VRChat and what have you learned in the last year?
[00:00:32.561] Jesse Joudrey: Really what we're focusing on right now is the ability of users to generate crazy virtual worlds all by themselves. The content that people are coming up with has just been mind-blowing lately, some of the worlds that people have created. So we want to take the capabilities we have for people to build their own content and really expand on them and make sure that whatever you can imagine virtual reality to be is something that you can build.
[00:00:57.842] Graham Gaylor: One of the things I think everybody expects out of virtual reality is some sort of metaverse. There's multiple people doing it and kind of our thoughts on it is that we want to give everybody the ability to add to that metaverse and we've found that Unity is the easiest way to do that, and that's how we build our stuff, and that's how a lot of our friends build their stuff. So we figured, hey, using those tools, we want to make it super easy for people to build their own virtual worlds, their own virtual avatars, and share it with their friends and do cool stuff with their friends. So we're really just focused on making that easier to do. We really want it to be a three-step process to be able to build whatever you want and have it online in under X number of minutes.
[00:01:40.880] Kent Bye: Yeah, so maybe you could talk a bit about that process if, say, I wanted to create an environment for VRChat. Would I create something in Unity and then install a Unity package that you create and then release it as an individual app? Or would it be submitted to you and then approved? And maybe just talk about that workflow and process in order to build your own environments.
[00:02:01.088] Jesse Joudrey: Yeah, so what you do is you download our SDK from VRChat.net. What that gives you is some Unity scripts that you can put into a Unity environment. Then you can take any model in any format that Unity imports and drop it in there as well. And that can be your avatar or it can be an environment that you've created. If all you want to do is explore this space in virtual reality with your friends, you're basically done. And you can package it up using a menu option in our SDK, put it somewhere web accessible, and then it's good to go. And you can join in and go with your friends. But if you want to go a little bit deeper than that, we have some scripts, pre-made scripts, that you can drop onto your environment. and provide some basic functionality, doors, audio, sound effects, animations, stuff like that, to bring a little bit of life to your environment. Now, if you're really into it, what you can do is you can do your own custom scripting and bundle that up into a DLL, and that will allow you to provide the ultimate level of interaction to your virtual world. For example, We have one virtual world which is a racetrack, and all the vehicle physics and stuff like that were custom scripted, so we don't build vehicles into VRChat specifically, we give the user the ability to create their own vehicle physics, their own vehicle system, and then build a virtual world that uses that.
[00:03:23.162] Kent Bye: I see, and so was that something that a user created, this race car track, or was that something that you guys created?
[00:03:29.230] Jesse Joudrey: This one was created by me to illustrate how a user could create vehicles. And just recently, last week actually, we found out that that came to fruition because One of our users took the video we made of how to make vehicle physics and they built a fully working hoverboard for VRChat. So now it's possible for people to fly around on hoverboards in his level and that was built by a user with essentially no interaction from us at all.
[00:03:59.146] Kent Bye: And in terms of the actual social interactions in VRChat, it seems like people are going there to interact. And I'm curious what you've observed happens when you start to throw things in there, like racing around in a car. How has that changed the type of interactions that you have?
[00:04:13.905] Graham Gaylor: I feel like it really, I mean, adding things like that, more gameplay mechanic concepts, just allows people to have more fun. I mean, it does affect the social aspect a little bit. For instance, you can have a driver in a car, you can have your passenger and have two people in the back, so it could be... driving around talking to your friends or usually laughing or that's something that you've done but really it's just adding more to what you can do with each other like any other I guess game or environment or experience but having the virtual reality aspect to do that with your friends in a little bit more realistic manner I think has added a lot.
[00:04:49.470] Jesse Joudrey: Yeah, that's exactly what I was going to say. The thing that happens when you throw this stuff in there is laughter. People wiping out, people going around the track backwards, deliberately trying to knock people off the track. It's more than possible for us to go and prevent that in the scripts if we chose to, but right now it's fun to draw everyone's attention to one thing, because social isn't a platform anymore. Social is a feature, right? So you've got to have more than just social in your social VR application to make it in today's world.
[00:05:16.852] Kent Bye: Now in terms of the actual social interactions, I'm curious some of the things that you've been focusing on to actually increase the immersion or the sense of identity that people have through avatars or other features and face tracking, hand tracking, what are the types of things that you're putting in there?
[00:05:33.994] Jesse Joudrey: Yeah, so we support the Hydras and the Kinect and we believe the STEM already and we continue to implement all of the best solutions for input as they are released. Obviously the Vive is forefront on everyone's mind right now and we can't wait to get that in there as well. But the ability to gesture with your hands adds a lot to virtual reality. And the connect to just shift your weight back and forth. I mean, it doesn't seem like much, but natural shifting of weight back and forth that's timed with your speech adds a lot. There's a lot of discussion recently about eye tracking, and there's no good solution to that yet. There are some solutions out there that say well you should just fudge it and that kind of make the eyes move to Somebody that's nearby or somebody that's in your vision cone, but that works really well in a multiplayer game It worked really well in a one-on-one environment, but in an environment where there's lots of different social stuff going on all at the same time It matters a great deal who I'm looking at when I say something. And people are standing close together. So I think that if you just kind of fudge it and have the computer decide who you're looking at, you're going to generate a lot of false information. And so I'm not exactly sure how to handle the eye tracking solution yet. Hopefully we'll get hardware for that in the next year.
[00:06:50.949] Graham Gaylor: And I guess one of the things that's been kind of cool, and just because we've been around a while and these other social apps are making social VR more of a thing, as the community has grown, the social interaction between everybody has gone in a direction that a normal community in real life would go. You know, the people who have been there longer have become very close. and treat each other as old friends, even though they've never met in real life. And they greet newcomers as, you know, new potential friends. And they want them to come back, and they want to get to know them. So although that's not, it's using the new hardware to increase social interaction, just having the community behind it coming back every week is pretty awesome.
[00:07:29.217] Kent Bye: Yeah, and I'm curious, you've got some photo scans in here, and they're pretty realistic, and there's the whole phenomenon of the Uncanny Valley, where if it looks realistic, you kind of expect all the other social interactions of, like, body language, eye tracking, blinking, all these things. And so, as you have these high-res photos, I'm curious if you've experienced that creepiness factor, or if the avatars that tend to be a little bit more low or middle fidelity tend to be a little bit more immersive or accepted.
[00:07:58.018] Jesse Joudrey: We definitely do notice it. I mean, that's the thing about having like this is one of the first high-quality photo scan avatars in virtual reality that I'm aware of and it does Cross the line in some ways and as funny as I'm just gonna say is that the eyes are dead, right? But this is progress. So we put this in and we measure the reaction we figure out what needs to change on it and and we push forward. And obviously, like right now, it doesn't even lip-sync like other avatars do. That's gotta change. But we take these steps to try and push it forward as much as we can. I still think there's a market for realistic and humanoid and emotive avatars. I think there's also a great market in the abstract, you know, floating heads or anything like that. But I think people demand both of those solutions, and I'm happy to provide this one.
[00:08:49.362] Kent Bye: And so, yeah, I'm curious about some of the other, like you mentioned lip syncing, how does that work?
[00:08:54.925] Jesse Joudrey: Well, we have fairly simple lip syncing. We basically do the amplitude of the audio that's coming out so that it remains actually in sync with the audio as people are speaking. We don't do proper vizem or phonem analysis of the audio, but we get pretty good out of just moving the jaw up and down from the mouth with the amplitude. I mean, this is the thing about social VR that everyone was kind of surprised about at first, is that its simple cues are actually decent. The thing about Simple Jaw Flap is it's actually something that'll draw your eye when someone is speaking. It compels you to look at their face as you would in reality. Even with Simple Jaw Flap, it's something your eye can process and your eye is drawn to it, and so it adds to the interaction.
[00:09:39.141] Kent Bye: Great, and maybe you could talk a bit about some of the other features that you have on the horizon or what you're excited to have people start to dive into in terms of social VR.
[00:09:47.723] Graham Gaylor: Sure, so one of the big things I'm excited about is our web portal. Up until probably tonight or tomorrow, we haven't had user accounts in VRChatbase. So you can log in, you can have your username, somebody else can take your username, which was nice, and it let people easily use the app. But there was this concept of not having my own avatar, although people did create their own avatars and uploaded them and were able to use them. There's no sense of, I own this thing. So one thing we're really working towards is having this concept of my VRChat inventory, where I have all my avatars, I have all my world blueprints. Then we have a web portal coming where you can easily upload, share them with your friends, and then access them directly inside of VRChat. So you don't have to jump through any adding URLs into the app. Basically, we just want to make things easier from one end to the other of the app.
[00:10:37.669] Jesse Joudrey: Yeah, we had a demand, fairly constant demand from our user base that they say, well, I want to know when my friends are online. And it's like, well, we don't even have friends lists. And it's like, well, I want friends lists. And it's like, OK, well, we need accounts for that. So I mean, there's a lot of things, such as the access to, like knowing which avatars you have access to, knowing when your friends are online, when events that you've expressed interest in are running, so that you can pop into the social VR experience at the right time. I mean, we're really looking forward to providing people with that.
[00:11:08.933] Kent Bye: Are there, like, personal VR chat rooms as well? If, you know, you see a friend and you can kind of go off into your own one-on-one room? Or is it all sort of in an open world where you have to kind of be in an environment with other people as well?
[00:11:21.855] Jesse Joudrey: No, we definitely have lots of different options for personal rooms. You can hide a room and nobody can see it on the list. You can put a password on there so that only you and your trusted compatriots can get into it. If you open a room, you get some moderator controls to kick people out or to mute them or to prevent people from opening portals in that room to other rooms. Lots of different options like that and we just keep pushing it ahead.
[00:11:46.445] Graham Gaylor: With accounts, too, what'll be cool is you'll be able to, let's say you create your own private room, you'll have your own whitelist of, oh, only my friends can come join in this. So you don't have to worry about a password, you don't have to worry about hiding the room, it's just, hey, this is my place, my friends are allowed in, no one else is. So that'll be really cool.
[00:12:01.811] Kent Bye: Yeah, it's like Ready Player One coming to fruition here. The basement, yes. And yeah, I guess any other final thoughts in terms of what you get excited about in terms of the future of social VR as well as telepresence?
[00:12:14.893] Jesse Joudrey: I think that, like everyone in social VR, the thing I'm most excited about for the coming year is the appearance of a market. Getting more people into VR experiences is really important because right now, in any social VR application, there's not somebody online most of the time. And so when that explosion of new users comes this year and people start using it more and more frequently, it's the network effect. It'll scale exponentially and it'll become much more engaging.
[00:12:47.725] Graham Gaylor: I'm really excited to continue to see what the community builds. We build a few worlds and avatars to show what's capable, but really, every day I come online and see three new worlds that somebody has built, and I get the chance to go explore them with or without other people, depending if they're people online, and they just continue to blow me away. Each one is better than the previous, and for me, that gets me excited and drives me to continue to build those tools and make them even easier so we can see even cooler rooms.
[00:13:18.637] Kent Bye: Awesome. Great. Well, thank you.
[00:13:20.059] Graham Gaylor: Thank you. Thanks.
[00:13:21.962] Kent Bye: And thank you for listening. If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash Voices of VR.
