Jesse Joudrey of Jespionage Entertainment talks about the weekly VR Chat gatherings that have been happening, and how meeting people in virtual spaces compares to meeting them in real life. He talks about the what type of body language cues translate, and how they’ll be expanding that with hand gestures.
Jesse also talks about the process of stress testing gatherings in VR Chat, and the current bottleneck is when too many people speak at the same time. They’re currently limited by what the uSpeak Unity plug-in provides, but they can also host additional servers to help out with the load. He also talks about how the VR Chat SDK provides the ability to customize your own avatar look with animations as well as customized spaces. Gunter has also gathered an archive of past VR Chat meet ups.
Finally, he talks about his VR Game jam game of Snow Drift, which is an extreme sports game. He was surprised that it made a lot of people motion sick because he doesn’t suffer from any symptoms of simulator sickness, which he talks about as well.
Reddit discussion here.
TOPICS
- 0:00 – Intro to VR Chat
- 0:30 – The VR Chat experience. Almost like being in reality in that you’re meeting other real people from around the world.
- 1:13 – Body language that translates. Where people are looking. Hand tracking is coming. Fidgety people also move around a lot in VR
- 2:00 – VR Chat stress tests. How to measure bottlenecks. Simultaneous in-voice communication
- 3:12 – Different challenges with audio. They use the uSpeak Unity plug-in. Can add servers to help with the load. Lots of data is being pushed when people speak simultaneously
- 4:15 – VR Chat SDK – Import avatar into the chat. Create own animations, and their own meet-up spaces. Build environments to express themselves.
- 5:30 – Can people host chats on their own server? Chats pare
- 5:56 – What it’s like meeting people face-to-face after communicating with them in VR Chat.
- 6:38 – What’s next in VR Chat? More personalization.
- 7:16 – Snow Drift, extreme sports VR Jam game.
- 7:53 – Doesn’t get motion sickness.
- 8:39 – Get more info at Snow Drift and VR Chat and Jespionage Entertainment
Theme music: “Fatality” by Tigoolio
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:11.974] Jesse Joudrey: I'm Jesse Jodry. Online, I go by Jespionage. And VRChat is a VR social application where everyone just shows up and we meet and we have avatars and we hang out and we talk for hours, usually on Sunday evenings at about 5 p.m., but also for Europeans, Monday at 8 p.m.
[00:00:30.936] Kent Bye: I see. And so maybe talk a bit about like what's the experience like when you go and what do you see and what type of experience do you have?
[00:00:38.260] Jesse Joudrey: Being in VRChat is almost the same as being in reality. You show up at a coffee shop to hang out with your friends, you see your friends, you talk to your friends, and you have all the same experiences you would have meeting people in real life. In fact, it's some of the most compelling virtual reality experiences we have so far because the fact that the other characters in your environment are actually other people from around the world, you automatically feel their presence in the room, and their presence in the room affects your presence in the room, and you feel like you are in a real place talking to real people.
[00:01:13.815] Kent Bye: Yeah, maybe you could talk a bit about the head tracking and the body language that you're able to pick up in this environment.
[00:01:20.248] Jesse Joudrey: So it's kind of funny, there's some obvious forms of body language are where someone is looking, who they're paying attention to, and you can see that. We're also going to add hands to allow people to speak with their hands and gesture while they're talking, hopefully pretty soon. But there's another interesting bit of body language that happens in there as well. If somebody is a fidgety person, If somebody can't stand still when they're talking to you in real life, they also can't stand still when they're talking to you in VR chat. And you'll see their avatar move slightly from left to right and forward to back as they play with the keys. And this kind of non-involuntary body language coming out in virtual reality was something that totally surprised me.
[00:02:00.044] Kent Bye: And maybe you could talk a bit about the stress tests that you have in terms of inviting people to come in and kind of push the limits of the system.
[00:02:06.587] Jesse Joudrey: So we post on Reddit every time we're going to have a stress test. We basically invite anyone, whether they have a Rift or not. If they have one, that's great. If they don't, we take enthusiasts and they have a good experience as well. And basically we just try to get as many people into the room as we possibly can and strain our servers and strain the client and see what kind of great experience we can have.
[00:02:27.827] Kent Bye: Maybe you could talk a bit about the bottlenecks and how you measure when things are getting too stressed.
[00:02:32.979] Jesse Joudrey: Well, it's interesting. Anecdotally, we had a meet-up on February 22nd or 23rd, I can't remember exactly. It was right after Bruce's stream, and I had just turned 40. And Bruce had sung me happy birthday on his stream, it was awesome. But then we sang happy birthday in VRChat as well, and right near the end of the song, people started to blink out. as the server got overloaded with 20 people singing simultaneously in voice communication. And so that was the most interesting kind of bottleneck stress test that we'd put on up to date. And that's one we continue to work on. We changed the speech SDK since then. Or not the SDK, but the codec. Yeah, so we continue to improve it that way.
[00:03:12.578] Kent Bye: I see. And so, yeah, maybe you could talk a bit about the different challenges with either multicasting audio. Like, how is that actually physically working?
[00:03:21.483] Jesse Joudrey: We used a package called YouSpeak from the Unity Asset Store, and we basically let them take care of it.
[00:03:28.089] Kent Bye: I see, I see. So are there any other custom developed aspects of their code that you're able to tweak then?
[00:03:34.755] Jesse Joudrey: Not really.
[00:03:35.135] Kent Bye: I see. Okay, so you're basically creating a Unity plugin and creating a binary that people are then entering all at the same time, and you're kind of at the limits of this plugin, is what I'm hearing, I guess.
[00:03:47.941] Jesse Joudrey: So we have some plans to add a European server, for example, and that will, we believe, greatly improve the experience for our European audience. And we can do that in many other locations. We can do an East Coast and a West Coast server. But the fact is, we're just pushing a lot of data when people speak simultaneously. So there's nothing we can do to kind of avoid that. We're already using a fairly efficient codec, and that's about the limits of what we can do there and still have people talking to each other.
[00:04:16.389] Kent Bye: And so you had mentioned earlier a SDK that you're working on for VRChat. Can you tell me a bit about what's involved in that SDK and what that will allow people to do?
[00:04:25.012] Jesse Joudrey: Sure, the first version of the SDK that we released was simply a custom avatar package all by itself and people in the audience were able to import their own models and use those models as their avatar in the game. Reverend Kyle comes as the Incredible Hulk and he's very impressive and he towers over us. But we had another character recently who decided that they wanted to be taller and that arms race went on for a couple hours while people went off and made their own avatars. But since then, we've released the first version of the full SDK, and the first version of the SDK allows not only people to create their own avatars, but also to use their own animations on those avatars, and to have their own control of those avatars using keys. and to import their own spaces so they can host online their own meet-up world and allow other people to join it without the other people needing to download anything in advance or anything like that. So we're building environments for people to express themselves in both with their character and with the place that they hang out.
[00:05:31.118] Kent Bye: I see. And so what is involved in terms of hosting your own location? Does that mean that they're able to host it on their own servers and their own infrastructure?
[00:05:39.994] Jesse Joudrey: So the game still connects to our central server. However, the data for their own world is on any web server they choose. We don't have any kind of centralized control over that. They can host whatever they want in whatever kind of genre and content they want. That's up to them.
[00:05:56.800] Kent Bye: And so since you've been having these social interactions in virtual space, can you talk a bit about what's it like to be here at SVVRCon where you may be meeting a lot of these people face to face for the first time?
[00:06:06.943] Jesse Joudrey: It's totally fascinating because a great number of us don't look anything like the default set of avatars that the game comes with. Matt Stomps, aka Matt Carroll, is actually pretty tall. Who knew? And Rev Kyle doesn't look like the Hulk at all. And that's exactly it. But the funny thing is that I've heard their voices so much that a lot of the connection is still there. So when I speak to Matt, he doesn't look the way I expect, but he sounds exactly the way I expect. So I still get to have the rapport with him.
[00:06:35.978] Kent Bye: I see. And so what do you see as like the next frontier of the VR chat? And what are you kind of waiting to see happen?
[00:06:43.390] Jesse Joudrey: Well, the SDK is going to allow people to bring a lot of their own personalization into it. So what we want to do right now is we want to get that SDK in front of absolutely as many people as we can and have them start building their own spaces. We did a version of one of our meetup rooms for the European meetup that has a EU flag on the table just for them. And it's stuff like that that allows people to meet where they want to. And we think that when that gets out into the wild, it'll greatly increase the amount of use people get out of it.
[00:07:17.074] Kent Bye: And you also developed a game called Snowdrift. Maybe you could tell me a bit about that game.
[00:07:21.337] Jesse Joudrey: Snowdrift was from the VR Jam last year. Lots of fun. It's an extreme sports skiing game. And I feel absolutely no motion sickness in the rift, so I had no idea that it was going to affect people this way. But since then, I've backed off. There's a normal mode and an intense mode, where intense used to be the only mode. In normal mode, you can't fall down and you can't do any flips or anything extreme like that. And it's a much more comfortable experience. It's available in Oculus Share. Go on, play it, like it, rate it highly.
[00:07:51.012] Kent Bye: And so what's it like to just not get motion sick? You can basically do anything in the Rift and you don't feel anything?
[00:07:58.314] Jesse Joudrey: Pretty much, yeah. I'm really looking forward to playing Rev's catapult game. I can't remember what it's called, but it's Human Cannonball or something like that. He promised he'd send it to me, but I haven't heard it yet.
[00:08:09.058] Kent Bye: I see, so like in the roller coasters or anything else, when you move your head around, you don't notice any of that latency, and it just, you don't ever feel the motion sickness then?
[00:08:17.406] Jesse Joudrey: No, that's right. The most I've ever gotten was a little bit of vertigo, but I would expect vertigo with some kind of heights, with the heights that I was experiencing.
[00:08:25.444] Kent Bye: So a lot of people, when there's a rough cut and you're kind of cutting between scenes, is that disorienting to you as well or not?
[00:08:33.634] Jesse Joudrey: It's strange because I don't necessarily know what I'm looking at right away, but it doesn't affect my stomach.
[00:08:39.581] Kent Bye: Great. And finally, where's the best place for people to track you down and to learn more information?
[00:08:44.904] Jesse Joudrey: Well, they can go to Oculus Share to download Snowdrift or snowdrift.jespionage.com. And for VRChat, they can go to vrchat.net. Great. Thank you. Thank you.
