I interviewed Dirk Van Welden, Studio Lead and Creative Director at I-Illusions, at Meta Connect 2023. See more context in the rough transcript below.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the future of special computing. You can support the podcast at patreon.com.voicesofvr. So this is episode number 11 of 12 of my coverage from MetaConnect. And today's episode is with Dirk van Welden, who is of iIllusions. creator of Space Pirate Trainer, which was a launch title on the Vive, it was a launch title on the Quest, and is now working on a mixed reality game called BAM, which is a multiplayer game that was showing there at MetaConnect. So I wanted to sit down with Dirk just because he's kind of an OG of the VR developers involved with a lot of the innovations of virtual reality, creating, you know, really compelling shooter that's out there and yeah, just has had a huge impact on the development of virtual reality as a medium. And so I'd be very curious to hear some of his thoughts and reflections of being one of the first developers to be launching a mixed reality title. That's going to be launching with the Quest 3 of BAM, which is more of this Multiplayer experience, I think, you know, part of the dynamic is that it's going to be an experience that is for people to be co-located and it's going to work both on the Quest 3 as well as in the Quest 2. Quest 2 only has black and white passthrough, whereas Quest 3 has color passthrough. Yeah, just wanted to get a sense of his journey into the VR space, a little bit of a historical reflection on the early days of room scale VR and now looking into the future of mixed reality, you know, with the augmented reality overlays and, you know, what gets him excited around this going back into this LAN party feel of bringing people over for these different gaming experiences and trying to unlock the fun and some of the more embodied gameplay that can have the tabletop aspect of the mixed reality, but also have some embodied movements that you can play with each other and really emphasizing the different social dimensions of the interactions that are happening in the context of these mixed reality experiences. So that's what we're coming on today's episode of the Voices of VR podcast. So this interview with Dirk happened on Thursday, September 28th, 2023 at MetaConnect at Meta's headquarters in Menlo Park, California. So with that, let's go ahead and dive right in.
[00:02:20.147] Dirk Van Welden: I'm Dirk van Welden from iIllusions and Foampunch. iIllusions is known for Space Pirate Trainer, and now BAM, which was just announced at MetaConnect. I've been an original backer for the DK1. It was a launch title for Vive when it was released, a launch title for Quest, that's always Space Pirate Trainer. And then we did the DX version with Arena, and it's a 10x10 multiplayer game. So now BAM, it's an AR game, a multiplayer party AR game.
[00:02:49.320] Kent Bye: Awesome. And maybe you could give a bit more context as to your background and your journey into this space of XR development.
[00:02:55.845] Dirk Van Welden: Yeah. So I'm an engineer. I worked in the music industry, actually. And while I was working in the music industry, my hobby was game development. My first game was a mobile game, I guess. I released back in 2008, 2009 or something. And I worked really hard on an indie game called Elemental, which was released on Steam in 2012. This was my first moderate hit. But with those connections back in 2012, I got introduced to people at Valve, back when it was without green light, so you really had to apply to get your game in there. And when they announced the Vive headset, I was so compelled by it, especially with the 6 DoF, because I tried the DK1, I was getting sick with just rotation, so I really liked the 6 DoF. And I wrote an email to Matt Nickerson, who was my DevRel at Valve, and without him replying, I just got a Vive dev kit in my mailbox. I was super happy. I dropped everything because there was only one test dev kit available. I was working on a snowboard game back then. And I dropped everything, put it in the fridge, I just wanted to work one month on something else, and actually in that month I made the prototype of Space Pirate Trainer, put it on the forums for free. I think people at Valve were actually using it as a demo, and then they invited me in January, like one month later, to demo it to press. And I was like, dude, but this isn't a game yet, like, it's a demo. Yeah, you don't have to change it. I spent like two more weeks working day and night to just make a game out of it, and then I demoed it somewhere in Seattle together with a lot of the other people, like Job Simulator, Tilt Brush was there, Arizona Sunshine, a lot of those people. And then they just said, everyone was like, we don't know what's going to do VR, but everyone's pouring their souls in it, so I kept on working on it. In the end, yeah, it did really well. It was one of the top selling games when the Vive came out. And I was so compelled by it, so I kept on working on Space Pirate Trainer just because of the success for like two years, set up a lot of free updates, ported to different platforms as well like PlayStation VR and Quest 1, and integrated chipsets for Microsoft back then when they released their headsets. Yeah, and then we released, I think, the arena version, but yeah, it was so hard to set up and we actually thought Hardshore was going to catch up with what we were doing. It was so hard to set up that people weren't really playing it a lot. And I get it, I know it's hard to set up and you can't play it outside so that you find like a spot without sun or at dusk. And so I was kind of like underwhelmed by the fact that so little people were playing. So it kind of demotivated me in some way, because it's super fun. And then I just spent some time on the snowboard game I was actually making back when I started working on Space Pirate Train, but a completely different thing. I spent like two years working on Shredders, which is a PS5 and Xbox. Game Pass game. In the meantime, we released it, and I was compelled by AR again, because then we saw the whole AR headsets coming up, and I was kind of feeling like maybe we should do something with AR that's not been done yet, maybe something. I really like the social aspect, so maybe something that's a party game, and then that's when BAM happened. Like, it's actually one year ago, this time one year ago, it's actually when I started making the prototypes of BAM.
[00:06:14.619] Kent Bye: And did you start prototyping it on the Quest 2 then?
[00:06:17.332] Dirk Van Welden: Actually, yeah, we started prototyping on the Quest 2. I know the passthrough was black and white, but we knew the Quest Pro was there as well. And the passthrough was getting pretty good, so we ordered a Quest Pro. I think the Quest Pro was already available, so I think we were working on that as well. Yeah, I think it was born with K-1 wanting to bring the whole LAN party feeling back and I want other people to see each other and shout at each other and maybe commandeering like small puppets fighting each other, like being a kid again on a table and having action going on. So I was inspired by Quake 3 and Super Smash Bros. I think those were the two games that inspired me to make BAM. I think at some point it could have stopped, the whole process, because I couldn't figure out the aiming, how to aim with your character, because it works if the tiny character is facing the same direction as you are, it's easy to aim. The moment it starts rotating, it's impossible to aim right. And we figured it out, it took like a week or two, and the solution is super simple, but if you don't have a solution, it's actually hard to figure it out. But the way how we do it now is like, when you point at something yourself, like when you're aiming physically yourself, your character who's standing at a different position will aim at the spot you are aiming. So it's kind of like a third-person version, it's like you're the 3D mouse pointer of your character that's going to aim, but you can still move the character the same way. So we'll always aim where you're aiming. And that thing, we had to figure out more stuff, of course. But that thing was kind of the, OK, and we need to do it now. It's going to be fun. And I think we have a playable demo back in March. And yeah, we just continued working on it. Meta supported us for this as well. And now it's coming out at launch in two weeks.
[00:08:00.072] Kent Bye: Yeah, and you mentioned also that it has the shared spatial anchors. Maybe talk about what are some of the other core technology that you needed in order to pull this off.
[00:08:08.210] Dirk Van Welden: Yeah, so special anchors was an interesting one because like there's real magic in looking at same miniature characters that the other player in your room so I have a video I can probably share it to you it's a video of where I'm taking a table you can scale it wherever you want and put in a specific position your partner can actually grab the same space and run away with it and you'll actually see like it's the same as like a deck of cards if you would pick up a table and you run away it would actually go away and this is the case as well but you actually are looking at the same thing so if you take your character and you shoot at its face like the tiny lasers actually go in his direction and you can see him dodging and like that interaction between player and a puppet and waving your hands as a tiny character towards the other player it's magical I really like that aspect, so we tried to amplify it a lot in the game.
[00:08:59.922] Kent Bye: Yeah, this is in a section here at MetaConnect that I had to wait in line to go see it as a multiplayer game, and at that point, I had already seen all the demos, and I was in my interview mode. I didn't actually have a chance to see it, but I'm curious if there's any embodied interactions, like if you move your body around, does that impact anything, or is it all from the controller? Maybe just walk through a little bit of the control mechanism, since I haven't seen it, and just so, as people who are listening can get a better sense of It sounds like you're looking at a tabletop scale, like these little robots that are fighting each other. You can point to aim, but are there any other mechanics where you're moving your body or impacting how this character is moving around, or if it's just with the controllers?
[00:09:39.593] Dirk Van Welden: Yeah, so moving around is just with the joystick, so you can move it around with that. It's always your point of view, so if you're standing on the other side of the table, if you press up, it'll go away from you. There's one more button, it's jumping and rocket pack, but all the rest is like... In the first version, you actually had punching in all directions, but you felt that once people really know how to play, they're only using the triggers. But the direction of your arms is super important, so if you're waving, you see the tiny character waving as well. So you can actually have interaction, you can actually pet the other character if you really want to. But yeah, for example, punching is the same as dashing. So actually, if you put your hands upwards and you dash, you actually fly in the air. So you can move around, like direction of your hands are super important in hitting the other player, for example. So you can dash upwards, dash downwards, if you put your arms forward. Actually, the direction is super important. Moving isn't really important. You will probably move because you want to see the whole arena. So sometimes there are some things that are in the way, so you'll probably move your head a bit, but that's kind of the only thing. And also something interesting, at some point, we were always looking down on the table, and we think maybe there will be some neck restraint if you're playing it for half an hour. So we made the combat very vertical, so you can jump around and dash up. You can be really high in the sky and still do, like, punching in the air. And so it's something we really took into account when making the game, and there was a lot of, like, movement throughout, like, the whole 3D cube around the battlefield. Then the NTS just puppeteering a tiny character, just using his hands with very simple buttons. Like if you play the 3D Mario, you can control the character. So it's just the jumping, just the D-pad and your arms with the triggers.
[00:11:22.099] Kent Bye: Nice. And so, what's been some of the reactions here at MetaConnect?
[00:11:26.010] Dirk Van Welden: Yeah, one of the best reactions was someone that said, like, my first experience with VR was Space Pirate Trainer, and it was awesome, and I had my first real AR experience with BAM, and it was also super awesome. Thank you, iIllusions, for trying new stuff, or, like, being innovative in that field. And that's, for us, that's a big compliment, because also tech is something that drives us. Like, the reason why we did, why we didn't do VR, like, three years ago, like, we did that Shredder title on console, so it's actually because I was thinking like the hardware is moving too slowly or there's it was going to like you had new chipsets it was faster but I really needed like new things to experiment with and good AR and pasture and depth was was really interesting for us to start experimenting again with the AR game.
[00:12:11.333] Kent Bye: Yeah, I think you had asked me what I think about the new MetaQuest 3, and I said, well, I think that's a good technological platform that's going to need a lot of great software that's going to make it a truly compelling platform. And I feel like there's something around the new platform capabilities that perhaps inspired you to start to tinker around with it. There's a part of me that's skeptical around where this is going to go, this mixed reality. I think VR is amazing in terms of entertainment, and I feel like there's this kind of more, like Denny Unger said, a utilitarian utility that happens with AR, but yet with VR, you're fully immersed and you have entertainment, and I feel like that's like so much more of a medium that's well-suited for the types of immersive and interactive and fun transportive experiences and I feel like by bringing in the physical reality it sort of diminishes that. But you mentioned the social dimensions and maybe yeah just kind of expand on what you thought was the compelling new aspect that inspired you to kind of tinker around with the medium again.
[00:13:02.987] Dirk Van Welden: Yeah, it's an interesting thought because I tried Magic Leap for example, I was a judge at a DICE event and I tried most of the games and for 90% of the games I was like, this would be way cooler in VR, right? It would be way more immersive and that's actually the case with a lot of games. But the thing that compels me the most in AR is the social aspect. like seeing other people, seeing their reactions, so I know it's kind of co-located and it happens less, but it's just it brings some magic from LAN parties before people start shouting, you see other people's reactions, you see them dodging like a dropship that comes in and bringing power-ups and those social connections are super interesting and people might say like, okay, but what's the chance that someone has like two headsets in their house? Well, a lot of people that have a Quest 2 might buy a Quest 3 and then they have like an extra headset and all the tech, like the passthrough, it's still working on a quest too, okay, it's black and white, it's less resolution, but you get the same feeling and there's some kind of magic with the whole multiplayer, seeing each other and looking at like, it's actually like playing a board game, but really intense and that's what I really like, but I completely understand for a lot of experiences VR, like if you look at immersivity and especially first person, Yeah, it's way better to make VR experience. And I also completely agree with the, like, AR will be mainstream because of apps, right? Like, we saw the whole AR thing that was embedded into glasses, and if you look at something, you can ask, what is this? Okay, that's a really interesting part, but most interesting part will be like, okay, can you now show me the way to the nearest Best Buy or whatever? And it will draw a line on the floor. It's simple stuff like that that will change people in life or just gain them time or make it easier to move around. That will be the things that make AR successful as a mainstream thing.
[00:14:52.294] Kent Bye: Yeah, I know at the early days of room-scale VR, back in 2016, I had managed to get a dev kit. And so I was on this list that included all these games, including Space Spy Trainer. So you're on this, like, select list of games that people are tinkering around with. So love to hear just any reflections on that moment in time of both innovating with the technology, but also receiving what other people were doing and having this cauldron of innovation with room-scale VR back in the early days.
[00:15:19.073] Dirk Van Welden: It's funny that you mention it, because room-scale VR, or like the positional element, was kind of a thing that made me believe in VR immediately. Like the fish demo that we had. The blue? Yeah, the blue. The big whale. It was so awesome. I was like, OK, this is really cool. There were some very, very early demos, because I actually had a dev kit that was 3D-printed controllers. It made me think like, okay, I want to shoot stuff, because there was nothing to shoot. I had a bow demo or something, and made lasers, and I needed enemies, and I made those typical Space Pirate enemies. I wanted them to be smart, so I made them very physical. But then the most important thing was like, okay, but now I need to make people move in a way that's like... When they will move, they'll be like, oh, this is cool, I can actually move around. So then we made like the lasers and made them slow down and made them beep and like really loud and really in your face. And people would just step aside after the one hits or maybe even most of the time just naturally. And a lot of people went like, oh man, you can't move in 3D space. And you had to move people, because a lot of people weren't used to moving around. Most of the demos, actually, if you put them on in Space Pirate Trainer, they would just stand still until there was this first laser. And then they moved aside. And you saw it in their movement that, oh, this is an actual 3D world I can move in. And they start moving around. And what are the edges? But that first step? was super important and I think most of the demos that Valve selected had that kind of moving around and they knew that moving around in a virtual world kind of like it was way more immersive and when I look back at all the studios that demoed there like a lot of those studios were super successful and I think they did a really good job like picking the right ideas and putting it in front of the press because it was very broad in terms of gameplay but also very interesting to see like all those different people that actually didn't connect much back then after the event of course we've been friends from that moment I guess back in 2016 or 17 and yeah it was magical to be in that group.
[00:17:29.284] Kent Bye: Awesome, yeah, and as you look forward, what are some of the features that you're looking forward to in terms of the MetaQuest 3 and what's going to maybe unlock new gameplay capabilities as the software continues to develop and evolve?
[00:17:42.398] Dirk Van Welden: I really like, it's been said, not really at the keynote, but they'll have body detection. They'll detect hand poses and even AI leg posing and stuff. That's really interesting. And maybe if they can do that, they can probably do it with people walking in front of you as well. And then it gets really interesting, because then you have this really... What is real, what isn't real, and having real information inside of your headset. Like, for example, in Space Pirate Arena, we made this whole level builder, so you can actually build your own levels, even the multiplayer in the game, procedurally. So now with the whole, like, mesh API, we could theoretically, like, make a Space Pirate Arena level based on the mesh that the house makes, so you'll have a full-blown Space Pirate level with those colors and, like, those textures in your house, and you can play against each other. And that brings in a lot of opportunities. Awesome.
[00:18:37.742] Kent Bye: Yeah, well, you've certainly been on the bleeding edge of innovation in this medium. And are there any other experiences that you want to have that haven't been created yet?
[00:18:47.348] Dirk Van Welden: Oh, if I wouldn't have those ideas, I would definitely start making them. In terms of content, there's a lot of good stuff coming out. I mean, the productions are getting better and better in every genre. I even have a full-blown... Breachers, for example, is a great example of titles that are getting graphically better, and that's a whole community. We have a lot of artistic titles that are still, from the early days even, some of those titles still mean a lot in terms of research. But yeah, I'm not sure. Hopefully all of the new tech that's being introduced gives room for new experimental stuff and hopefully there's new gameplay types coming out. Hopefully as less PC ports as possible so it's actually made for VR instead of being a part of something that was released on PC.
[00:19:38.918] Kent Bye: Awesome. And finally, what do you think the ultimate potential of virtual reality, augmented reality, and mixed reality might be, and what it might be able to enable?
[00:19:49.587] Dirk Van Welden: Yeah, I think ultimate would be that, yeah, just in terms of hardware, for example, as well.
[00:19:55.952] Kent Bye: Just experience, or what type of things it might be able to enable in terms of what types of experiences, or where the technology could go?
[00:20:03.150] Dirk Van Welden: Yeah, I think like with for example when I see things like the gaussian splitting coming around and having recreating environments that are like super realistic. Also with like the new avatars that are being scanned and like realistic humans. It's really going to be like this alternative reality that looks super realistic and you can have actual emotional experiences within that medium. Hopefully, experiences will be as compelling as, for example, movies, with the whole story, and that you feel that you're really into it. We already do it, but the graphical power isn't up there yet to make it really, really... I mean, now people should look at ways to artistically express themselves still instead of being super realistic, but I see that changing in the next amount of years and also with detection of body poses, it kind of makes the difference between the virtual reality and the actual reality smaller and smaller, so you feel more and more like actually as being in another reality instead of a made virtual reality, if you understand what I mean. I want it to be as close to another reality. It doesn't have to look realistic, but just that it feels like an actual experience. I think the more we can do better resolution, better lenses, better sounds, it just opens up more possibilities to have better experiences, I guess.
[00:21:29.181] Kent Bye: Awesome. Is there anything else that's left unsaid that you'd like to say to the broader Immersive community?
[00:21:37.906] Dirk Van Welden: Yeah, we talked about BAM already, so hopefully a lot of people will play it. It's a different experience and hopefully all the other VR creators will be as creative as possible, try new things. It's still a new medium, I know there's a lot of other people copying other titles and stuff, The way how to succeed with VR is create experiences that don't apply to other media. So I would say, like, it's still early days, and just try to be as innovative as possible, and use the new tech, right? So, yeah.
[00:22:08.329] Kent Bye: Awesome. Well, you've certainly, throughout the course of all the different experiences you've created, helped to shape the trajectory of where the medium is going, and has been going, and continues to evolve, both with VR and AR, with Space Pirate Trainer, Space Pirate Arena, and then coming now with BAM. So yeah, thanks again for just taking the time to help tell a little bit more about your story and what is getting you excited about where this is all going. So thank you.
[00:22:31.748] Dirk Van Welden: Yeah, thank you all. And thanks for the whole VR community to be as supportive as they've always been. Thank you.
[00:22:38.192] Kent Bye: So thanks again for tuning in to one of my dozen episodes about MetaConnect. There's lots that I've been unpacking throughout the course of the series, and I'm going to invite folks over to patreon.com to be able to join in to support my work that I've been doing here as an independent journalist trying to sustain this work. Realistically, I need to be at around $4,000 a month to be at a level of financial stability. I'm at around 30% of that goal. So I'd love for folks to be able to join in, and I'm hoping to expand out different offerings and events over the next year, starting with more unpacking of my coverage from Venice Immersive, where I've just posted 34 different interviews from over 30 hours of coverage. And I've already given a talk this week unpacking a little bit more my ideas about experiential design and immersive storytelling. And yeah, I feel like there's a need for independent journalism and independent research and Just the type of coverage that I'm able to do and if you were able to join in on the patreon $5 a month It's a great level to be able to help support and sustain it But if you can afford more than 10 20 50 or even $100 a month are all great levels as well and will help me to continue to bring not only you this coverage but also the broader XR industry. I now have transcripts on all the different interviews on the podcast on Voices of VR and in the process of adding categories as well into 1,317 interviews now that have been published after this series has concluded. So yeah, join me over on Patreon and we can start to explore the many different potentialities of virtual and augmented and mixed reality at patreon.com slash Voices of VR. Thanks for listening.
