Eggscape is a mixed reality, table top platformer game similar to Lucky’s Tale, but since it’s mixed reality then there’s a lot more of moving your body through space as a control dynamic. I previously got the whole backstory about Eggscape from director German Heller at Venice Immersive, where it picked up a third place price. Now that the Meta Quest Pro has been released, they’ve ported it to take full advantage of the color passthrough, which I have to admit has been the most immersive and compelling implementation of mixed reality that I’ve seen yet. It was so incredibly immersive to see how they utilized their physical installation as a context grounded in physical reality in order to overlay and alchemically transmute this physical installation waypoints into an entire dynamic playground for these cute and tiny little egg characters. I had many moments of experiencing the VR giggles where the piece is melding elements of the physical reality into this magical alternative domain of existence that starts to blend together in a way that feels completely plausible.
There were moments where my little egg character was floating in the air with a balloon that my mind completely bought into that I was somehow controlling it’s movement as it were a remote controlled balloon-like drone object. It’s these glimmers of believing the alchemical blending of digital and physical reality where my mind seamlessly integrates the different domains into a coherent and plausible reality that makes me realize the transformative power of mixed reality as companies like Meta, Apple, Samsung, Google, and Qualcomm are all working towards this vision of bringing the next paradigm of computing out of our hands and onto our face where we will have a magical lens that allows us to blend the digital and physical realms through our visual, auditory, haptic perceptual inputs.
Heller said that Eggscape is ready to be deployed as a location-based experience as they’ve more fully developed it as a four-person multi-player journey where your tiny little egg characters compete for collecting the most number of coins and points for killing the robot enemies. There are lots of opportunities for emergent co-op and competitive play as well, and with a few tweaks could start to create a truly replayable experience with many side quests or other co-op or competitive gameplay. Either way, the 3dar team has made some incredible progress over the past three years of working on this, and it’s definitely one of those experiences that is pushing the boundaries of mixed reality gameplay mechanics melded with social dynamics and the power of a good physical installation to help sell the magic of mixed reality.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the future of spatial computing and the structures of immersive storytelling. You can support the podcast at patreon.com slash Voices of VR. So, continuing on my 24-episode series of South by Southwest, today's episode is with Eggscape, which I've covered before at Venice. but this is a mixed reality game-like experience that has a whole installation. You walk in and you see the eggscape letters, but on top of the letters you have all these models that have these little characters, these little eggs, and then it's a four-person collaborative experience. in the context of the MetaQuest Pro. So you put on these MetaQuest Pros, everything gets calibrated and this is overlay on top of this installation where you have these kind of like Lucky's Tale style immersive experience where you're controlling from a third-person perspective but you have these little tabletop characters that are moving around and you have to actually move your body around this space as well and also negotiate the movements of these four other people. And so there's a bit of a mini competition where you're collecting all these coins, but you essentially go through a series of different stages. You're interacting and you're fighting these different robots, but it's kind of like a tabletop mixed reality game, but you have this interaction with the world around you. There were some moments in this piece that just gave me the VR giggles, just maybe laugh out loud with pleasure and delight. because they didn't have the blinders on the side. And so because it's mixed reality, you have your peripheral vision of really being grounded and centered into the physical reality. And on top of that, it's a color pass through and they have this installation that they're doing this modulation on top of. And so there was so many different ways that it was priming me to really be some of the deepest levels of immersion in a mixed reality piece that I've ever experienced before. and just really deeply immersive and that level of possibility where you see this digital object floating in the air and you know it's not real but your brain's kind of at a deep level viscerally convinced that this thing is real and you're interacting with it. This expression of agency through mixed reality is such a powerful dimension of this abstractions of using the video game controllers to move this character around. It gave me such deep joy to be able to have that moment. We're in the very beginning phases of mixed reality. And so to see this piece at Tribeca last year, just a brief preview in a Quest 2, and then at Venice, it actually picked up a third place prize. But what I saw at South by Southwest was more of a complete holistic experience on the MetaQuest Pro, and it's ready to be deployed as a location-based experience. So I do a very brief update because I have a much more deeper dive with him on from Venice. And if you want to listen to more of the backstory, go check that out. This was more of like, Hey, what's the updates since we last chatted, they ported it over the quest pro and some of the different experiences that he had with this, that we talked about in this quick update. So that's what we're covering on today's episode of the voices of VR podcast. So this interview with her mom happened on Tuesday, March 14th, 2023. So with that, let's go ahead and dive right in.
[00:03:02.992] German Heller: I'm Germán Heller. I run 3DAR, an Argentinian studio that we do innovation in XR and animation and storytelling. Now we're presenting a project called Xscape. It's a mixed reality multiplayer adventure about troubled Xs that run for their lives in complicated situations like meteor showers and alien attacks and stuff. So it's a comedic video game. A little bit of a tribute to the old-school 80s gaming that I used to play when I was a kid.
[00:03:35.923] Kent Bye: Yeah, well, the last time we talked was back at Venice of 2022, and we did a whole backstory and talked a lot about the eggscape, and then, you know, after we talked, you ended up actually winning, like, the third place at Venice. What was that like to sort of come home with that prize?
[00:03:49.735] German Heller: That was great, really really great because I felt like we were presenting a little bit like a prototype there because there was a lot of innovation but there was so much work that needed to be done there and still fixing things and adding more things and coming back with the prize was a huge push and a reassurance that we were doing something worth the energy and effort and that it was really renewing my energies to keep working and six months later we have a version that is like much much more evolved still a lot to do though but
[00:04:27.051] Kent Bye: Yeah, I was a little surprised just because you know I think of the Venice is like the storytelling and what pieces that are there But I think it speaks to how compelling the mixed reality Experiences and how delightful it can be and I think here that you have the meta quest pro where you have like a whole other like color pass-through I think you evolved with the early iterations from what I saw last year at Tribeca and then at Venice now you have like a whole mixed reality experience with the pass-through and I felt like there was moments where I saw like these little characters flying up in the air and I just felt totally convinced that these things were there because it was like looking up and I just I started laughing and if I call it the VR giggles where it's so delightful that it's like transcending your expectations and just making me laugh and I felt like that level of delight was something that is really hard and rare to cultivate and that you were able to really nail it with this latest version. So congratulations on that, but I'd love to hear some of your perspective of moving from the Quest 2 onto the MetaQuest Pro and to have this next iteration that you're showing here at South By.
[00:05:27.888] German Heller: Well, I mean, it means a lot that you say something like that and it's really wonderful when you can make a good use of the technology. I mean, a laugh is so vital these days, like you're actually having fun with something. So, I mean, the technology side of it is when it works, it's wonderful. When it doesn't work, it's really the most painful part because it's like you're innovating in some ways. The companies that you're working with are also in innovation. And when innovation is like you're doing something that is not done yet, so then it's like, But everything is yet to be discovered and many things are not working right in the headsets front. Like we moved to the Quest Pro and I love it when it works. I love it when it works. But when the controllers lose pairing and when the Wi-Fi gets disconnected, then it's It's really frustrating because like the game is kind of like a trip and takes you to and feels like the fun is ruined there. So we're still like really trying hard to fix all the little gaps that could make this deflate a little bit. I think the technology is still has to fine-tune a few things. The controllers losing pairing was like, oh my god, come on. So frustrating. But then when it works, it's like, yeah, it's wonderful to have such a great product that has full-color pass-through that can make you feel like something is there when it's only animation. So that's huge. And I really wish it can work perfectly really soon.
[00:07:14.077] Kent Bye: Yeah, I thought that there was something about the black and white that I felt like it was really immersive because I could see the world and then the virtual apps were highlighted, but I think actually having the color with the Quest Pro you can have a peripheral vision both from the side and below and you don't have the blinders on and so there's something about the subtle aspects of my body that makes me feel extra present in the physical reality, and then having, like, you have a series of different eggscape installations with the letters, and on top of that, you have, like, these little toy models that then you're using to do, like, overlays on top of, but I felt like the mixed reality overlays on top of those, it was sort of like, it was able to kind of erase what was there, or, like, I took it off, and I was like, oh my God, these are all, like, built out in these installations, because it just felt like like you were able to cue it in such a way and override it so that my mind was just more remembering the virtual digital reality rather than the physical installation which I was like it wasn't until I actually took off the headset I was like oh my god look how much they've actually like altered my perception of reality through the mixed reality and it was such a deep level that I was just really blew me away.
[00:08:19.877] German Heller: Oh, thanks, thanks. So, in a way, the reason for a physical installation to be around is to create, like, to bring the worlds closer, the inside the headset and the outside the headset. And it's the same we did with Gloomy Eyes and Paper Birds, which, I mean, in theory is totally unnecessary because the whole experience happens inside the headset. In mixed reality, It's a little bit like that too, between quotes, unnecessary, because you can create everything in an animation. But when you're passing by in a festival, we're human beings living in a physical world, and it's nice that you can see something outside the headset, and then you connect already to the experience, you get closer to the feeling, and then when you put on the headset, it comes to life at another level. So it's kind of like creating a middle step there. And I love it. I really hate when we have something to offer that is just a headset and it's like just you have maybe a poster and a headset and I feel like we can give much more and in this kind of projects they should take care of the onboarding of the experience in that sense, you know.
[00:09:31.382] Kent Bye: Yeah, it's the onboarding and offboarding that I think a lot of, like, really good VR pieces that have installation components that you're able to enter into the installation and start to step into the magic circle of the piece, and then go into the virtual experience, and then when you take it off, then you're still within that magic circle, and then when you leave the installation, then you're exiting that magic circle. So I feel like you're able to have those different phases of the magic circle by having an installation like this, and I didn't really fully appreciate it until I had the end of the experience to see how transformative it actually was.
[00:10:01.098] German Heller: That's awesome. It's beautiful to hear the feedback and and it makes you like guess when you come to a festival there are many many things happening and and sometimes you you just feel the heat when there is like a little technical issue at some point or people that they cannot test the experience because there's too many in line and everything and Good feel like that is really like comforting and makes me happy
[00:10:26.468] Kent Bye: Yeah, and so you had a four-player experience, so I was doing it with three other people, and I found myself, like, able to kind of navigate around them, and, you know, sometimes they were in my way, but I feel like, more or less, we were able to, like, not get in each other's way. I had, like, a little avatar on my character that I dressed him up, and each person had theirs, but I didn't necessarily know who was who, and there was also a game component. I didn't know who was winning, but probably it's maybe a minority use case that there's gonna be four MetaQuest pros, like, in one location, unless you do an LBE approach of launching stuff. So yeah, maybe talk about what's next here.
[00:10:58.574] German Heller: Yeah, the plan is like, we have this project that's ready to go, like it's ready to go to the world, not only in festivals. So the plan is to start offering in locations. Having a location gives you the chance to really offer a controlled environment where everything works perfectly and people, they don't have to install anything or buy any equipment. It's just like you're walking by a mall or any area with circulation and then you see this Models and it's like what is this? And then you just get a ticket like a cheap price and then you put on and you go some Magical space, you know, and that's what we want to do with it And if it's gonna be for a while like that because as you said like I don't think Four headsets are going to be in the same place and all links together and ready to play just like that What's been the reaction to the beast here at South by I? No, they're wonderful. Like, I mean, everybody, like every reaction was really, really positive. And the best is like, not their reactions. I mean, I love the good feedback, but when I'm watching people playing and they crack laughing and they just bend because they like, I don't know, their egg got smashed or like run by a car or like this gigantic pig exploded and this kind of things are really, really rewarding.
[00:12:19.599] Kent Bye: Yeah. Getting some good feedback in terms of where to take the game design next?
[00:12:24.280] German Heller: Not that much. I mean, I got a few good comments, some yours, but like in general, it's mostly appreciation. And yeah, the people are not giving us tips or it's more like that they really, really liked it and it's their favorite. the thing that you really like to hear but I would definitely appreciate feedback only that now I'm like so full of you know meetings and stuff and everything and so I don't have the health to absorb this but after this I'm sure like many ideas after the festivals they come because it's a very inspiring experience.
[00:13:00.827] Kent Bye: Yeah, I think that thinking about the replayability of, like, when people come back, then they're able to, like, what are the ways that they can kind of trace what's happening and compete against each other or gang up against each other, and, yeah, just, like, stuff like that where you're able to have more insight as to, like, if there is an end goal of a competition. So I think there is a gamified element that actually kind of makes it fun, but I think you can have different, like, oh, maybe there's a team aspect, and so, yeah, I think there's a lot you can do with having a multiplayer type of experience like that.
[00:13:27.808] German Heller: 100% and we want to give awards, like real awards to the people that earn the most scores and if you play many times maybe you get a little egg toy and something like that. The structure, like, because Xscape is three years old actually, like the original idea had nothing to do with mixed reality, it wasn't even a multiplayer, it was more like born from the characters and the world and the feeling of like being in a constant threat and running like because life sometimes has that quality for all of us and making fun of that so there is a whole world behind and we're going to do many things with it like short films and hopefully we can even do a feature film about it because there is enough to tell about this X to do it
[00:14:16.883] Kent Bye: Yeah, maybe some also ways to differentiate the eggs even more so because they do actually look very similar and I lost my avatar a lot of times of like I was looking at the wrong character and then I realized that I was dead and I was like respawned somewhere. So yeah, just finding ways to really track your character in a way.
[00:14:33.838] German Heller: Yeah, yeah, like to that matter particularly, we were going to put like an arrow at some point when you respond to help you figure out where you are, also like the names on each one of them every now and then when you switch levels, who's first, who's second, like we're brainstorming around all these things and we have to try and we had the scoreboard at some point but then it was kind of like bothering a little bit. So it's always, I mean, all these between quotes problems are also opportunities to innovate in mixed reality and how you introduce a solution that actually is innovative in that sense. Like, for instance, if you see, like, the scoreboard, like, they're integrated to assets or there is, like, a plane with a sign that says who's winning and then it crashes and it explodes. You know, those kind of little jokes. But we have to go back to Argentina and get to work.
[00:15:31.822] Kent Bye: Awesome. And finally, what do you think the ultimate potential of mixed reality might be, and what it might be able to enable?
[00:15:39.357] German Heller: I mean, the potential is infinite, and it has the potential to be the replacement of the smartphones and everything. It's not around the corner, like, definitely not. When I am struggling with the controller pairing, I realize that it's a long ways before this is ready-to-go technology. But like, yeah, I mean, screens with the computers, like, you just put a good quality mixed reality headset, and you don't need a screen or a TV, that kind of potential. But think of that as a little bit more for sci-fi at this point. And I think we have to wait some good five years. I mean, I've been saying these five years for a while. But yeah, hopefully AI gives us a push, too.
[00:16:28.300] Kent Bye: Awesome. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:16:33.368] German Heller: No, just my appreciation for this family of creators in XR, innovators, and it's always a deep pleasure to be surrounded by people trying to do weird stuff like this and to push the boundaries somehow. So yeah, and thank you for being a voice to that too.
[00:16:53.584] Kent Bye: Yeah, and thanks for creating this experience. It's been a real pleasure to see it evolve from a sneak peek at Trebeka, then Venice, and now here. I feel like it's gone through quite an evolutionary journey over the last three years. Yeah, it's come a long way. And I think, like you said, it's right at the cusp of being able to go primetime for this location-based experience. And I think people are going to have a lot of fun. And I think it's going to be a big hit. So thanks again for coming on the podcast and helping share your story and give a bit of an update for all that you're working on.
[00:17:18.163] German Heller: Absolutely, again, thank you.
[00:17:19.993] Kent Bye: So that was Hermann Heller. He runs a studio called 3DAR out of Argentina that does a lot of animation, XR, and immersive storytelling. And they've been working for three years now on this mixed reality experience. Well, it didn't start as mixed reality, but now it's a mixed reality piece called Eggscape. And it's a really amazing location-based experience. Maybe you'll get a chance to see it on the festival circuit, or it'll be distributed around in different places around the world. So I have a number of different takeaways about this interview is that first of all, well, there were a number of different mixed reality integrations this year at Southwest Southwest. There was a piece called Unearthed that was also on the MetaQuest Pro. At the very beginning piece, you just are in this booth like area and you see these digital objects come up and that was a really nice moment. But for the most part, most of the other aspects of that experience turned into more of a VR experience. And so it was less of a mixed reality experience from that point forward. There's a piece called Yuki Mixed Reality, which is a video game called Yuki from Avore, which is actually doing a lot of the pieces like Pixel Ripped. They have a new Pixel Ripped 1978 that was announced at GDC that was looking at Atari and they have this really exciting stuff that they're doing with the Pixel Ripped series. But they also have created this Yuki game and they had a Mixed Reality demo there. little bit of a prototype to see how you could, again, turn a wall into a portal where you see these objects coming out at you. Remind me a lot of the lab where you have these moving around and kind of shooting objects that are flying at you. And so, yeah, just moving your body through space, having these virtual objects, but you're grounded with world around you that yuki vr actually had this big poster so looking at the poster already was kind of like a transportive experience but something like eggscape you see this installation that you've seen physical reality that has a spatial dimension that i think worked a lot better in terms of using that as an installation piece rather than a blank wall with a big poster that already when you look at that poster you feel like you're in an alternative Reality from this animation poster So I think really thinking about how do you ground people into the sense of plausibility of what the physical? Installation is and then from there add the mixed reality components so that you're adding that level of magical realism I think that's what they were able to really succeed with with eggscape in this latest iteration with the meta quest pro and The first versions I saw at Venice were the black and white pass-through. You can see that it's there, and you're aware of it, and it has the color pass-through, so you see these dimensions of these objects that really stand out. But having the color pass-through, there was much more of my body that felt like I was really grounded into this physical reality, and that it kind of tricked the plausibility of my mind, a much more immersive context with the MetaQuest Pro, especially, like I said, with the peripheral vision of being able to pick up everything. It just felt really grounded in the physical reality. There was another piece called Rockets by Pillow by Lucas Rizzotto that was using the Quest 2 as a mixed reality mode but you were laying down in a bed and looking up and most of the different immersive experiences in a VR experience but when you did look down you saw more of a black and white pass-through but they weren't really using the pass-through elements of that to make it more of a It was more of a virtual reality experience that had some elements of mixed reality, but that weren't really integrated into the overall experience, but just to have a situational awareness of what was happening around you. But with the mixed reality use case of eggscape, it was really quite effective and transportive and really gave me the VR goals, which I haven't had since actually the first time that I played Lucky's Tale back at GDC in 2015. Yeah, just to have a chance to play an early demo and to get a sense of this really transportative nature of this platformer type game, but in a 3D spatial context, in a very similar way that this eggscape reminds me a lot of the dimensions of Lucky's Tale. But instead of just kind of moving your head around, you're moving your full body, and you're taking a whole journey. And yeah, just the design of the characters, the little comedic moments and the different bosses that you have to fight. So yeah, I think it's This blending of being rooted in a really compelling interactive gameplay, but also having these other story elements that they're starting to weave in there. And as they move forward, how are they going to make it more of a group experience or a competition or to have this replayability component? And so I think there's some other things that can continue to tune to make sure like, you know, who is winning at any certain moment, or do you have co-op play where you start to team up against people? And it's very basic, you know, collect these coins and fight these little robots, but to take you on this whole journey of navigating this little creatures of these eggs that you get to design how they look and feel, and then to take them on this whole adventure through this installation. And when I took the headset off, it was really like, Oh my God, I can't believe how much I was really believing that this digital reality was a part of this experience that I had when I saw a difference between what the actual physical installations were and how much they were able to take that and overlay on top of it, this whole level of digital reality that just kind of tricked my brain in a way that just felt really plausible. So anyway, lots of really interesting mixed reality experiments with eggscape and excited to see where they take this in the future. So that's all I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
