I did an interview with Resolution Games CEO Tommy Palm soon after the Apple Vision Pro launch last year where we talk the in the Game Room game commissioned by Apple as well as the exploration of the relatively new gaze and pinch mechanic that’s enabled with the eye-tracking of the Apple Vision Pro.
After seeing the Neural Band at Meta Connect, then I’m reminded about how ultimately the gaze and pinch mechanic is a lot more efficient and more optimized for quickly selecting items in a fully volumetric context. Meta’s Neural Band announced at Meta Connect in the context of the Meta Ray-Ban Display glasses is only within a 2D context in a head-lock HUD display screen, and so operating the Neural Band feels a lot like what it feels like to navigate TV menus with a TV controller, but rather than a controller, then your thumb and side of your index finger are being transformed into a two-axis D-pad. Again, the ultimate form factor is likely going to come back to gaze and pinch, but that will require shipping with eye tracking. And so this unpublished conversation with Tommy Palm takes on a new context as we reflect upon the latest HCI innovations that were announced at Meta Connect and where the ultimate form factor may be headed.
Resolution Games also has quite a history of launching games on newly XR devices, and so this conversation with Palm is also within that spirit, and we’ll be diving into Battlemarked within the next conversation You can also see more context in the rough transcript below.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.458] Kent Bye: The Voices of VR podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. It's a podcast that looks at the structures and forms of immersive storytelling and the future of spatial computing. You can support the podcast at patreon.com slash voicesofvr. So at MetaConnect 2025, Resolution Games had a game called Battlemark, which was kind of a mix between Daimyo, which is a game that they've released, and also Dungeons & Dragons. And so basically for every new XR device that's come out, Resolution Games has had a game. So this conversation I actually did last year with Tommy Palm. He's the CEO and founder of Resolution Games. And so they had collaborated with Apple in order to create something called Game Room, which was essentially using all of the Apple SDKs and devices to be able to create Game Room, which was all these different games from like Solitaire and hearts and kind of like a battleship clone as well as chess. So I just had a chance to get my Apple vision pro is playing through all these different games. And so it was very early in this kind of new human computer interaction of the eye tracked look and pinch and different interactions with that. So at the time I was covering a lot of what was happening with South by Southwest and other stories that had been coming out. And I was planning on doing a big kind of retrospective of different Apple vision pro kind of news leading up to WWDC, but it just turned out that some of the different key interviews that I was trying to get didn't come through. And so happened to not publish a lot of the other things that I had gathered during that time period, including this interview with Tommy Palm very soon after the launch of the Apple vision pro and talking around all the different human-computer interaction and interface designs that they had done with the Apple Vision Pro with Game Room. Now, in the context of releasing this interview from MetaConnect, we just had the launch of the neural band, and so it's interesting for me to go back and listen to some of the different talks around gaze and pinch. So, eye-tracked enabled gaze and pinch is basically where we want to get with all the technologies. And the meta Ray-Ban display glasses don't have any eye tracking. And so you're only left with the pinching part of that without the looking. And so you end up having to turn your hand into what's essentially like a remote control with a two axis D-pad where you can go left, right, up and down. But you're kind of moving around in a way that is kind of like what it feels like to operate your TV where you're Kind of have to move around and click, and it can take a long time if you're entering text. I mean, with a neural band, you can actually eventually start to scribble out text with your index finger and then from that be translated into words, but that won't be launching until later. And only one journalist at MediConnect got to try that. As far as I could see, Victoria Song from The Verge got a chance to actually try it out, but everyone else just either saw the demo or didn't even see that demo. Mark demonstrated on stage of being able to do that. We're kind of moving into a realm where using your hands to pitch and do different types of interactions. If it's camera-based AI enabled, something like the Apple Vision Pro, you have to usually have it at least be seen. They have bottom-facing cameras on the Apple Vision Pro, so it's a lot better. Within the MetaQuest, they don't have as much camera coverage, so you end up having to raise your hand a little bit. And so not having to raise your hand at all with a neural band is going to be a big innovation. you're still essentially like carrying around a TV controller. And until they start to add eye tracking, then you're not able to have more of that depth interaction that you can get with the look and pinch and gaze that we start to talk about in this interview with the very launch of the Apple Vision Pro of kind of more gaze-detected, eye-tracked look and pinch, which I think is ultimately where things are going to go. So I felt like it was very appropriate to air this unpublished interview, both because of Resolution Games. tends to be always on the cutting edge of like new platform launches and games that they're coming out. And so in the next interview, I'm going to be talking to Gustav Stenmark, talking around Battlemart. So be very curious to see as they continue to have all these different platforms come out, if they continue that streak of having like a new game that's out there. I don't think they have anything coming out with the Meta Ray-Ban display glasses, just because the SDK wasn't even announced until the second day of the developer keynote. And It's still unsure as to when developers and third party developers are even going to get access to that heads up display. Anyway, this is an interview that I did with Tommy Palm back on March 8th, 2024. And yeah, I feel like it explores some key concepts of human computer interaction that are continuing to be developed. But with MetaConnect, it's going in a different direction and we're kind of building up some of the potentially key peripherals that are going to lead to like what ultimately we're going to get with eye track enabled pairing with something like the neural band. So by covering all that and more on today's episode with the Voices of VR podcast. So this interview with Tommy Palm happened on Friday, March 8th, 2024. And this was right after the Apple Vision Pro launch and talking around the Apple Archive game room experience that Resolution Games created. So with that, let's go ahead and dive right in.
[00:04:55.646] Tommy Palm: So my name is Tommy Palm. I'm the founder and CEO of Resolution Games. We're primarily a Swedish game company with a small part in Austin, Texas as well, so some Americans, but we build VR and AR experiences or games primarily. We've released somewhere around 18 games or in that vicinity, plus count. It's been a little bit more than nine years now.
[00:05:26.308] Kent Bye: Yeah, you've been really prolific of pushing the edge of what's possible with immersive media. And so maybe you could give a bit more context as to your background and your journey into the space.
[00:05:36.652] Tommy Palm: Yes. So I've always been super interested in computers in general and games in particular. I started my career as a programmer and been working a bit with game design. And then I started, this is my fifth startup in the game space space. And I saw VR early and I was convinced that must be the perfect future of games when you can be completely immersed into that world and you can mix real world and virtual world. And yeah, it took a lot longer than I had expected when we started. But it's slowly getting better and better. And some of the games that we have released, I'm super proud of, been a part of making them. I think they look fantastic. um vr is one of these technologies that we're all kind of expecting to happen around the next corner but it's at least growing slowly and there is a lot of active users now who love playing vr games right and so i know with the apple vision pro you actually had a launch title on the platform of game room and so maybe you could give a bit more context for how this project came about Yes, so we have experimented with games before, like the first game we released was a solitaire game. And then eventually we made Demio and we said, you know, these board games is one of these things that's going to work really well for both VR and AR. You know, the platform is great for social interactions, feels like you're right there with your friends. And these are very familiar things, especially in game room, where it's really back to chess and cards and dice games. So it's an extremely fun platform to stand on, where you start from things that are very familiar to people. People already know the rules. And then you add a little bit of fantastic flair to it. With the chess game in game room, your chess pieces can be animated and crush the other pieces as you put your queen in the take over a pawn or something.
[00:07:42.508] Kent Bye: Yeah. And so maybe you could describe a little bit about the different types of games that you have within game room. Cause there's like what I think of as Yahtzee, but it's called Yacht. There's like chess and some card games. So maybe you could describe each of the different games that you have featured there in game room.
[00:07:57.512] Tommy Palm: Yeah, so they are all parlor game and a few things that hopefully a lot of people are familiar with. Most of them are multiplayer games. So a lot of it is about being able to test these things with other people. But obviously, early days of any headsets, you're not going to have a lot of people with the headsets. So the chance that you have a friend that you can play with from day one is small. So it's also Frostplay with iPhone and iPad, which is a great feature. So if you bought one home for your family, you can play it instantly with other devices, which is great. And all the games also have like some sort of single player mode, so you can test it out. It's not the best AI in the world when it comes to like, there is a selection of multiple games. So implementing great AI for all of those games is a very big project, obviously. But yeah, it's something that people seem to really enjoy. And I think not only gamers, which is something that I'm very passionate about. And we spoke about last time we had this interview about the Wonder Glade after Google IO in 2016. I think even though VR and XR is great for games, there is a lot of people who want to experience this, but without having to learn how to play a shooter or even play a shooter at all. For gamers, that's great, but there's a lot of people who are not gamers out there who wants to also take out this new technology.
[00:09:31.324] Kent Bye: And are you using Unity's polyspatial integrations, or did you end up building some of your own custom interfaces?
[00:09:38.001] Tommy Palm: This is natively built with RealityKit and Swift, Apple's new programming language.
[00:09:45.769] Kent Bye: Okay. And so what was the process of figuring out some of the best user interfaces? Because obviously some of the affordances of say chess and moving pieces, but was there like an iterative process for how to best interact with some of these different games?
[00:10:01.556] Tommy Palm: Yeah, that's to me, at least in the process that I really enjoy with game design is to iterate a lot, get something up that is playable, test it with both people in the team, but also people who are not familiar with it and see what feels natural, what doesn't work as well as we thought, and then get to something that is fun. intuitive as it can be. And I want to say hand tracking is one of these things that is really a blessing and a curse at the same time. We all have a lot of experience using our hands, but interacting with virtual objects is new to a large extent. So there is a lot of very difficult user interface problems that you have to overcome, like In game room, you can use both direct control, where you put your hand and grab a chess piece with like a pinch grip, or you can use this gaze and pinch to grab a piece and move it across the board. And it takes a lot of iterations to get it to a place where it's natural and i think you know as time goes and these interactions become more and more standardized it'll be easier to implement them and people will be more they will know straight away we all know how to close the window we look for that little cross in the top but there's a lot of things here in xr that isn't fully standardized yet of course
[00:11:30.066] Kent Bye: Yeah, I found that the battleship with the grids actually ended up being one of the more stringent stress tests for the accuracy of the eye tracking with the pinch, because sometimes I would accidentally click the wrong square. And, you know, you can also, I guess, use your finger to push things. But with a lot of the other chess pieces or other games that I was playing, like the solitaire, the bounds of what you're looking at and what you can do with it is a lot more constrained. And I don't know if that was because of some of the eye calibration or I'm using the lenses and I don't have the additional spacer. And so sometimes it says, oh, your face is too close to the lenses. And I don't know if that reduces some of the accuracy sometimes. Sometimes when I go through the recalibration, it fixes it. But I'm just curious to hear, like if any one of the games was maybe higher error rates or if it came down to individuals with how they had it calibrated and some people were 100% fine or some people did have sometimes where they were like accidentally clicking the wrong thing.
[00:12:29.232] Tommy Palm: Yeah, no, it's definitely something that we have been iterating a lot over and still debate internally in the development team. With chess, for instance, some pieces are smaller than others. So if you have a pawn behind a queen, it's going to be much harder to gaze at that pawn from a certain angle. In reality, that isn't a problem because you would always use direct control, of course. You would lean and move. But one of the really great things with Apple Vision Pro is that you can be very comfortable because you can have your hands resting in your lap while you just gaze at something and grab it like that. And that takes some getting used to because our eyes is very quick. And many times we will look at an object and we'll already start thinking about, okay, I'm going to move it there. And then you grab it and I feel like there is definitely, you can see people learning how to use the gaze and pinch to become better at it. But also, as you touched on, the Apple Vision Pro is like a device that they want you to come in and have it fitted, especially for your shape of your head. And it's hard if you, for instance, borrow somebody else's headset to get that to work. And I think the lenses, the inserts they have help with the eye tracking as well. So it's all, it's very exciting tech. It's like very early days still of everything. Personally, for me, the Apple Vision Pro works really well i use it every day for work and not only for like game making but for also just getting this large screen up in front of me so i can multitask at several things at the same time and i've been hoping for apple to come into this space for quite some time so i was really really happy to see that they finally decided to do i think it's a It's always great also from a consumer perspective when you have big companies competing for space and they will inspire each other and then kind of make it move faster, which I think we're all, everybody working in this industry is wanting to see more people in VR. It's still quite limited, right? Even if we have tens of millions, it's still small numbers compared to most other computing platforms.
[00:14:56.656] Kent Bye: And just speaking of the excitement around getting access and using this, is there anything that you can share in terms of how this collaboration with Apple came about?
[00:15:07.859] Tommy Palm: Probably not the whole bunch. Like we've been working, they have a lot of really great tools that is pretty mature when it comes to Xcode and things like this that we've been able to use. And it's been... The area of XR is something that we have a lot of experience with working before. Now, a lot of new tools from our point of view came about. We normally use Unity for development, but I had a lot of programmers that was very curious to use something and learn something new. So that was also, I think, part of the fun for a bunch of the team to test another way of working. And that worked really well. I'm happy with the result that we were able to get to.
[00:15:53.837] Kent Bye: Nice. And I know that in terms of the different social applications on the Apple Vision Pro, there haven't been a lot of different social applications. And so this is one of the few experiences actually that had launched with some type of multiplayer dimensions. And so maybe you could just talk about the process of developing the social applications, especially because you're not using Unity and a lot of the other normal networking ways of doing that type of multiplayer. So just maybe elaborate a little bit on implementing the social and multiplayer dimensions in this experience.
[00:16:24.277] Tommy Palm: So I think we're doing social through two different ways. We either connect with your friends and family through SharePlay, which starts with an invitation to a video call with FaceTime. And I think a lot of people who use iPhones and iPads are not using this so much yet. There's like really cool functionality where you can, for instance, watch a Netflix movie together if you're not sitting in the same room. You start with a FaceTime call and then you can select to share play the apps and you get a list of the apps that you have on your device that is enabled for share play. So our game is one of those things. So you can start it that way, which is really neat functionality, but obviously it requires a little bit of knowledge of the iOS ecosystem or what you call it. And then you can also quick play through Game Center. That's the other way. Like if you just want to find a match, but then you won't use your voice, then it will just be a quick game with some other individual somewhere that you don't know very much about.
[00:17:31.397] Kent Bye: Yeah, and I was just playing some Solitaire and I know in some Solitaire games with an iPad, you can kind of like double click and it'll automatically move things around. But I noticed that you have to click it and move it. And so as you're developing some of these different experiences, there's the abstraction Already the pinching gaze is one level of abstraction. And so on one hand, I liked being able to kind of pick it up and move it around because it's like recreating the feeling. But sometimes it'd be like, oh, there's only one place this could go. And I'm just wondering if you've explored some of those other types of abstractions or optimizations where like a double click or something like that, where you could be looking at something and then have it perform an action based upon rather than like picking up and actually moving it with your hand.
[00:18:15.871] Tommy Palm: I do think double tap actually works on the Solitaire.
[00:18:19.601] Kent Bye: Oh, it does? Okay.
[00:18:20.771] Tommy Palm: Yeah, yeah. But I don't think single tap doesn't work. I think some of the other Solitaire might be like, that is optimized for just being a Solitaire game might use another interaction model. It's one of those things where we, as you said, developing for Apple Vision Pro is very different from developing for an iPhone on the user interface part of it. So there's been a lot of discussions and back and forth on like, How do we reuse as much as we can and make it so it feels native to iPhone and iPad as well? And I think most players who've tested the game is not aware that it's a multiplayer game between Apple Vision Pro and iPhone. They just download it to their iPhone and they play it there. And that's what they comment on. And that's great. That's where you want to be. But it requires a lot of effort to compete with the best. Solo game experience, obviously. So much great games out there for mobile phones at this point. It's a very mature market.
[00:19:23.223] Kent Bye: Yeah, and so you do have the ability to do cross-platform play across the different iOS devices with iPhone, iPad, but also with Apple Vision Pro. And so I'm just curious to hear if there's any special considerations of getting that work of how you design it so that it has kind of like a responsive design so it is able to work within a fully spatial context, but also work within like more of a 2D flat context as well.
[00:19:48.231] Tommy Palm: I mean, it is very challenging since a phone has really limited screen real estate, right? And preferably, if you're focusing on a social game, you want to show off the social aspects of it, which is really tough in that small screen space. And the Solitaire, as we mentioned, maybe isn't the best example of a social game. In this case, you can only play that in single player mode, so to speak. But then you need the whole screen space as much as possible to fit fingers. So you tap the right card when you're trying to do it. So I think when you're a player, you just expect things to work. in the way that you want it to but when you're a developer there is a lot of work behind this and and also something that feels very intuitive when you do it might turn out that a lot of people don't want it to function that way so it's always great if you can continue development and react a little bit to how players are interacting with it and react to that I think one of the key reasons we added a solitaire to this collection of games is that it's always great to have something to do while you wait for, if you want to play it in multiplayer, it's great to be able to do that. And it also feels like it fitted pretty naturally in a concept like Game Room, where you have your boxes of games that you like to do. And obviously a lot of people, especially with mobile phones, really like to play solitaire games.
[00:21:17.130] Kent Bye: Nice. And so this is a part of the Apple Arcade. And so I imagine that anybody who's subscribed to that service, they're able to have access to it. And so what have been some of the reactions or feedback that you've heard so far on the Apple Game Room?
[00:21:29.876] Tommy Palm: A lot of very positive feedback. People really like the visuals and the way the game plays. Some reactions in the beginning about rule sets. So we have reacted to that and released an update on some of that things. But these parlor games are like simple games. It's also very difficult because they're incredibly local. So the way you play them in Sweden, where I'm from, might be different from Norway, which is your neighboring country, or even more so, like you have a local version of a game that you're playing. So hopefully we'll be able to add stepping options so you can tweak the games a little bit to the way you're used to play them.
[00:22:11.943] Kent Bye: Is that with Hearts? Because that was the one game that I played that I thought I knew the rules, but then when I was playing it, I was like, I don't think I actually know the rules for how I play this because there was certain moments that I was like, I need to watch a video online to see what the actual rules are because I'm lost as to what's happening.
[00:22:28.587] Tommy Palm: Yeah, now Hearts is one of these games that if you know the rules, you can be really sneaky. And it's a really fun multiplayer game. So I didn't know that before we implemented it. And now I'm a big fan of Hearts. You feel so incredibly accomplished when you manage to shoot the moon, as it's called, when you trick your opponents into giving them penalties.
[00:22:55.919] Kent Bye: Nice. And so what were some of the lessons that you have taken from your other titles? You said you've done 17, 18 other games now. And what were the different things that you felt like you've got a lot of experience in this spatial computing, XR, VR, and mixed reality types of immersive experiences that you've been creating? And so what were some of those principles that you were applying to this Apple Game Room game?
[00:23:19.059] Tommy Palm: Well, I think some of them are just how to anchor virtual objects. In the beginning, when we were doing AR things, we continued on the same path as we've been working a lot in the history of the company, where we do very cartoony style graphics, very stylized things that work well on battery driven devices. You know, you get much more performance. that way. But I do think for AR, you want to be as close to reality as you can visually, because that's when you create this really magic. It's something that you are very familiar with, like a die. And it looks exactly like a die you have on your table to the point where a screenshot, it's almost hard to understand what's going on because all you see is like a cluttered table. And that requires a lot of focus on materials and reflections and things like that. And the Appalachian Pro is absolutely magic when it comes to those things. You can't see the pixels. When you're succeeding in those virtual objects, they become almost indistinguishable from the room you're in, which is... incredible but then of course whenever a player or user sees something that they are familiar with they will also bring their memories of what you can do with that item and I think that's something we saw a lot in VR from the beginning a lot of game companies went into that trap where you come into a room and there's tons of objects there and now players want to test all the objects out and they want to combine them and very quickly you get this permutation of like so much implementation that you have to do. So yeah, I think it's such a fun space to work in because there's so much that still kind of remains to be discovered and played with.
[00:25:08.884] Kent Bye: Awesome. And finally, what do you think the ultimate potential of this type of mixed reality XR spatial computing might be and what it might be able to enable?
[00:25:19.193] Tommy Palm: I am, as you know, very hopeful about the technology. I think that there is a lot of really interesting applications within productivity and work and education and all these different things that comes together. I see it very much as like the next natural step for computing, that you hopefully will have a computer that is context aware. And I think it ties in very, very nicely with this AI assistance that is on the rise right now, where you can get a lot of help and suggestions and be more efficient with what you're doing. So games is always great because we get to lead. It's use cases that people are willing to pay for. We get to do a lot of experimentation that actually ends up in products that can be profitable very early start. And we get to discover a lot of what doesn't work and what works well. It's definitely a lot of fun.
[00:26:15.190] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:26:19.938] Tommy Palm: No, yes, I think it's fantastic that you keep doing this. I'm a big fan. Thanks for having me.
[00:26:26.587] Kent Bye: Awesome. Well, Tommy, thanks for joining me here and to talk a little bit more about the development of Game Room. Like I said, at Resolution Games, you've been doing lots of different immersive games over the last nine plus years or so. And now being a launch title, it's really cool to see. And also just using the native integrations, it was difficult for me to actually tell. So I think that's telling in terms of how you're able to really create this type of immersive experience in a way that is pushing the limits for gamers. what's possible with the technology. And so, yeah, looking forward to see where you continue to take this in the future. And thanks again for, for joining me here to help break it all down.
[00:27:00.324] Tommy Palm: Thank you very much. My pleasure.
[00:27:02.806] Kent Bye: Thanks again for listening to this episode of the voices of your podcast. And if you enjoy the podcast and please do spread the word, tell your friends and consider becoming a member of the Patreon. This is a, this is part of podcast. And so I do rely upon donations from people like yourself in order to continue to bring this coverage. So you can become a member and donate today at patreon.com slash voices of VR. Thanks for listening.
