On February 13, 2024, Felix & Paul studios announced that they had secured multi-million dollar financing for their next Location-Based Entertainment (LBE) VR experience that was their “most ambitious project yet” and based upon a brand-new IP that’s set to launch in 2025. They had such a big success with their previous LBE of The Infinite (based upon documentary footage shot over the course of 2.5 years on the International Space Station) that they realized that LBE experiences were going to become a significant part of the future of their business.
The press release didn’t provide a lot of other details, and so I wanted to catch up with Felix & Paul Studios co-founder Paul Raphaël to share a bit more context for how this came about, more details on their LBE successes with The Infinite, and whatever else he could share about this new project. In the second half of this conversation, we talk about his experiences with the Apple Vision Pro, how Apple is emphasizing entertainment experiences and 180 videos, and how they want to eventually bring many of their 360-videos Felix & Paul Studios productions or potential adaptations to the platform. We also talk about how Felix & Paul is continuing to move beyond just cinematic 360 documentary video, and into CGI-based fiction projects like their Magic Leap 2 AR piece Jim Henson’s The Storyteller, which premiered at Venice Immersive.
It’s certainly an exciting time for the industry, and Felix & Paul Studios will be building their LBE experiences from the ground up taking what they’ve learned from cinematic 360 documentary videos and translating that into full, 6-DoF, fiction narrative VR experiences with social dimensions with even more immersive installation onboarding and offboarding plans as well.
Podcast: Play in new window | Download
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. It's a podcast that looks at the structures and forms of immersive storytelling in the future of spatial computing. You can support the podcast at patreon.com slash Voices of VR. So back on Tuesday, February 13th, 2024, Felix and Paul Studios announced that they had secured multi-million dollar financing for their latest location-based entertainment VR production. So there isn't a lot of information in their press release. It basically just said that Felix & Paul Studios is working on their most ambitious project yet. It's a location-based entertainment VR experience. It's based on a brand new IP, and it's set to launch in 2025. And they announced their funders that include a number of different Canadian entities, including the Export Development of Canada, the Bank of Canadian Entrepreneurs, SODEC, which is a French acronym, which translates to the Cultural Business Development Corporation, as well as the National Bank of Canada's Creative Industries Group. So a number of different Canadian organizations that are funding it at a level of multi-million dollars, and they didn't really give much details as to what they're actually working on. And so I wanted to catch up with one of the co-founders of Flex and Paul to get a little bit more context for how this came about. They've worked on The Infinite, which was a really successful location-based entertainment experience that was featuring a lot of their footage from Space Explorers as well as this amazing footage that they have from putting a VR camera on the outside of the International Space Station, so you can see the spacewalk, but also amazing footage of the Earth. So, they ended up having a huge amount of success for this LBE experience, and they decided that this was actually going to be a pretty significant part of their business strategy when they think about how to bring this immersive entertainment out to audiences and consumers around the world. And so I wanted to talk and get as much information as I could from Paul, but also to talk about the Apple Vision Pro, since there's a lot of entertainment possibilities that are on the Apple Vision Pro and Felix and Paul Studios have done over 30 different immersive storytelling VR experiences so far. So I was just trying to get a sense of whether or not some of their backlog of different experiences, we're going to make it onto the Apple Vision Pro at some point. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Paul Raphael happened on Friday, February 16th, 2024. So with that, let's go ahead and dive right in.
[00:02:23.387] Paul Raphaël: I'm Paul Raphael, co-founder and chief of innovation at Felix and Paul studios. And we've been in the, uh, VR content game for the past 10 years. We started with strangers with Patrick Watson. That was the very first cinematic virtual reality experience in the 360 degrees in stereo. And since then, we've created over 30 experiences in the realm of documentary, fiction. Most of those are filmed experiences. Although lately, the past couple of years, we've also started exploring location-based experiences. So The Infinite is a project we launched about two years ago, and that's been touring North America. And that's a large-scale, location-based free-roaming exhibit where over 160 people can experience the International Space Station simultaneously. And yeah, so we're currently exploring more experiences of that kind. We're still also creating cinematic virtual reality experiences. And we've also dabbled in augmented reality, as we've last spoken about the Storyteller augmented reality book that was launched at Venice just a few months ago.
[00:03:35.883] Kent Bye: Right. And I know we've had a chance to do a number of interviews over the years, but I always like to ask if you give a bit more context as to your background and your journey into this space, considering this long journey from 360 video to doing location-based entertainment and AR and just kind of how you tell the story of your journey into this space.
[00:03:55.332] Paul Raphaël: So basically my original career was filmmaking. So I went to film school here in Montreal and actually I met Felix right out of film school. We went to the same school, but we really started hanging out after film school and we quickly started working together on what was at the time more traditional content. So whether that be music videos, commercials, short films, we both had a fascination though for immersive storytelling. And that could mean an immersive experience such as a live show, projection shows. It was even certain films that had immersive qualities, right? We really felt a draw towards how immersion changed the way people perceived what they were being shown, right? And so we slowly migrated away from making basically a flatties, right? Traditional cinematic content and started exploring such live immersive experiences. Now this was before we got into VR. So it was, like I said, live shows, projection. We got into 3D, so stereoscopic capture and projection, not so much making straight up films using the technology, but making installations where we lock down the variables of how you shot and projected 3D content in a way to create an illusion of space, of actual continuation of space from the real to the virtual. It wasn't virtual reality, but it essentially was a form of narrow field of view virtual reality because you couldn't look behind you, but the frame basically became a window into another world rather than just an abstracted cinematic frame. And sure enough, when the Oculus Kickstarter came around, we got one of the first devices and almost immediately decided to dedicate ourselves to the medium. So by then Felix and I had been working together for about 10 years. It's now been another 10 years since we've been working together, since we founded Felix and Paul Studios, which has been entirely dedicated to mainly virtual reality, but also some augmented and mixed reality. At the time, the original Oculus was really mostly targeted as a video game device. Of course, the endless possibilities of the medium were always kind of on the table, but the focus was mostly in gaming. While there had been a few 360 videos, there was no stereoscopic 360, either cameras or experiences that had been made. And so having explored that concept in the installation space, as we had been doing, we set out to build that first camera and made that first experience, which was Strangers with Patrick Watson. And that really catapulted our studio that brought us to where we are today, having built a team of We're now, I believe, over 70 people at the studio. We've built cameras, so hardware, we've written software. We have post-production team, obviously production team and creative team. We have the past few years built interactive team. We have actually even people that are come from the space industries because we've spent so much time working on space projects and in the space industry. So yeah, that gives you a bit of a portrait, I think, where we come from and where we're at.
[00:07:21.660] Kent Bye: Yeah, lots of dimensions to being an early mover and innovating on storytelling, but having the development of camera technologies, doing 360 video, 180 video, you even got some more CGI stuff. You have location-based entertainment with the Infinite that came in, moving into a little bit more augmented reality. So since this announcement that just came out this past week that you've raised a multimillion dollar financing for a new location-based VR experience, Maybe you could take me back to the first LBE that you did with The Infinite that I had a chance to see in Tacoma and do an interview about that when I saw it there. But it seemed to be like a collaboration with the Phi Center and expanding off from this project that you had from shooting these videos in space. And so maybe you could give a bit more context for how this first LBE came about and this idea to take these 360 videos and to create more of an embodied kind of moving through a space and having a whole CGI recreation of the International Space Station and just kind of recount that story for what's leading up to this announcement this past week.
[00:08:26.107] Paul Raphaël: So Space Explorers as a series began, I believe in 2015 or so, when we started conversations with NASA in the hopes of one day sending a camera into space. But this long journey actually began more as building a relationship with NASA and gaining their trust and understanding what it meant to work with NASA. And so they were really interested in the potential of the medium, and they had seen some of our work. And so the first thing we decided to do together was to create a documentary series on astronaut training. And we figured, okay, this would be a great way for both of our organizations to get to know each other, but also to start telling the stories of the astronauts that we could then follow into space later on, if we got that far. And sure enough, after doing two episodes that were Earthbound, we got basically the approval to start developing the International Space Station cameras and to start figuring out how that would work, the technology, but also the logistics and then the creative, of course. And so after a couple of years, we sent two cameras to the International Space Station. These were both IVA cameras, so intravehicular cameras that were going to be complemented eventually by an EVA camera later to shoot outside, to shoot a spacewalk and to shoot images of the Earth. This whole project ended up becoming the largest production of any kind shot in space. It ended up lasting over two and a half years. But about a year into production, we'd been shooting and getting footage back and started putting together the episodes in VR. we really, really, really felt like we needed to be able to show this, not just to people who own headsets, but to all people, basically, to as large an audience as possible. We had the opportunity ourselves to show it, obviously, to people we knew, and these were people of all ages. And the reactions were just so incredible that we were compelled to find a way to have an independence of the, basically, the adoption of headsets by Meta or by whoever was making them. Now, obviously there's a large growing audience over there, but we really, really wanted to try and figure out a way to get it out there. And really also that was encouraged by the launch of the original Oculus Quest, which was kind of the first time you could even imagine doing something like this, right? Before you were either tethered to a computer or you didn't have six degrees of freedom. And we very early on had the idea of creating a real-time rendered life-size international space station as a map to explore the content. And then that evolved into, okay, well, what would that look like? How many people could we get in there? What would they look like? How would they find and interact with the content? How do you create some form of progression? How long is the experience? What comes before it, after it? How does it all end? You know, all these things came basically with time. And before we even launched it, though, we were really excited to see that at least on paper, it seemed like there'd be a way to make a viable, large scale, high throughput, experience. And so, you know, we're basically across the street from the FI Center here in Montreal and they've been huge supporters. In fact, they were the first investors in our studio too. So we've always had a very close relationship and partnership with them. They have throughout the years presented almost everything we've ever done at FI and around the world in festivals and galleries and have become in their own right experts, if not, you know, one of the biggest experts in VR presentation. since they've been doing it since we started off pretty much, and have since distributed Iñárritu's Carne y Arena. They're now distributing other VR LBEs. And so together, bringing their exhibit design, installation design expertise, and then everything we were doing in VR itself, we came up with The Infinite. And we first launched it in Montreal in the summer of 21, I want to say. And we were hoping to have shot our spacewalk by then, but that had been delayed. So the spacewalk footage was not part of the original launch. We instead created an experience that was shot from the cupola. So still images of the earth, but from inside the ISS. And of course, when we first launched it, it was the first time we were doing this. We learned a whole bunch of stuff. It was still quite a success. We really weren't expecting it to do that well. In fact, it did so well that we realized, OK, this is going to be now a huge part of what we do as a studio, not just this thing which we thought we were going to do as a one-off, but probably one of the main verticals of what we'll be doing moving forward. So from everything we'd learned in Montreal, we made an improved version for Houston that actually then also premiered the spacewalk footage. It did amazingly in Houston as well. As you might imagine, everyone we had worked with over there, the hundreds of people we collaborated with and their friends and families all came to see the show. We just had an incredible reception. The show has since toured a few more cities. So it was in Tacoma. It was in Richmond. It came back to Montreal last spring and it closed just a few weeks ago. And since then we've created a second kit. So we now have two kits traveling at the same time. One of them is currently in Vancouver. The other one is in Denver. We are building more kits and these are going to keep traveling and hopefully cross the ocean sometime later this year, we're hoping. And yeah, and we have since also been developing more shows of the LBE in this similar configuration.
[00:14:30.105] Kent Bye: Awesome. Yeah. That's a really great recap because I know that I had a chance to see it in Tacoma and I was just super impressed with not only the layout for how you're doing the throughput and rearranging where people are moving through the space with all the tracking technology you had to implement when it comes to these Quest headsets and make sure that you're not bumping into people or you have ways of tracking people that you're coming with. And so you had all these social dimensions that are on top of that, that make this a social experience, even though the core content is still coming back to a lot of these 360 videos. You had a lot of interactive, embodied, immersive spaces that you're walking around in that glue it all together. And so it sounds like it's been a great success as you've been continuing to show it around despite launching while the pandemic was still happening. And now with this announcement that just happened this past week, what can you tell me around whatever the news is in terms of what's new? Like what's next as you've taking these past successes with what you've been able to accomplish with The Infinite, now where are you going to go in the future?
[00:15:32.731] Paul Raphaël: Well, you know, one thing that has happened since we've launched The Infinite is it's definitely gotten a lot of people interested in exploring this format with us. So we are in talks with different potential partners. We are developing several different experiences. This round of funding that we just got is for a new show that we are producing ourselves. It's a new IP. I can't say too much about it other than it's taking everything we've learned from The Infinite and basically, you know, as much as our CEO would have liked for us to kind of take that recipe and just replicate it with a different skin, we're taking it somewhere completely different. We're changing a lot of things that we did on The Infinite. It's not going to be kind of a cookie cutter re-skin of it, but we're doing a lot of really cool new things with it. We're also going to be moving away from a more documentary form of storytelling as The Infinite was. So it's not going to be Infinite Part 2 or anything like that. And of course, we're going to adopt new technology, newer headsets. And yeah, I have to say it's the most ambitious and exciting and fun thing we've ever done. It tends to be the case of whatever the latest thing we're doing is, but this one is quite a leap for us. We're really putting it all on the table to make this thing. And yeah, we couldn't be more excited to say more about it to start with, and then to eventually show it to everyone.
[00:17:01.903] Kent Bye: Yeah, I know the press release was very skim when it comes to any specific details. I think you've been able to share a little bit more context around that in terms of like the formats. It says it's a virtual reality production, but you know, the latest headsets are mixed reality. You've done AR, you've done 360 video. And so what can you tell me in terms of like the formats that you're planning on exploring in this next piece?
[00:17:23.907] Paul Raphaël: So format wise, it's mainly going to be virtual reality. So we have obviously explored cinematic virtual reality, interactive virtual reality, augmented and mixed reality. I think we're still exploring all these things. This next show though, is going to be mostly virtual reality based. Yeah.
[00:17:42.693] Kent Bye: Okay. And I know that in the infinite, you had this sort of non-linearity where there's a bunch of content that you have, that you've produced, that some of it had been released and some of it you could only see in the infinite. And we're walking around a spatial context in order to get these little bits of information. I'm wondering what kind of story structure you're imagining if you imagine that it'd be something similar, where there'd be a certain degree of non-linearity or that you could feasibly go with somebody and they might see something completely different. And then at the end, everybody sees the same thing. So everybody, at least at the end, has a shared experience that they're walking out of with seeing the spacewalk. But as you think about the story structures, what kind of structures are you thinking about experimenting with as you go into your next pieces?
[00:18:26.512] Paul Raphaël: Yeah. So the infinite really was the result of, you know, we had this incredible bank of footage, right? That we were shooting up in space and it was originally going to be put together into a documentary series. And then we realized, okay, we can take that same content and explore it in a more interactive, more embodied, physically embodied way, which is not the case with what we're the next. show is basically designed from scratch natively for this medium, right? For this free roaming virtual reality experience. And so it's going to have a very different shape, very different format. It is going to be more interactive. It's going to have more of a, I would say, narrative structure that is, you know, yeah, again, it's hard to say much without revealing too much, but it is really a different beast. And I think The Infinite is an amazing and a kind of a one-of-a-kind project in that the preciousness of the footage is what makes that show. And then the layer of interactivity basically adds to the visual immersion, this physical immersion, and this social aspect. And these things are separate but intertwine in a way that augments the impact of the filmed content. What we're doing here is just, like I said, like fully original and made from scratch as a, you know, you're going to walk in there and explore this experience.
[00:19:56.256] Kent Bye: Yeah, it's obviously we're dancing around a lot of stuff that you can't really go into much details around. But I think about VR, I think about the different qualities of presence of like, there's a social presence where you see other people in the space where I think the Infinite did a great job of creating a social dynamic where you're able to see other people as a ghostly operation, but you're not bumping into them, but you're still actually able to maintain where the people that you came in with, with your party. So you're able to maintain this small clustering of people In terms of agency, you have the ability to kind of move around. So you have locomotion, but you don't have any narrative agency, except for you're choosing what bits to have. In body presence, you have the sense that your body's actually immersed in this space because you feel like you're walking through this environmental context. And then the emotional presence, you have a lot of the stories that you're weaving in. And so I'm wondering if there's any innovations that you've taken from the previous lessons or pushing forward any one of those dimensions of presence, because I I know that the social dimensions were actually really quite profound when it comes to having a shared social experience within the infinite that I feel like people tend to think of it as very isolating, but yet you can actually be very social within that context. And so, yeah, as you start to move forward, I'm wondering what kind of lessons you're pulling forward and other things you may want to be innovating on.
[00:21:10.162] Paul Raphaël: Yeah, for sure. I mean, we've learned so much, right? The experience was mostly designed before we had any experience doing anything like that. And then we obviously made some tweaks and adjustments before it was released with some test audiences. And then over the years, it's evolved a little bit, but the concept hasn't changed from what we came up with in a vacuum of having never really done this. So there's just so much that we're bringing. basically at the ground level on this new experience that will enhance the things people loved about The Infinite, correct some of the annoyances people might have had. The social aspect is a huge part of what people enjoyed and we're going to push that a lot further. And yeah, you know, just streamlining the whole thing in terms of also how do you make that user experience feel as diegetic as possible? How do you enter this building and just get lost in the story world as much as possible. That's another thing for us. It's how immersive can the experience be before you even begin the experience, right? The more onboarding and offboarding you can have, the more that world feels real. Right? And it's hard to do when it's an experience you just download and watch on a headset, because it's like you don't control almost anything other than the moment someone starts it. Yes, you can have a website, you could have maybe a larger story world, but when people are coming to your show, you control so much more. You know, it's why people love theme parks. It's why people love going to these immersive experiences. And, you know, we're definitely going to push that further as well. So, yeah, it's hopefully going to be an upgrade all around.
[00:22:50.831] Kent Bye: Awesome. Yeah. I know that you really had it dialed in when it came to the onboarding, just the spatial architecture of going into this space. It felt super exciting and really just the streamline for how to get that many people into VR so quickly. And then the offboarding of having all these immersive art pieces that you're walking through was really quite a nice touch to have that decompression process. So I really appreciated the whole experience of The Infinite. I'm just curious, it makes me think of the Star Wars had a whole hotel experience. Did you happen to go get a chance to see that?
[00:23:21.588] Paul Raphaël: I did not, unfortunately, no. Okay. Did you?
[00:23:24.669] Kent Bye: No, I did not. No, I know Noah Nelson from No Proscenium had a chance to go and I was just listening to him. And as he was talking about the onboarding for that, it just felt like the whole next level for immersive entertainment. It's a shame that they had to shut it down, but yeah, it sounded like something that was very unique of having all these actors and everything. Well, the other big news in the XR industry is the release of the Apple Vision Pro on February 2nd. Curious if you had a chance to try it out, if there's any plans of bringing any of your existing content onto the platform and what can you say about your experiences with the headset?
[00:24:00.680] Paul Raphaël: Yeah, I've definitely spent a lot of time with it. I mean, it's from the minute we tried the original Oculus, it was like, okay, so when's Apple going to take a stab at this? And it literally took 10 years, but here it is. And it is a very, very Apple device. It is incredible in so many ways. It also points at some incredible potential, you know, a lot of stuff that's not fully fleshed out yet. What's really interesting, probably most interesting for us is that has been a huge supporter of our studio, but storytelling has always been kind of a side gig, right? Like the main course has been games and maybe fitness. And whereas Apple is kind of putting entertainment front and center, right? Whether it be the virtual giant screen at home, 3D films, the spatial videos that they've been doing. So they are very, very interested in creating that kind of content. So that's super exciting for us because that's basically where we come from and what we've been doing for a decade. And it is a very, very, very capable device in that respect. The resolution, the dynamic range, the processors in there that allow you to play very, very high resolution videos. What's interesting is that Apple is focusing in terms of immersive or spatial videos, as they call them. Mostly they're focusing on 180 degrees, 3D, 180. Everything we've ever done was 360, 3D. So, you know, it's an interesting challenge to think in that sense, you know, if we're going to work with them. But you just gain so much within the clarity that is possible on that device that it's in some ways is a trade-off that is interesting still. Yeah, I think there have been enough opinion pieces out there that almost every opinion on this thing has been expressed. I think it's incredible for the industry, for Apple to be in the game. And it just feels like the time has arrived now, right? Everyone's the major players have shown their cards and the ones who have shown them in the past and removed them are about to show them again because clearly there's momentum. And what's also great is that we're really at an era where every revision of this hardware and software is going to be major, right? It's going to be really exciting to see how Vision OS 2 is going to look like, the Vision Pro 2 or whatever, the non-Pro Vision and what's going to come out of the gate at Meta in the next months and years and the other players. So it feels like a really fertile environment right now, more so than ever probably. And for a studio like ours that is 100% invested in exactly this, it's just incredibly exciting.
[00:26:39.596] Kent Bye: Yeah, I saw some folks saying that they suspected that some of the resolutions could go up to like 12K for Apple Immersive Video. And I'm wondering, like, what resolutions have you shot a lot of your stuff in? Is it that high? Is it up to 6K, 8K, 12K? Or imagine with like AI up-resing that it may not be. as much of an issue, but I think always you probably prefer to always shoot as high resolution as you can. And that I would imagine that you've been shooting at higher resolutions and having to kind of downgrade it in order to meet whatever the capacity of the existing headsets that are out there. But it seems like that Apple is really able to push the limits on some of the resolutions that are even available today. So curious to hear about some of your reflections on whether or not your backlog of all your material would be able to take advantage of whatever the full capacity of Apple's headset would be.
[00:27:27.550] Paul Raphaël: Yeah, so none of what we've ever made has ever been shown in its full resolution. It just wasn't really possible. The earlier stuff we were shooting was at 5K per eye. Later, it's been 8K per eye. With upscaling, you can basically go as high as you want. So, yeah, we're actually working on those tests right now, seeing how far we can push things. And, you know, what we did back in the day, too, to push the resolution further than the hardware could allow it is we did all sorts of, you know, we had our custom player where we could basically segment off the sphere and freeze bits of it, up-res bits of it, and put pixels in different places and even change the frame rate in different places. So, you know, in 2000, 14, we were pushing twice or three times as many pixels as the average video that was just raw MP4 that was just playing, right? And so eventually the hardware caught up to all our optimizations. But, you know, if a camera is moving, then you can do a lot less of this stuff. And early on, our screen system did be a little more static. So, yeah, I do think It's going to be really interesting to rediscover our content in this much higher quality that is now becoming available with the Vision Pro and other devices. I mean, we've been pushing the resolution on our experiences as even the Meta's devices have been evolving throughout the years.
[00:28:53.293] Kent Bye: Well, I know that Meta has taken an approach where they haven't really ever charged much for their immersive content, at least for Apple. They have Apple TV and they have at least some business model that they have, but it seems like that Meta has treated like the games you pay for, but all the immersive stories you get for free. So they've been subsidizing and funding a lot of content, including a lot of the works that you've done over the years. Is there anything that you can share in terms of, is that content locked in Meta's ecosystem forever? Or is there like an opportunity for some of that stuff that you've made to someday potentially show up on other platforms like the Apple Vision Pro?
[00:29:27.929] Paul Raphaël: Yeah, we own almost a hundred percent of everything we've ever done. Some of our collaborations with Hollywood studios are either co-ownerships, or we just simply were hired to create it. But the vast majority of our library belongs to us, and they were either timed exclusives with different partners that might've invested in the content, such as Meta. Some were never exclusives. Many of these exclusivities have run out. Some of them are still going, so it's kind of scattered. But yeah, you can definitely expect to see our content on different devices.
[00:30:02.176] Kent Bye: Awesome. Definitely looking forward to that. And as you start to look forward in your future productions, I know that with Infinite, you use the Quest 2. With the Apple Vision Pro, it's $3,500. I don't expect that to be a very friendly location-based entertainment device just because of like customized lenses, it's $3,500. So do you expect that as you move forward, you're going to stick with the meta quest hardware or are there other players that you're evaluating or anytime you're diving into these new efforts, there's always like a, see what's out there. So I'm curious if there's anything that you can commit to in terms of like the hardware that is out there in the ecosystem.
[00:30:39.379] Paul Raphaël: Yeah, we're obviously, uh, you know, looking at, you know, when we started the Infinite, we were developing on Quest 1 and we were lucky enough to have the Quest 2 come out just months before we were going to open. And so before we put in those massive orders, I think each kit has over 400 headsets, even though there's only 160 people simultaneously. We do need to cycle them, to clean them, to charge them, right? So it's pretty much a little over the two to one, right? And you need some backups, right? In case of failures. So that's a lot of devices. So certainly the price is prohibitive if it's literally six times or seven times the cost. That doesn't mean it couldn't happen, but it's certainly not as likely. So yeah, we're going to pull the trigger as late as possible on the device itself. It certainly will not be a Quest 2, but yeah, there's not a firm commitment yet done on any hardware.
[00:31:37.615] Kent Bye: Yeah, it was really nice to also see on the Apple platform some of the pieces that I've seen from immersive storytelling pass from like Battlescar and Gloomy Eyes from Arte as well as Atlas 5. Atlas 5 has a whole swan dive and Apple TV has all these immersive videos. You have 3D content from Disney. Curious if you've had a chance to just recreationally use the Apple Vision Pro or if there's any content that you've been really paying attention to both in terms of the immersive content, mixed reality, but also especially some of the 3D video. I personally was super impressed by watching Avatar in 3D and some of the Marvel movies in 3D on Disney Plus and Apple immersive video. I thought it was like super high resolution. Some of it gave me chills just being like, oh, wow, this feels like I'm actually there in some ways. And the encounter dinosaurs for me started to get at a level of fidelity of real-time CGI content that seemed to give me a little bit of a peek in a window into the future for where this was going. But just to hear some of your own impressions for stuff that's sticking out to you from what you've been able to check out so far on Apple Vision Pro.
[00:32:42.751] Paul Raphaël: Yeah, I've checked pretty much everything out. My favorite probably is the 3D movies, I think. Certainly just in terms of the experience of that, it is the best way I think you could watch a 3D movie. And it also is no non-cinematic project has ever been made with the ambition of a film, right? So to be able to watch something like Avatar 2 on a screen that's either as large as my living room or beyond if I go into one of those immersive environments, was quite striking. It's funny because the very first thing that I tried in a VR headset that made me realize the potential and that I actually used as a first demo before we had made anything with it was the movie theater app that was out there for the first Oculus Rift. And you could play anything you wanted in that movie theater. And it was like, okay, wait, if you can put a movie theater in VR and watch a movie, then VR literally owns film. It's like, you know, from there you can do whatever you want, right? And so it was interesting a decade later to be once again, kind of impressed by that concept, but for the first time at a level of fidelity that realized the dream of that 720p version of the theater I had tried 10 years ago. So that spent the most time doing with the Vision Prime is watching some films, especially 3D films. The spatial videos are incredible in terms of the quality. We have yet to be able to play any of our content at that resolution. And so just seeing that, seeing what video could look like, like immersive video can look like, was really, really impressive. And the other thing is, you know, again, everyone's already commented on the device, but the OS is just phenomenal. I mean, it's really, you know, it was one thing I was anticipating and expecting. Apple to do is to, you know, do something different and really interesting and intuitive with their operating system. And that's just a lot of fun. When I demoing the device, which is one of the pitfalls of the devices is sharing the damn thing. You know, people just get, you know, most of the time blown away by just the minute they're in the OS and they pinch and slide things around. And and that, you know, that's important, because if you're going to wear this thing and you're going to do anything with it, you want the baseline to be as intuitive and pleasurable and smooth and magical as possible. And I think that's really, really a big, big win. And it's, I think, going to inspire a lot of different ways to interact within virtual reality and mixed reality. But yeah, the sharing of the device needs to be worked on. Guest mode, calibration every single time, single user, you know, they really need to work on that because again, this is a device people are going to Not a lot of people are going to buy. It's just the nature of the cost and the level of evolution of this ecosystem and this hardware. And so everyone who does buy one, whether it's for themselves to share with their friends and family, or whether you're at a company, it's just a lot of friction to the point where you know, it will discourage people from sharing it as much as they otherwise would. You know, if every time you're going to pass it over, you've got a minute just to get back to the OS, that's a hell of a lot of friction, right? When we first started working on the original Rift, And we had that very first video demo that we had. I just popped it on my parents' heads and they saw it and they didn't have to know how to do anything. I can't do that. You know, I haven't shown them the Vision Pro because, you know, they're in their 70s and it's like, all right, well. You have to go through the whole calibration thing. And it's kind of a, we'll get there eventually. But it's certainly much more friction than, hey, look at this thing. Boom. I get that you have to calibrate your eye tracking. Hopefully, that can be quicker. And hopefully, you can save multiple profiles. And yeah, that, to me, is the biggest thing that needs to improve right now.
[00:36:43.154] Kent Bye: Yeah. For me, I watched The Marvels with my wife last night on Disney+, and then I had realized that, oh, well, it's also in 3D, and so how do I watch it at the same time? Because I would love to see it in 3D, and so I'll probably experiment a little bit just for the sake of science. like, how does this feel for me to be in VR, but she's not in VR, but the audio is not good enough for me to just use the onboard headset. So I wanted to connect it to my soundbar and to have the full immersive 5.1.2 audio from my Vizio soundbar. So there's like all these things where it's not quite like a self-contained experience, but yet if you are watching a movie like that with a partner or with other people, then you have this asymmetry dimension of either one person's going to be in, other person's not. And if it's too dark, is it, am I going to get all those warnings of like, you need to have more light to track your hands. And then, so anyway, there's the things like the shared experience of watching cinematic movies is something I'll, I'll be experimenting with a little bit more. Yeah.
[00:37:41.018] Paul Raphaël: And I would imagine that syncing an Apple TV to a vision pro should be on the, on the roadmap. Like, you know, you can have people watching it on the TV. It's perfectly synced to the headset.
[00:37:52.097] Kent Bye: I think it's already there with either with show that is possible. Yeah. I've seen some people do it. I haven't actually tested it yet, but yeah, it's either you do a FaceTime share on Apple TV, which I got an Apple TV to also kind of play it out or there is mirroring. So I'm going to like test some of that out. But yeah, when you're bought into the Apple ecosystem, stuff like that should be easier when you're not, it kind of gets a little bit more friction.
[00:38:15.653] Paul Raphaël: Yeah. And we're jumping through the extra hoop of being in Canada. So we can't actually use our, uh, I'm not using my personal account on the device. So I can't do a lot of the sharing stuff, connecting to my laptop or things like that as easily. But yeah, hopefully that changes soon.
[00:38:32.367] Kent Bye: Great. Well, I know you've got this big news that you're announcing and we talked as much as we probably can. Is there anything else that you wanted to talk about or say, as we reflect on where the XR industry is right now?
[00:38:42.477] Paul Raphaël: Yeah, I think it really, I think, like I said earlier, it's just a really exciting time. If anything, we've got now, like I said, these multiple verticals that are kind of major focuses of the studio. What we've always done, the cinematic content, and now more and more these location-based experiences. So I think the challenge for us in the next couple of months and years is going to be to find the scale to be able to do both those things because Either one of them, I mean, making a show like this show we're preparing right now is, I'd say, the equivalent of making a feature film. And that's an all-encompassing exercise, right? And everything else we're doing, it's like we have to grow to do anything other than that, right? And so we're developing a lot of things in parallel. But the minute any of these things go into production, we're definitely going to have to scale, which is exciting, but also a huge challenge. So yeah, I think it's great to see the industry have such momentum after all these ups and downs over the last 10 years that we've been doing this. So I think it's a great time. You know, if anyone's listening who's been in this space and who's bailed and who's been kind of yearning to come back, I'd say, well, now might be a good time to do it if you haven't already noticed, because yeah, there's just a lot of not just demand, I think, from the industry to create content, but a lot more, you know, as we've seen with the success of The Infinite, people who are willing to buy a ticket to do a VR experience, right? And that, you know, there's definitely been sentiment, you know, positive and negative sentiment throughout the years. And it does feel like the sentiment is positive. Sure, a lot of people are mocking the price of the Vision Pro or things like that. And it's like, well, I think we need to have hardware coming out across the range. Definitely an accessible device like the MetaQuest is incredible. What they can do for that price is mind-boggling. And then at the other extreme, what Apple's doing with the Vision Pro is mind-boggling, even at that price. And there's going to be sweet spots in the middle there, right, where other manufacturers and maybe these two players are going to slot in. I don't think we should poo-poo one end of the spectrum or the other one. This is a great spread, I think, and the more options in the middle that we have. And again, the quality of both these devices is also going to push other manufacturers to really give it their best, to bring their A-game, right? You can't just make a headset and put it out there and hope people are going to do something with it. Like, no. You need these things to be user-friendly. You need these things to have a great content library, great support. So both the main players right now are doing that in their own way. And this is what makes it feel like the medium is kind of starting to arrive really. And yeah, I just couldn't be more excited about that.
[00:41:40.413] Kent Bye: Awesome. And I know that as I've been interviewing people over the years, I always like to end on asking about the ultimate potentials, because we're at this stage where we've seen new potentialities with the Apple Vision Pro, but I'd love to hear your latest thoughts on what you think the ultimate potential of spatial computing and immersive storytelling might be, and what it might be able to enable.
[00:42:01.983] Paul Raphaël: Well, I mean, as I said a little earlier, I do believe this is going to usurp at least where we're coming from, which is storytelling, traditional forms of storytelling. And that doesn't mean they're not going to exist anymore. I think this is going to be the next main way people are going to entertain themselves. And that way they're going to do anything, really. The way we're going to interact with the digital realm is going to become more spatial and more natural, really. And it's been multiple decades with cinema over a century where our main way of representing art has been abstracted, right? It's been two-dimensional. It's been in a frame. Visual art, I should say, and I'm not talking about performative art. And that's the same for applications, for the internet, all these things. There's just so much more density in the information and in the potential of immersion within all these things that I don't think we can even imagine how much that's going to change the nature of the content that's going to be created, of the applications, but the way that's going to affect us, right? Which obviously could be used for good or bad. As an artist, as someone who's using this medium creatively, I just see limitless potential. And I plan to explore as much of that as possible. But yeah, it's definitely transformative. It's definitely less of a pipe dream every day that goes by. It's becoming realer and realer. And it's as real as it's ever been by now. And I don't think you need to project yourself that much further into the future to see this becoming you know, I think the main way we interact with information.
[00:43:46.706] Kent Bye: Awesome. And is there any information on like timelines for where we can expect to hear more information or like a launch of your next plans?
[00:43:55.752] Paul Raphaël: Well, we have announced that this is coming out next year. So if we stay on schedule, you should be hearing more before then. That's okay. as much as I can say, unfortunately. But believe me, we're bursting at the seams to say more, because this is a lot of fun, a really, really, really lot of fun.
[00:44:17.637] Kent Bye: Awesome. Well, certainly look forward to hearing more information about whatever you have cooking up coming next. I know from my own experiences of your prior work and experiences of the infinite, I know that you've got a lot of right combination of things to pull this off. And so I'm really looking forward to seeing where you take this in the future. And thanks so much for taking the time to share a little bit of your journey into the space and what you've been doing in LBE. VR experiences and also where things might also be going with Apple Vision Pro as well. So thanks again for taking the time to help share all your latest insights.
[00:44:49.054] Paul Raphaël: Thank you, Ken. It's always a pleasure and I'm sure, I hope we will be doing this again soon. So take care until then.
[00:44:55.640] Kent Bye: So that was Paul Raphael. He's one of the co-founders of Felix and Paul Studios. So I have a number of different takeaways about this interview is that, first of all, well, just to hear that location-based entertainment, or LBE as it's commonly known within the XR industry, was so successful for them with The Infinite that they decided that it's going to be a significant part of their business moving forward. And so they've gone out and raised a multimillion dollar round of financing to do this brand new IP that's going to be launching sometime in 2025. Not a lot of other details as to exactly what they're working on, but it is going to be a VR experience. It's going to be original IP. They're going to be focusing on kind of like full immersive theater, like setting and context so that the onboarding and offboarding is going to be really feeding into this immersive experience. And they're going to be building ground up for the medium, which means that they're going to be leveraging all the different affordances in a way that is going to be taken advantage of the medium, including a lot of the different social dimensions that they're exploring within the context of the infinite. So The Infinite was based upon a lot of documentary footage and reminds me of the conversation that I had with William Uricchio talking about how documentary is often at the forefront of a lot of new explorations within the context of the medium. And Felix and Paul have definitely used the documentary form to explore the potentialities of what the affordances of VR are and now they're taking all those lessons and starting to get into more fiction-based content, so that they're not going to be looking at cinematic 360 video documentary footage in this piece, but it's going to be more CGI, VR, immersive content that they're developing from scratch, that they started to explore a little bit more. Myubi, they started to have a little bit more of interactive components within the cinematic VR. And then with Jim Henson's Storyteller that they've been working on for a number of years, but was able to premiere at Venice back in 2023. And I had a chance to actually talk to Paul about that. And the whole studio, they're building up a much more capacity to have more of these types of CGI-based VR experiences that they're working on. So with the success of the Infinite, they had a lot of different people coming up to them and wanting to partner with them. So it's a little bit unclear to me. They say it's a brand new IP. I don't know if it's a brand new IP for them or a brand new IP just in general, something that's never existed, or they're going to be adapting something. They say it's a brand new IP, so I'll take it at their word that it's something that we haven't seen before. Like I said, there's no details. I tried to get as much information as I could without being annoying of trying to have things that he obviously can't talk about. So, I guess we'll kind of wait and see to see. But they did say that it's a brand new IP. I did have a previous interview about the Infinite that I had a chance to see it in Tacoma, Washington back in the spring. of 2022. And so I have an interview with Felix Lajeunesse, as well as one of the executive producers from the Fi studio, Julie Tremblay, when we talk a lot about the infinite, my experience of that and that conversation. So highly recommend going to check that out to get a little bit more context, because like Paul said, they're moving much more into having LBE as a key part of their business as they move forward. So 160 people at the same time is an incredible throughput. I haven't seen very many other folks pull that off with that many people within the context of an hour. That's in small groups. And so it's kind of a rolling 160 per hour, but still that's an amazing Logistical feat that they were able to do that in collaboration with the Phi Center that we're helping to kind of sort all that stuff out and so 160 people they would need like 400 VR headsets and they're gonna decide what they get once they're closer to launch to see whatever the market is Could be a quest 3 could be quest 4 could be you know, some other headsets that haven't even launched or announced yet So that's all to be determined And in the last half of this conversation, just to get a lot more context for the Apple Vision Pro as this entertainment first type of device, where one of the things that Apple has done is really ensure that they have things like Dolby Atmos and other formats that you see for 2D content that is supported in the context of the Apple Vision Pro, where you don't necessarily see that same type of emphasis on the Quest 3. So with the very high resolution that they have, they're able to give a very crisp, like what feels like around 4K type of experience. It's not actually 4K, it's a little bit less. The iFixit teardowns showed that each of the micro OLED displays were a 3660 by 3200, or 23.4 million pixels total. And so that's less than 4K per eye, but also normal 4K resolutions aren't actually 4K, and it's actually technically larger than what most TV sets call 4K. So when you see the stereoscopic 3D movies, you're essentially getting roughly around 4K per eye. You get to see stereoscopic effects that are absolutely incredible. And so the 3D movies within Apple Vision Pro are one of the most compelling things that I've seen so far. And Paul was saying that most of the content that Philips and Paul has done has been 360 video. And a lot of the Apple immersive video content that they're showing on Apple TV is just 180. So I don't know if that means they're going to try to do this translation of some of their existing content into a 180 format or if they're going to try to push Apple to expand out to more 360 video content. So they'll have to do some adaptations to a lot of their content. They haven't necessarily been producing a lot of 180 content. Although they did show some of their content that they had shot in the context of some of MetaQuest's theater that they have during some of the MetaConnects during the pandemic. But I think it was not as good of an experience because a lot of their content really affords you to look around full 360. So to translate it into 180 is missing a lot of what they've been able to capture already. So be very curious to see how they handle that. if they're gonna launch their own app. But it was great to hear that Paul said that they own most of the content that they shot, aside from some of the stuff that they did in collaboration with other folks, and that you can expect their content to be appearing on other platforms, including things like Apple Vision Pro at some point. So it's very interesting to hear that Paul said that they've never been able to show their content at the full resolution that it was shot, so very curious to see some of the stuff that they have been able to shoot on that amazing displays that you have on the Apple Vision Pro. So yeah, just a very exciting time for the industry in general. And they've certainly been at the forefront of pushing forward what's even possible with immersive storytelling in the context of cinematic VR. And they're moving into more CGI-based, location-based entertainment content. And I'm very excited to see where they take this in the future, where they're bringing all their lessons together to push forward what they say is the most ambitious project that they've done yet. And it's equivalent of producing a feature film. So very excited to see where they take it here in the future. So, that's all I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. If you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a less-than-supported podcast, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voices of vr. Thanks for listening.
