The White House VR documentary People’s House by Felix & Paul Studios won a Emmy for the outstanding original interactive, and I had a chance to talk with Paul Raphael about how the challenges of producing a high-profile piece. They didn’t know how many rooms they’d be able to shoot, and President Obama was such a fan of the project that he literally opened doors for the crew to record more than twice the number of originally scheduled rooms. They were limited to only two 15 minutes interviews with Barack and Michelle Obama, and so they collaborated with speech writers to capture memories and stories for this virtual guided tour.
Felix & Paul Studios create their own VR camera hardware, and they’re starting to use their fourth generation cameras while designing a next-generation, digital lightfield camera. Raphael said lightfield VR shoots are essentially visual effects shoots, which require shooting in different wedge segments that need to be composited in post-production. He also said that they’ve been consulting with most of the major HMD manufacturers including Facebook on an open standard for immersive 3D audio. Even though they’ve been creating a lot of hardware, they’re more interested in using it to stay on the bleeding edge so that they can continue to innovate and push the creative limits of what’s possible in immersive storytelling.
LISTEN TO THE VOICES OF VR PODCAST
Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Support Voices of VR
- Subscribe on iTunes
- Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye and welcome to the Voices of VR podcast. So in today's episode, we're going to be going behind the scenes of the making of a high profile documentary, which was a 360 degree tour of the White House. It was a collaboration between Felix and Paul Studios and the White House and Barack Obama. So he tells a story of how the project came about and just the logistics of how they even put the piece together and the constraints they had to deal with, with the limited time and access that they had to the president. So their video, The People's House, actually just picked up an Emmy for the Most Outstanding Original Interactive Program, and it premiered at the Tribeca Film Festival this past April. So I had a chance to catch up with Paul Raphael to talk about this experience, but also where they're going in the future with volumetric video and sound design and all the different consulting that they're doing in order to help forge the future of immersive storytelling. So that's what we'll be covering on today's episode of the Voices of VR podcast. So this interview with Paul happened at the Tribeca Film Festival, which was happening in New York City, New York, from April 19th to 29th, 2017. So with that, let's go ahead and dive right in.
[00:01:20.906] Paul Raphael: I'm Paul Raphael, co-founder, creative director, and director at Felix and Paul Studios. We've been making cinematic VR since 2013, started with Strangers, and I've done about 20 pieces since. Our latest here at Tribeca is The People's House, a project we were incredibly lucky to be able to make with Presidents Barack, Obama, and Michelle. just around the time it was towards the end of their term, so it's a pretty personal and kind of intense piece. It's a tour of the White House. It's about the history of the White House, but mostly it's also very much about their personal time at the White House, and it's all sort of intermingled. It's also a quite different piece from a lot of the stuff we've made in the sense that it's one of the first pieces where we're not so literal about the viewer being there, you know, physically, which has been kind of a trademark of almost everything we've done. So with the exception of the moments where you're actually in front of Barack or Michelle as they talk to you, when you're actually going through the house, it's more of a internal going through the memories of Barack and Michelle of their time through the White House. So it's this kind of very subtly floating camera, which really was a result of We knew when we were making this that this would come out at a time where they would no longer be there, of course, and that this was really going to be looked at through a lens of the past. And we wanted to feel that way because I think it was a way to make it more timeless and more appropriate to the context.
[00:02:55.997] Kent Bye: Yeah, to me, I think that this is a genre that we're going to see a lot more of, which is this guided tour type of experience, whether it's a docent giving you a tour of art through a museum, which is something that's pretty established, or someone giving you a guided tour through a place. But, you know, you could take a tour of the White House, but you're not going to get those personal anecdotes and stories of the meaning that's attached to people like the President or the First Lady, and you're not going to have access to a lot of these places as well. Maybe you could talk a bit about, you know, that process of a guided tour and how are you maybe looking at what was already established in other realms of guided tours and how you're trying to translate that into VR.
[00:03:34.862] Paul Raphael: Yeah, so I think, you know, there's a version of a tour that can be very dry, very didactic and strictly informational and we didn't want to do that. We really wanted this to be more cinematic. You know, the educational, quote-unquote, side of it is inherent to this kind of piece but we wanted it to go beyond that and really be a very emotional I think it's one of our most like cinematographic pieces I think it's you know I'm really proud of the cinematography and also the score I think is really magical and every room is a little world in and of itself you know every room is very distinct especially when you're talking about the colored rooms and Of course, the iconic rooms, such as the Oval Office or the Situation Room, and the score kind of really emphasizes that. Every room has its theme, even though the whole score kind of holds together. And, of course, the anecdotes by the President and the First Lady really make it, you know, a very special package overall, yeah.
[00:04:35.784] Kent Bye: Well, the one thing that I wanted to see that I didn't get to see is, like, their bedroom. And was that more of an issue of privacy or security that you couldn't get access to, like, every room in the White House?
[00:04:47.034] Paul Raphael: Well, I mean, we do have a few of the private quarters. We actually began with a much smaller set of rooms that we would have access to for security reasons, of course. But the president himself was a huge fan of the medium and a big proponent of the project. So this is actually our second collaboration with them. We did the Yosemite piece through the ages in the summer of 2016. And whenever we would hit any sort of roadblock with the Secret Service, Barack himself would usually be the one who quite literally opened doors for us and led us into rooms that we wouldn't have had access to otherwise. So I think we have about 20 rooms in the piece. And initially, we would have had maybe access to half of that. So it's still quite the tour. Yeah, maybe you could tell me a bit more of the story of how this piece came about Sure, so we'd actually been in contact with the White House for quite a while So probably at this point about two years ago was when we first had contact with them so with with their digital media team and at the time it was They were very interested, it was kind of a curiosity, but there wasn't much urgency or they didn't really see how it would be worth their time. And then a few things happened. First of all, things like 360 video and, for example, Facebook's Newsfeed, you know, kind of opened up the door to, well, yes, we would make this highly immersive, high-end VR version, which would be accessible to the few millions of people that have access to a Gear VR or Rift, but it would also be accessible to the billions of people that are on Facebook through this new platform. So that was a big accelerator in our discussions. And then we were always kind of working on this project, this is what we were kind of pitching them. And we never even thought that was really going to happen. And at some point, they kind of got in touch with us and were like, hey, the president and his family are going to Yosemite in a couple of weeks. And it's going to be beautiful. It's going to be about nature conservation and the environment. And do you want to do something then? And we're like, yeah. I mean, you know, who knows if this White House thing is ever going to happen. And yeah, that sounds amazing. Let's do it. And so that was kind of our unexpected first thing we did with them. And then once we did that, the president absolutely loved the experience, the result, and I don't know, you might have seen that photo of him watching the Yosemite piece. So he was a huge fan, and then he kind of helped accelerate the production of this more ambitious project, which we then started developing and shot in November.
[00:07:23.705] Kent Bye: So were you a part of the team that was actually giving Barack his first VR experience? Maybe you could talk about that.
[00:07:30.947] Paul Raphael: Yeah, well, so his very first VR experience, I think, was something he saw on cardboard. So I don't remember what that was. But we certainly were the first ones to make something with him in VR.
[00:07:44.012] Kent Bye: Yeah. And what was his reaction to the piece that you made about Yosemite?
[00:07:48.014] Paul Raphael: You know what? I think, and he talks about that himself in the piece, but a lot of what he and Michelle stood for was inclusion and opening doors, opening the doors of the White House to the public. So they had one of the most open access tours going on at the White House while they were in office. And they wanted to make that available to even more people through virtual reality. And this was the perfect sort of medium to do that.
[00:08:15.083] Kent Bye: Yeah, there was a great moment when they were in where Roosevelt would record his fireside chats and he starts to talk about the evolution of communication mediums and kind of directly saying, OK, now you're participating in this next evolution of virtual reality as kind of looking at the history and evolution of technology. and how, as a communications medium, politicians were using it through different ways. So, yeah, I don't know if you can comment on that in terms of, you know, he's saying all these things, and did he write the script himself? Or maybe you could talk a bit about, you know, as he's telling the story, where that came from.
[00:08:46.835] Paul Raphael: Sure. So, you know, we actually started working on this piece early in the fall, and the process basically was, we came up with a set of questions for the president and we wanted to start with an interview. We didn't want to script anything. We wanted to get this very natural, these very spontaneous responses. So the very first thing we did was shoot those interviews that you see parts of in the piece. And then from those interviews, we distilled what seemed to be the through line that the story of the piece could be, right? And that's how we sort of started sculpting the narrative, and then it was a process of bouncing back and forth between ourselves, the President's speechwriters, and the President Michel themselves. So it was kind of bouncing it back and forth for a couple of weeks, and then we sort of started making kind of a rough edit, they then recorded the voiceovers, so there's both voiceover and interview material in the piece and it all kind of comes together to tell the story.
[00:09:46.950] Kent Bye: Yeah, I felt like that was a bit of an interesting tonal shift that would happen. Like, I could tell when he was reading a script and reading it. And, you know, there's different points where he's making announcements. Okay, this is where I was when I announced the killing of Osama bin Laden. And so I felt the same thing in the previous piece that he was in, where there was like this kind of subtle, or maybe not so subtle, political messaging versus the more emergent, authentic, emotional memories that were arising. And so it makes me kind of question and wonder whether or not those scripted, you know, kind of the needs of having that nice narrative arc that you're having from the scripting but yet you lose the live vitality of the authenticity perhaps because you're kind of coming from a different realm of your mind. versus something that is emergent in a conversation. So, I guess, while you were making it, you didn't think that you could completely construct a narrative solely with that, because that's where you were starting. But it felt like you needed to kind of turn to that. But to me, as I'm watching it, I kind of feel these different tonal shifts of things that feel like it's coming from his brain and other things that's coming from his heart.
[00:10:50.454] Paul Raphael: Yeah. Well, you know, one of the reasons for that is we didn't have unlimited time with him for the interview. I believe the actual interviews lasted between 15 and 20 minutes for each, the president and for Michelle. And at the time, we still didn't know what we were going to have access to. And this is true with a lot of these VR documentaries, especially with high-profile personalities like the president and his wife, or whether it's the LeBron piece we did a while back, or other projects we're working on. You really have to think on your feet. and you need to really be malleable in your approach to how you're going to make the best piece possible. Because the second you're rigid, it's either going to break or it's not going to be really any good. So we really needed that voiceover and I like the modulation. Of course, the strongest moments are the ones where it becomes really personal, but at the same time, he didn't have that extremely personal stories to say about every room in the house either, right? So, you know, some of it is anecdotal, some of it is historical, some of it is very personal and, you know, that's a modulation that we did end up playing with and it was also a necessity to make the piece come together.
[00:12:00.597] Kent Bye: Yeah, I think that totally makes sense. And I think that the thing that I was appreciating about the piece was that also there was this almost like annotation or meaning of the emotional history of these different places. Not that, you know, you kind of go into the history of the specific places, but his own personal memories of this is where this happened within his time there. So I could see that given those constraints, that's one of the ways that you could do that. The one other question that I had was, I kind of wanted to see a model of where I was in orienting. As I'm going through this tour, usually when you go a tour, you're walking through and you're able to maintain that integrity of those spatial relationships. But by kind of teleporting all around, it was hard for me to get a real good map of where things were at. So I didn't know if that was something you thought about or trying to experiment with how to give you that mental map, or if that was a security issue where you weren't able to actually do that.
[00:12:53.388] Paul Raphael: You know, I think that was kind of a result of the direction we chose to go with this piece, that it would be not so much a didactic educational piece as we wanted it to feel more like a film, like a more emotional, more personal story. So what's fun is that In a lot of the rooms, you can look around and see the other rooms and kind of piece parts of the White House together. But, you know, at no point do you select where you want to go or see a map, which, you know, there's no graphic overlays or, you know, like motion graphics or anything like that in the piece. We wanted to keep it very real, very At most, it was maybe more of an internal kind of journey, but very organic, you know, and not so much blueprints, overlay, or anything like that. So, yeah, that was kind of a choice that we made. And, you know, if you're curious about the geography of the White House, you can find that, you know. It's not inaccessible. Interestingly, though, there were restrictions in, even in some of the rooms where we were, where we could put the camera and what we could or couldn't see. based on for security reasons. So, you know, they didn't want someone to be able to watch this and be able to triangulate, you know, lines of sights and all sorts of things, especially near windows. So a part of it was also security.
[00:14:10.389] Kent Bye: Is that part of the reason why you shot at night a lot of it?
[00:14:14.125] Paul Raphael: Yeah, so sometimes, you know, not being able to see through the windows was a concern. Some of it was also, you know, the time that we had and the access that we had, and some of it was also part of creating those modulations. I know some rooms, you know, if you're talking about the green room or rooms where there were soirees or events like that, you want to be more in the context of these usually happen at night you know and the ambience and the soundtrack also contribute to that and then when you're in the Oval Office and he's talking about the beautiful light coming in well of course you wanted to be during the day so that was also part of it.
[00:14:47.146] Kent Bye: And can you talk a bit about some of the other projects that you have coming up? I know that you have Mayubi, that's a big, long narrative that's going to be released at some point, but you have a lot of different projects in the pipeline, working with different Hollywood studios. I'm just curious to hear where you're at right now and where you're going in the future in terms of these longer form narrative pieces, what you can tell me about what you're working on right now.
[00:15:08.780] Paul Raphael: Sure. Well, you know, as you can probably imagine, I can't go into too many specifics about unannounced projects, but what I can say is that with MIUBI and the People's House, we've really kind of pushed the length of what these experiences can be, and we've seen the reaction You know, we've had at least a few hundred people see both of these experiences by now and overwhelmingly people have been very comfortable with these lengths and either comment on the depth that it has allowed them to kind of explore in virtual reality for the first time Many have said it did not feel like they were in the case of Mayubi, which is about 40 minutes long. Many people don't feel those 40 minutes at all. They feel like it's much shorter. So that's really encouraged us to explore that further. And Mayubi was also the first real scripted narrative piece we've done, with the exception of a couple very short. companion pieces we've done for feature films. So it's just been very encouraging. So a lot more storytelling, narrative projects in the works, a lot more longer, more ambitious, more robust, experiential documentaries like The People's House with other interesting characters coming up. And we've been doing a lot of Cirque du Soleil stuff. So the latest we've released was O, and we've got another one coming out very, very shortly. that will be kind of the wrapping up that series of experiences that we've been doing with them. Yeah, that's all I can really say right now, yeah.
[00:16:35.892] Kent Bye: Well, I'm curious to hear your thoughts on this current trajectory of volumetric video filmmaking, because right now you've built your own camera. It's, I think, certainly the most sophisticated, high-fidelity. To me, it's the best stereoscopic effects, but you have a lot of, like, time-lapse photography, a very keen awareness of different lenses. So the quality that you're able to achieve with your pieces is above and beyond anything else that's out there. But we're kind of moving into this new realm where you're either using the Surround 360 to do post-processing, creating a point cloud workflow using Otoy's system and then putting Orbex files into Unity and then started to just kind of change the pipeline as to what you may be currently doing with your 360 video. So I'm just curious to hear your perspective as a creator, what makes sense to use volumetric and kind of go through that whole process and the types of pieces that maybe that's more documentary where you can't recreate things. So your thoughts on sort of the strengths and weaknesses of your existing method and what you're able to do and not do with this new volumetric video. Sure.
[00:17:40.907] Paul Raphael: So, you know, we've been developing our internal tools from the very beginning and it's not been a static thing. It's been very iterative and I think one of the reasons we've been able to have this level of quality is because we're not making a product that needs to, you know, be, you know, when you're releasing something, you need to lock the specs, QA it, manufacture it and then release it and from the moment you've designed it to the moment it's out, there could be a year or more. not having that sort of burden has allowed us to be very fluid and constantly iterate on our cameras. And so we're on the third generation of our camera, we're about to go into our fourth, fifth is being designed as well, and positional light fields are part of that development. And I think that up until now, It's been very hard to get a very good non-positional image, as you can see from a lot of stuff that's being made, and it's obviously an order of magnitude harder to get a good positional tracking out of a camera as well, and it's really starting now to reach a point where you can get a level of visual fidelity in positional that is not overly distracting, that doesn't have too many edge artifacts, or that doesn't have to restrain viewing volume. So, you know, we're just now starting to really explore these tools. Of course, they're much heavier, both in terms of how you shoot with them and the post that's involved, so the costs go up and you can't use those on every project. You really have to think differently to make those. We could not have brought such a device into the White House. It just for many reasons would not, and we considered it, but it would not have been viable. But mostly it's a dramatic shift in how you build an experience. So, you know, if there was a leap between going from cinema, traditional cinema in a frame, to the kind of experiences we've been making in VR, there's another leap between that and positional in terms of how you craft an experience. You know, our emphasis has always been on presence, and that's kind of the foundation of everything we do. Any emotion that you want to elicit in a viewer is going to be made or broken by how present they feel in a virtual reality experience, which is not true of film. And with volumetric comes a level of freedom that also creates more edges that you can rub against and that betray the technology. And whether it's the visual fidelity or the viewing volume or just the fact that you no longer know where the viewer really is going to be. That's a major thing. In VR, you don't know where they're going to look, but you know where they're going to be. In a positional piece, you don't even know that. I mean, you know within a range. But right away, that makes eye contact If not impossible, very difficult. Unless you're in a real-time rendered CG animation where you can kind of program the eyeballs of a character to follow you around, if you need to do eye contact. But beyond eye contact is, you know, just going through geometry, if you so choose as a viewer, there's nothing that will break an illusion quicker than, I just went through this thing, it's not real, it's a VR simulation. Whereas in a quote-unquote linear 360 3D video, that cannot happen. You know, you kind of, once you accept the fact that You can't move, and that's part of how the stories are made and how we sculpt the experiences that kind of don't emphasize the fact that you can't move. You know, that gives us more freedom as storytellers. So, all this to say that it's kind of a new frontier that we're very excited about, very interested in, that we're definitely exploring, but it's not to be underestimated how much of another leap it is on almost every level, which is what virtual reality has been from the beginning. It's been a very highly technical and creative challenge to make something that really works, and we're constantly learning, and this is another chapter in that.
[00:21:36.455] Kent Bye: Yeah, I just had a chance to see the Hallelujah piece here by Lytro, and just wanted to make a few comments to kind of follow on what you're saying here, both the edge detection, so, you know, just to give people kind of a visual as to what I've seen, the panel of the Lytro camera is basically like this big array of cameras, like 95 different cameras, and It's like a 90-degree field of view, and so they have to shoot in five different wedges. And so they have this experience where you have a person in five different locations. However, the lighting is basically the same in all of the different iterations of the person. So you have this kind of impossible lighting, such that you have one light source that To me, I kind of notice it as a way of like, okay, the lighting effects here, they need to think about it. If they're shooting in five different wedges, then you need to think about how do you create a kind of an organic lighting in a way that looks consistent across the screen. I don't think they necessarily pulled that off. But also, as you kind of move around and kind of break it, you start to see the edges as to which the volume under which you have the light field camera shooting is not capturing the occluded parts behind people. So then you have to kind of fill it in by this, you know, just filling in the textures as to where you would have this impossible area, which then begs the question as to whether or not you do light field shooting of someone up front and then do photogrammetry behind it so that you don't have that, or then it sort of changes the pipeline of compositing and all this stuff. And so, to me, it makes me wonder whether or not the digital light field is going to be even viable for live documentary stuff, where you don't have the time to be able to recreate a scene. If you're trying to capture something in the moment, then are you going to be able to have a 360 light field camera that is able to capture all those iterations of everything, and is it going to be economic enough to even produce that, or is it going to be such a high level of cost and production that it's going to be on a green screen, on a stage, and then you're going to basically shoot live actors, but then green screen and everything else.
[00:23:29.717] Paul Raphael: So right now definitely a light field shoot is a VFX shoot basically. You have to separate the quadrants like you said, you often have to separate the elements that you're shooting, the people from the background and all this. almost every shot, no matter how normal or natural it's meant to look, is that avatar-like complexity in terms of building that shot. So, that doesn't mean it's impossible, yeah, but it means it limits what you can do, the costs are obviously much higher. Yeah, and then what you do with that, creatively also, is a big challenge. Whether it's gonna be viable or not, you know, there's always someone somewhere that's willing to go that extra mile, financially, time-wise, and complexity-wise, and also taking on that creative challenge. to do it, right? If it's possible, someone's going to do it. And with time, all these things are going to get easier, cheaper, and you're going to have more flexibility. So if you're going to start doing this today, you're going to have many more challenges, but you're also going to be learning before everyone else does. You know, if you start, the sooner you start, that was true of VR as it is today, a few years ago when we started doing it. I see it all as a continuum and I'm thankful that it's there, that we can play with it and it's exciting to explore despite the limitations that are there. So I try to always be very open-minded about these limitations because we should always be critical but we also need to understand that Hey, there's a lot of really amazing, talented people, both on the technical side and on the creative side, who are just trying their best to make these recreations of reality and expansions of reality a reality. And it's all just amazing that this is happening now, especially when you think of other mediums like film that haven't evolved that much. in a very long time, so this is a very exciting time for this new sort of storytelling medium to explode in real time, you know, as we experiment with it.
[00:25:31.582] Kent Bye: And have you been in contact with Otoy to use a similar pipeline that the Surround 360 Facebook camera is using in the sense of, you know, they have 24 cameras and a ball, and in essence, they're sending that video up to the post-processing cloud proprietary solution that Otoy has so that you can get a point cloud out of that. But I'm just curious if your system that you're developing would sort of fit into that similar pipeline where you would be able to create these volumetric videos by working in the game engines and adding other layers of interactivity. getting that data from Otoy's pipeline that they announced at F8 here at Facebook within the last week or so?
[00:26:07.418] Paul Raphael: Yeah, so we're talking to most of the platforms that are out there, whether it's the camera makers or the cloud processing, or a lot of them are doing both, and we also consult with a lot of them, and we're developing our own stuff, and oftentimes we're also looking at ways to combine and to build bridges between all these different companies and technologies as with ours as well. So right now it's an interesting and challenging time because there's no standard. Everyone's trying something in their own way and there's pros and cons to every approach. I think the biggest challenge right now is these things are so new and heavy that even if you make it, you don't know yet who or how anyone's going to be able to see it. And that becomes another challenge. We already have challenges in terms of how many people have headsets at home, and then you have events like here at Tribeca, or theaters doing these things. So there's gonna be always a place to show these bleeding-edge pieces that use light fields but need to be connected to a server to run, but that's all gonna trickle down and eventually streamable on a Rift or a Vive or Gear VR or Daydream. And this is all going to kind of facilitate with time. So as a content creator, we're exploring everything all the time. And again, it's just exciting to see these tools pop up and everything is an opportunity.
[00:27:32.714] Kent Bye: So you mentioned the lack of open standards, and I think another realm where there's not any standards is audio, in terms of both capturing and distributing audio, especially if it's reactive. I know that there's ambisonic audio, multiple layers of ambisonic audio. AUSIC has a system where, you know, they may be wanting to have additional metadata that's coming, maybe something from a game engine. There's Dolby Atmos that has their own system for produced audio. But again, there's a proprietary solution, but not an open source solution to handle surround audio. So I'm curious your thoughts of whether or not there is going to be an emerging standard when it comes to audio, whether we're going to be moving into object-oriented audio coming from a game engine, or if there's going to be a way to be able to add positional information or higher levels of ambisonics to be able to deliver the type of immersive surround sound that you want to be able to do with your narrative experiences.
[00:28:23.772] Paul Raphael: Sure. So, I mean, I think it's a similar story here. There's a lot of tools, a lot of... There are standards in the making. Facebook is developing their own standards. We, again, have been consulting with a lot of these people. We also have a sound studio called Headspace, which has worked on all the content that we've created, but it also works with other studios. And just like everything that we do, Although we do develop technology, our real focus, what we really are all about is the content and the creative and the technology is there to serve that. So it's really about being kind of a compass, keeping that as our compass and trying to make that compass available to as many people as possible so that there is a convergence. Now there will always be multiple platforms and multiple standards and sometimes there will be open standards and some things will merge into standards. It's just the way it is. You never know where it's going and how long it's going to take. As long as it's workable, it's okay to have many standards. You could have Dolby on one end or I don't know what the other things are. It all ultimately can play on the same... As long as it's accessible, it's okay. You obviously want to consolidate as much as you can, but sometimes also that diversity is good too because different ideas pop up in different places and it keeps everyone on their toes too.
[00:29:46.378] Kent Bye: And so what do you personally want to experience in VR then?
[00:29:50.700] Paul Raphael: You know, I often say this, but my horizon is always not that far. I don't like to look too far ahead because things are evolving so quickly at every level, technologically, but also creatively, that I'm just excited about the next step. Of course, you know, as a company, you have to look a little further than that. But it becomes very speculative very quickly. So right now, what I'm looking at is, you know, coming off of me, you be and the people's house is continue. I've always had a very iterative approach to building on what I know and taking calculated risks moving forward. Some risks are bigger than others. Going from a three minute fiction experience based on existing IP to a 40 minute original was a little larger leap than we usually take, but again, it felt like we had done enough, you know, about 20 experiences by then to feel confident in taking that leap. And so we're looking at similar leaps moving forward, you know, more ambitious fiction narrative, more ambitious documentaries, starting to explore everything we've talked about, the positional tracking, the interactivity, always putting the creative first, you know, I've never thought of You may or may not be surprised, but a lot of people out there are just like, hey, if it's not fully interactive and positional, it's not quote unquote real VR. That's just so, so insanely semantic to the point of being meaningless to me. If it's a meaningful experience that you've never had before, that's all that matters. And if we can keep pushing that, then I'm happy. And that's all I'm looking forward to.
[00:31:31.317] Kent Bye: Great. And finally, what do you think is kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:31:39.370] Paul Raphael: Well, that's a lot further than my very short horizon, but I think that we are ultimately looking at the convergence of AR and VR that will really lead to mass adoption, you know, in a very light transparent form factor. So, you know, once that happens, that really becomes the next computing platform, you know, that we're no longer relying on flat screens as a highly abstracted form of digital interaction, then this is where the boundary, things become kind of limitless. Right now, you can't really think that way because there are too many different pieces that need to evolve separately and organically and who knows what shape they're going to take to take that big a leap. But once that happens, you know, we really reach a point where really anything is possible and it's just exciting that this is happening now and that we're probably going to see that happen within the next couple of years, you know, whether it's the next 3, 5, 10 or 20 or 50, you know, it's happening. You know, it's almost inevitable and, you know, some people doubt AR or VR's potential, although you can't predict how long it's going to take. I just cannot imagine a midterm future where we're just still looking at our 69 screens in our pockets or on the walls. It's inconceivable to me. A lot of people use 3D as an example of something that came and went and came and went and came and went. The fact is that 3D was never a full-on new medium. It was something that was slapped on to an existing medium and in many ways Even though it had the potential to be something entirely new and in fact that's what Felix and I did before we got into VR. I was trying to figure out what the full potential of 3D could be and then we kind of jumped ship into VR because it really just seemed to be what 3D wants to be. It's volume not confined to a rectangular box. It's all around you and in fact you become a part of it. So once you're there, once this medium is just a simulation of our reality, there's no question that that is the future of digital media. And it's just a matter of time.
[00:33:52.017] Kent Bye: Awesome. Thank you so much. Thank you. So that was Paul Raphael. He's a co-founder of Felix and Paul Studios, showing The People's House at Tribeca Film Festival. So I have a number of different takeaways about this interview is that, first of all, just huge respect for Felix and Paul to be able to even pull off this type of production. I mean, I think I had critiques in terms of the stiltedness and the context switches between when Barack Obama is speaking kind of off the cuff and more spontaneous in reaction to the questions that were being answered versus switching into a more reading of a script. I think if they had more time and access with Barack Obama, then they would be able to perhaps do a lot more in-depth documentary shooting in order to capture all those emotional stories. But even if they did, they still didn't know at that point which rooms that they're going to be able to cover. So they just had 15 minutes with Barack Obama and Michelle Obama. And then from there, shot all these different rooms, probably about 20 different rooms of the White House. Then construct a narrative and an arc to be able to tie everything together. And so they didn't want to make it just a didactic educational video, but they really wanted to get to that deeper emotion. And so this genre of a guided tour within virtual reality is just something that makes complete sense because you're able to give you a sense that you're actually there, but it adds so much more when you have somebody share their personal narrative, their story, or just like a docent at a museum, give you a larger narrative of the meaning behind what you're actually looking at. And so I expect to see a whole lot more of these types of like annotations of your personal memoir or story or memories on top of physical locations, especially when it comes to augmented reality. I think a huge inspiration for this type of genre is going to be the guided tour that you go on at a museum But if you're able to basically use the entire world as an annotation device, then you're able to connect places with people and their stories. And I think that's a huge application of these immersive technologies. Super interesting to hear that part of the reason why the White House wanted to do this 360 video experiment was because the platforms like Facebook and YouTube were starting to do things with 360 videos so that you could use a mouse and still see it on a 2D screen and have the experience. And it's not as good as doing it VR, but It just made some of these videos a lot more accessible than requiring something to have a headset. Also, it was really striking to me to hear a lot more about Felix and Paul's approach towards creating because so much of their creative process is in the process of actually pushing and evolving the technology forward. They're not using anybody else's cameras. They keep iterating on their own high-end camera to the point where I think some of their stereoscopic effects in their camera has been some of the best stereoscopic 360 video content that I've seen. And it sounds like they are doing this push into volumetric and digital light field. And that is just a whole other ballgame. I've since done an interview with Lytro at SIGGRAPH. And whenever you start to work with digital light fields, it becomes like a full-on visual effects shoot. That's what Paul said. It's like a VFX shoot. where you have to have green screen and just a lot of compositing. And just a lot of the post-production overhead of that just skyrockets. And so a lot of the content that Felix and Paul have been doing have been starting with documentaries and starting to move into narrative. And so moving into this green screen world, I think, is where you have to go when it comes to shooting this type of digital light field content. But there does still seem to be a trend towards volumetric video and digital light field capture, which means that right now most of the 360 video is 3DOF. And so when it comes to the sense of embodied presence, you don't feel like your body is fully completely there when you don't have those effects of parallax and being able to kind of move your body around. and actually kind of walk around a scene a little bit. And my experience is that there is a little bit more of a sense of embodied presence when you're able to actually either have a combination of photogrammetry with billboarded 360 video or something that's a little bit more like a digital light field or hype VR, which is another approach using LIDAR. And again, it's just a matter of cost and viability as to how much we're going to start to see a lot more of this type of volumetric video. So that's all that I have for today. I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member to the Patreon. This is a listener supported podcast, so I rely upon your gracious donations to be able to continue to bring you this type of coverage. So you can become a member today at patreon.com slash Voices of VR. Thanks for listening.
