KonceptVR has produced a couple of 360 videos for Broadway Theater productions that includes a promo with the cast of Hamilton as well as a music video with the cast of School of Rock. I had a chance to catch up with KonceptVR’s Max Bolotov at SVVR 2016 where he shared a lot of his lessons learned from working on broadway, some history of 360 video rigs with Freedom 360, and one of the very first 360 music videos produced by yellowBird in 2010 that inspired him to get into spherical video.
LISTEN TO THE VOICES OF VR PODCAST
Here’s the School of Rock video:
Here’s the Hamilton promo video:
And here’s the rocket launch video that KonceptVR was showing at SVVR 2016.
And here’s an unwrapped version of one Professor Green’s “Coming to Get Me”, which was of the first 360 music videos
Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Support Voices of VR
- Subscribe on iTunes
- Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to The Voices of VR Podcast. On today's episode, I feature Max Bolotov of Concept VR, and they're a 360 degree video production studio that's been working with a number of different Broadway shows. So one of the experiences that they just released a few days ago was with the cast of Hamilton, which is promoting the Tony Awards, which is happening this weekend. So since the Tony Awards were happening, I figured it might be a good chance to air this interview talking about what Broadway can teach virtual reality video production. And so Concept VR worked with the School of Rock in order to produce this really elaborately choreographed 360 degree music video. And so I'll be talking to Max today about some of the lessons learned, what was it like to collaborate with a Broadway musical show. So that's what we'll be covering on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. Today's episode is brought to you by Unity. Unity has created a tool set that has essentially democratized game development and created this lingua franca of immersive technologies, allowing you to write it once in Unity and be able to deploy it to any of the virtual or augmented reality platforms. Over 90% of the virtual reality applications use Unity. So to learn more information, check out Unity at Unity3D.com. So this interview happened at the Silicon Valley Virtual Reality Conference at the end of April in 2016 at the San Jose Convention Center. So with that, let's go ahead and dive right in.
[00:01:44.045] Max Bolotov: So my name is Max Bolotov. I am the account director at Concept VR. And the way I got into 360 video and the VR field is by doing spherical photography, and that wasn't enough. We really wanted to break boundaries and do more and introduce movement and action into our projects. And then me and my partner Alex, at the time, we met Jorgen Geertz, the founder of Freedom360. And they have been manufacturing mounts, GoPro-based mounts, for 360 videos since 2012, predating the Oculus. Everybody was based in New York, and basically we went out for a beer or two, and then one thing led to another, and then we joined forces. in pursuing Concept VR as the official production branch of our venture, and Freedom 360 being the manufacturing and educational branch. And that happened last July. We've had some great projects that we've done together. Our team grew from just four members to over 20 members. Yeah, that's where we are right now. We're a full-scale production company in the virtual reality space right now.
[00:02:55.743] Kent Bye: Great. So you're showing a couple of experiences here at SVVR Conference and Expo, and one of the ones that I'll talk about first is this School of Rock. And so, to me, it's interesting that doing 360 video experiences, I kind of find that people from a theater background actually kind of understand how to work with 360 video, perhaps even more than film, because in film, you have cuts and edits, and in theater, you don't necessarily have that. Talk a bit about, you know, what was it like to work with this Broadway production to produce this video?
[00:03:26.307] Max Bolotov: Absolutely. It was, personally, it was one of my favorite productions that we've been a part of. We got called in by Steam Motion and Sound, who were doing the majority of production on site for Angela Weber's company, and we were tasked to figure out a proper mounting weight for our camera, which camera to use, test the stitching lines, make sure that we can deliver the absolute perfect product that Andrew Lloyd Webber is known for. And working with Andrew and the director and their team was a fantastic experience where we've learned from each other and we commented on each other's ways of directing. And, you know, our DP would say, hey, this is not where the talent should be because this is right in the middle of the scene. Let's move them two inches this way or one foot this way, or let's place the camera in a different orientation. But my favorite part about this production was that the camera was the main focus. And in VR, that was the first time that that kind of happened in theater production. And we worked with the live cast and crew from the Broadway. So it was a pleasure working with them. All the actors were talented. And like you said earlier, they are known how to operate in that sphere to kind of perform and make sure that from anywhere they're on a stage, they're being seen. So they were very comfortable to this and they took one day of rehearsal and one day of 10 to 12 takes, full takes. It was a five minute piece and it was all one take. So, you know, the kids and the actors did a great job actually executing and going strong for a whole five minutes. So yeah, that's pretty much one of my favorite experiences that we've done up to date.
[00:05:01.543] Kent Bye: Yeah, I think it's kind of striking to see how there's kind of action happening all around you. And also that it seems like instead of being a director, you're actually kind of more of a choreographer. So maybe talk about that, you know, moving from directing into more choreography of these scenes, right? In this particular case, it's like you said, it's one take, it's five minutes. And so it is kind of like a little mini theater. But talk a bit about that process of like directing slash choreographing.
[00:05:27.514] Max Bolotov: Yeah, so for this particular project, we had a live preview for the directors and Andrew next door. So what we've done is we wired our cameras to basically have HDMIs out into a live stitching box where we'd stitch an entire panorama. in real time as the action is happening next door, so the director can see it on a big screen as a equirectangular. And what I mean by that is, you don't see it as a sphere, but rather an unwrapped sphere, where you can see all of the content in 360. It's a 2 to 1 ratio, aspect ratio, a picture that you can see exactly what's happening. So that was one of the key things for the director to see what's happening in real time. And we actually had a DP, a director, assistant director, and a choreographer from the Broadway being on site, constantly moving and adapting to this dance, as I call it. And I think choreography is much different. And it took their choreographer about a day or two to get used to how the kids and the actors need to move in a sphere in order for it not to be disruptive or anything like that.
[00:06:31.598] Kent Bye: Watching a film on equirectangular, it takes a little bit of practice in terms of being able to translate something that's essentially a 3D scene into a 2D representation of that. And so why not do that in VR? Is it because they want to be able to see the whole scene? Or what are the kind of advantages or disadvantages as to why you wanted to kind of project this back into 2D?
[00:06:52.159] Max Bolotov: Yeah, on that note, it's like watching The Matrix. The longer you do it, the you get used to this format. But it's very easy to miss things if you're in VR. And let's say if we had a DK2 or the Crescent Bay or the Vive attached to a live stitching box, which we can in real time. And somebody can observe by just tilting and moving their head left and right what's going on in that sector. But if you're trying to direct the entire piece, you want to see the entire thing as it's happening. So that way you could see, you know, on the left segment, something's off. The right is perfect. And you can comment on that and make adjustments as you go. And that's the only true way of directing in VR is through an equirectangular format to see an entire stitch all together in real time.
[00:07:35.582] Kent Bye: And so how did they divide and conquer their different roles and responsibilities between all these different cinematographers, the directors, and choreographers?
[00:07:43.313] Max Bolotov: Like, who did what? It was fun to talk to the director and say, hey, we kind of have authority a little bit here. you know, he had some thoughts about that and it was like, it took him a while to get used to it. But as we were showing him rough stitches and things that we were coming up on a spot and saying, Hey, this is why we think that we shouldn't do this. He was more prone to understand of, you know, okay, I need to adjust, I need to evaluate and just listen to these guys. And these new roles came about like, you know, you know, RGP is really a VR specialist that knows everything there is to know about production and post-production. Because that's really what it is. Our concept VR always gets approached to, okay, what if we buy the cameras from Freedom 360 and, you know, send it to Antarctica and have somebody film it? Can we bring back the footage for you guys? for you to stitch it and most of the time we say no because those two processes go hand-in-hand. We pretty much don't want to stitch anything we didn't shoot and vice versa because you need to know certain things when you're shooting so the post-production does not become nightmare and that's just a little to comment on that and that's how it works.
[00:08:52.815] Kent Bye: And so is that because with the kind of constraints of these GoPro cameras is that there's going to be stitching lines and that you have to work around that? Or maybe give some examples of what you mean exactly with the director may come to you and want to do something, but then you have to push back with your authority of VR production where what he may want to do may not work for whatever reason.
[00:09:11.250] Max Bolotov: Well, we're dealing with a full sphere, right? And how do we recreate that? So to take a few steps back, the only way to do that is to capture the environment from all angles at the same time. So we need obviously video cameras. And then take a step back even more and think about panoramic photography. How were we able to achieve that format and its own was to shoot from the same entrance pupil. So each camera has to be technically shooting from the same entrance pupil to achieve no parallax point. And that's how we were able to do it in panoramic photography because we were literally shooting and turning the camera, shooting and turning the camera. Here we needed to get any camera on the market as close to the center as possible. Hence why we chose GoPros, because they're smallest camera in the market that are able to achieve over 4K content per camera, and they give you the best results possible right now for less post time. But then you have requests like, you know, I want to shoot this on Reds, or I want to shoot this on Sony A7Rs, and most of the time, figure out a way to do that. But with that comes the responsibility of knowing where the limitations of those cameras are. So if, for instance, if the cameras are too far apart, we know to position the seam in a way that an actor or the main focus is not going to be in between two cameras. So it's a lot of play on site and then post-production.
[00:10:33.832] Kent Bye: And so I kind of see the GoPros as a little bit of a stopgap right now in a 360 video production and that eventually things seem to perhaps a lot of problems may be solved by going to digital light fields and other kind of advanced technologies. And so from your perspective as a video content producer, being involved in the production of these rigs to shoot these GoPros, what are you doing to look into the future for what's next beyond these GoPros that may have been created for other use cases?
[00:11:00.495] Max Bolotov: You're absolutely right. Right now, the industry is struggling to come up with a one-button solution, and that's not possible at this moment for obvious limitations of technology and screens and just in general of capture. But you did mention light field cameras, and we were at an A.B. show last week, and Lightro did a fantastic demo on their new technology that they're pushing for their cinematic 2D camera, but they already have a visual rendering of what their spherical camera would look like. And that's really the future of VR and 3D and stereoscopic, where you can have a few degrees of movement inside the experience and you can actually calculate for proximity. You know, if this pole is six meters away, I'll be able to calculate that and relay that to my pupil. And that's the future. That's what gets me exciting. But right now, GoPros are our workhorses for a lot of the productions that we're doing just because of the form factor. They're small. We can put them in compact spaces. Recently, we did a project where we filmed inside dune buggies or inside an infinity car or inside of Jay Leno's legendary cars. And those camera placements had to be very delicate. And on top of that, they had to be small and not noticeable, because you are shooting 360. So the mounting systems are going to be in there. And what we end up doing is rotoscoping a lot of it out. But going back to your original point, form factor is a very important player in the game right now. That's why we use GoPros for a lot of our productions.
[00:12:29.948] Kent Bye: And so you had an opportunity to shoot a rocket launch. And so maybe you could talk a bit about the experience that you were showing at NAB as well as here at SAVR Conference and Expo.
[00:12:39.936] Max Bolotov: Yeah, so our newest feature presentation is a rocket launching 360 that was done in Vandensburg, California Air Base. And we were contracted by the United Launch Alliance to capture from perspectives that no human has ever seen before. So we were given privilege access to actually be on the launch pad a little bit prior before the rocket actually launches. And we climbed one of the towers, and it was pretty spectacular of what they allowed us to do. But as you've seen, the footage is pretty fantastic. That was a push in the right direction for commercial brands to really utilize this medium in a very meaningful manner. to say, hey, why am I doing this in VR? Why am I making a 360 video? Is it just because of the gimmick, or I'm really trying to show something that complements virtual reality in 360 video? And this experience was the perfect example of that, because I think every person is a science geek in their heart a little bit, because everybody wants to be an astronaut at some point or another when they're young. And to see that and kind of look around and see, oh my God, I've never seen this, I've never seen that. You're really enjoying the immersive aspect of it, of looking in every direction and immersing yourself in that atmosphere. An additional thing that we've done for this experience is we introduced a little bit of haptic feedback. When you're ready to launch into this experience, we put this personal subwoofer by a company called Subpack on you and that attaches to your headphones and to the Gear VR that we use for showcasing this. And the way it works is as the frequencies get lower, the pack starts rumbling and shaking your entire body, which adds a whole new level of immersion and adds to presence. So that's where we're showcasing a NAB and that's where we're showcasing a SVR and I think people are really enjoying it.
[00:14:28.675] Kent Bye: Yeah, I think that the visceral experience of having that low-frequency feedback, especially at a rocket launch, I think it definitely adds to the level of immersion. You know, one of the things that I see a challenge with in storytelling within VR is that I think there's a lot of constructs within 2D film that are being ported over. For example, kind of like the voice-over narration, which in my mind, I feel like doesn't really necessarily work as well in a VR experience, you know, something more like the School of Rock where you're showing instead of telling. Anytime there's a disconnect between what is happening and what I'm listening to, what I'm seeing, I don't know, what are some of your thoughts in terms of how to actually tell a story within VR?
[00:15:07.330] Max Bolotov: Storytelling in VR has been the topic, I think the most famous topic among the production companies and you know the leaders in this industry and I feel like it's still developing and it's exciting because we are shaping this industry right now. I don't think there's a particular way to do it. Every project we approach there's a different thing that we have to invent. or construct or outsource. Every single project is different. Every single production is unique. Different movements, different choreography, like we talked about earlier. And I think storytelling is conventionally different because you have to reimagine the 16 by 9 ratio rectangle that you used to and break yourself out of it. You have to completely, and I tell this to all of our directors who would like to work with us, completely forget everything you know about film and start from scratch. Let me show you an experience. really think about what you just saw, and how would you do this differently, and why would you do it differently? And I think that's where it all starts. I think it's a personal journey for every single filmmaker, and it's necessary right now.
[00:16:09.914] Kent Bye: Well, what are some of the big takeaways that you've had so far? Things that you know seem to work really well, and things that just are terrible.
[00:16:16.237] Max Bolotov: Well, close-ups. People think that we can do close-ups with VR, but it's impossible, and they request it all the time. Like, how do we focus on this one topic? And this is more of a request that we get. And we're saying, hey, listen, you're immersed. You have to forget about that. You have to understand that you're giving your viewer the freedom to look anywhere. And that's your entire palette. You have to be able to paint in 360. But the thing that we know that doesn't work right now in production is first person. So there's a big difference between saying VR is first person because you're looking around, but some companies actually approach us and say, hey, we want a first person experience in the sense of we want to put a camera where somebody's head is or on the motorcycle helmet or something like that. And that just does not work because the micro-movements of your head, let's say a helmet mount, right? I've seen hundreds of experiences where it's just, it's failing. And it actually sets the industry back a little bit because if a person saw it for the first time and that's their first experience, they're going to have a bad experience. It might even cause nausea. And that's really not what we're trying to do here. So first-person VR experiences can be mimicked, but never, I'm going to put something on the body. Because as you move, and as our neck moves, or as our upper body moves, those micro-vibrations are very hard to stabilize. And in VR, any vibration can cause sickness. You know, just put it that way.
[00:17:45.441] Kent Bye: Well, what's one of your favorite memories of being in VR then?
[00:17:49.292] Max Bolotov: I'm going to reference a video that not a lot of people know. It was actually done by a Dutch company, Yellowbird, and it was a video done for Doritos. It was an ad for Doritos, and it was exclusively... Back in the day, we did not even have Oculus. This was before Oculus went mainstream. This is when we were shooting 360 video with just camera. It was just 360 video. There was no way to split the screen into two eyes and get you in the VR. So you can just browse it on a custom player that you had to design for that brand. You just browse around. It wasn't fully spherical either because you had a big nadir hole. That's the technical term for like the blind spot because there was not a camera. out there that did fully spherical 360 video, which on a side note, Freedom 360 was the first company to create world's first, I'm gonna use that line, fully spherical camera rig in 2012. So after that, people started using that technique, using their gear, and so on and so forth, and that became a real thing. But going back to my favorite experience, that experience actually jump-started my whole love of 360 video because it was a music video. I forget the name. I think Dr. Green was the artist. It was exactly like School of Rock in the way that people knew that they were shooting for 360. And it was shot in a way where anywhere you look, it was completely mind-blowing. It made sense. Everybody enjoyed it. It played with the music. It played with the audience. And that's my earliest experience and one of the most memorable ones. One of my favorites from this time is the rocket lunch, 100%. You can probably agree with me on that.
[00:19:25.907] Kent Bye: Yeah, it's pretty spectacular. And finally, what do you see as kind of the ultimate potential of virtual reality and what it might be able to enable?
[00:19:34.029] Max Bolotov: From the beginning, I see that virtual reality is going to be a big industry for education. I think there's a lot of uses to cure phobias and just put people somewhere where maybe they're uncomfortable or never can get to. and third world countries and so on and so forth, and really immerse them and give them a chance to experience something that they wouldn't be able to. Also, I'm working on something that I will call, it's a little blog, but long story short is, I call this First Steps into Teleportation. And I feel like we're moving forward much quicker than I think a lot of people anticipated, especially with VR. with big giants getting behind it like Facebook and Samsung. So I think, yeah, teleportation is gonna be here pretty soon and I'm excited.
[00:20:23.518] Kent Bye: Anything else left unsaid that you'd like to say?
[00:20:25.920] Max Bolotov: No, it was great talking to you. I hope you enjoyed our VR experiences. We have a lot of them on our website, on our social medias. If you guys have a smartphone, have a Google Cardboard, feel free to check them out. Put in your Google Cardboard and get immersed. Or just visit us at a couple of trade shows. We do a lot of them. So just stop by and say hi.
[00:20:47.731] Kent Bye: Awesome. Well, thank you so much. Thanks a lot, Ken. So that was Max Bolotov. He is an account director at Concept VR, which is a 360-degree video production studio working with Freedom 360. So there's a couple of quick takeaways that I have from this interview is that, first of all, I think that Broadway and theater productions in general, as a communications medium, is probably a lot closer to the strengths of virtual reality than compared to, say, film, where the language of film is a lot different than the language of VR. It's much closer to theater. It's interesting to see how what Max said is that this School of Rock production is really the first time where a theater production was really performing to a singular point, which is the 360 degree camera. Usually they're on stage with all the blocking and performances directed out to the audience. But in this case, you know, they're starting to move towards doing these theater productions towards the 360 degree camera. I'd expect that there's going to be a lot more productions like that and it just seems like it's a natural fit and a lot of these productions just seem super polished because these theater actors are very used to doing long takes because that's essentially what they do all the time when they're giving a performance. And this is actually the first time that I've heard of somebody using a live preview, which I think actually makes a lot of sense for the director to be able to be previewing what's happening in the entire scene and be able to look into the matrix where you're able to just do this translation of looking at a full spherical 2D equirectangular perspective so that you can actually see all that's happening at the same time. Also, I've never heard of this 360 degree video. It was by Professor Green. It was called Coming to Get Me, and it was by Yellowbird. And it was released in 2010 by Doritos UK. It's no longer on YouTube, so I can't see the version where it's actually click and drag around, but some people have copied over the version, and you can see an unmapped version of this video. It's actually pretty quite sophisticated for being one of the first 360 degree music videos that ever really been produced and launched as a part of a campaign back in 2010. So it was kind of interesting to get a bit of that history for people thinking about some of this 360 degree video stuff way before the advent of the Oculus Rift and this modern renaissance of virtual reality. But also to get a little bit more of the story of Freedom 360, which I think has really been a huge part of bootstrapping this whole 360 degree video revolution because, you know, they've been able to provide the 3D printed hardware in order to create the holder for these GoPro videos in order to even shoot the 360 video. So Freedom 360 has been playing a pretty vital part in the history of cinematic VR and all these 360 videos that are already out there. And finally, I think that using something like a sub-pack and really designing an experience that uses these low frequencies, I can't emphasize enough how important it is to add this extra layer of haptic feedback, because it just adds to the layer of embodied presence so much when you have that visceral feeling of actually feeling you're there and feeling the rocket thrust really just vibrate through your entire body. check out subpac it's a really cool and especially if you're doing installations you know i can't expect that everybody's going to have a subpac when you're creating an experience but if you're showing an experience in a trade show or with people that are coming into like an installation context then It doesn't hurt to add something like the SubPAC, which just takes the level of presence up to that next level. It's actually one of my favorite VR peripherals, because it's pretty dead simple. You just plug it in, and as long as there's low frequencies there, it just works. You don't have to integrate with any type of SDK or anything. So definitely check out SubPAC. So that's all that I have for today. If you are interested in meeting up next week in virtual reality, I'm going to start to try to have more virtual meetups with listeners of the Voices of VR podcast so that we can just have a place to meet in different social VR spaces and just talk and connect with each other. So if you're interested, go to my website, go to an individual blog post. There'll be a email that you can sign up for. Sign up for the email and I'll be sending out more information about different virtual events and gatherings. So with that, if you do enjoy the podcast, then please tell your friends and spread the word and consider becoming a contributor to this podcast at patreon.com slash Voices of VR.
