Nick “RedOfPaw” Pittom talks about the process of translating scenes from Studio Ghibli Films into VR experiences like The Boiler Room scene from Spirited Away and The Bus Stop Scene from My Neighbor Totoro
He makes the observation that translating a scene into a VR doesn’t mean that it’s the best way to experience the story or the characters from a film experience, and that really the best VR experiences will be custom made to take into account the pacing and environmental strengths of VR.
Nick also talks a little bit about his new Crystal Rift experience that he’s helping out with, as well as his successful Kickstarted trip to Oculus Connect and the current plan for his “RedOfPaw’s Big Crazy Stupid VR Adventure and American Pie.”
Theme music: “Fatality” by Tigoolio
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:11.975] Nick Pittom: I'm Nick Pittam. Many people know me as RedofPaw, which was my Reddit username, which then kind of expanded and became my vague VR identity. But I made originally two short demos, one based on Spirited Away, which is a Studio Ghibli film. And I think most people recognize it as one of the best animated films ever made. And the reason I did that is because I wanted to make something in VR that could look amazing, be different, be something people could step into, but also potentially would run really nicely, you know, performance-wise. And I also needed something that had great art design, because I wanted to try out some ideas, make some demos. And one thing I didn't want to spend a lot of time on is making sure my own designs were good enough, because there's so many challenges for VR. But once that was done, that was really well received, and I moved straight on to do one for my neighbor Totoro. The bus stop scene, for people that know the scene, they seem to really love it. They genuinely, like, love, love, love, love it. Because you can stand there next to Totoro. He's this big raccoon cat thing. And I think people seeing that in the flesh and being, well, in VR and being so close to him, I think, you know, if you know the film, that's something that's really special. I think people get a kick out of that, which is nice. People that don't know it, a little bit weird. There's this big creature thing, some animal bus turned up, a little bit strange. But, you know, it's good. But now what I'm doing is I've done a Kickstarter, and the Kickstarter was actually to raise money to come here to Kinect, because I didn't have the money to do it. I thought, well, I needed some cash to go. What can I do? And I didn't want to beg for money, though. So I said to people, look, if you get me money to come out here, I'll also make a game, like an adventure game based on the experience, somehow tangentially related to the experience. But I also got people to give me their ideas of what they wanted in the game. So I'll be adding all this together into some kind of adventure experience. It'll hopefully still be a good game that people might want to play even if they know nothing about how it started. And then I've also started working on a game called Crystal Rift, which is a dungeon crawler, a bit like Legend of Grimrock in terms of the movement, like a grid-based movement system, which actually works really nice for VR. because it's like being in comfort mode the whole time, you switch and you move in spaces that are very comfortable as a VR experience. But it's also got a really nice atmosphere and John, who started making it, did a really spectacular, I guess, game engine within Unity, the way he set it up. It's really efficient and it'll work really nicely on, say, Gear VR. Things like that. So I'm really looking forward to getting stuck into that. I'm really looking forward to yeah all the genuine possibilities going forward I've got my own ideas of other projects I want to do as well But when you got so much in your plate, you just have to I know do one at a time And so maybe you can tell me a bit about your background in terms of you know Had you done any 3d modeling before or was the first sort of studio Ghibli?
[00:03:02.158] Kent Bye: characters some of the first models that you were doing and as a way to kind of learn and
[00:03:07.188] Nick Pittom: So my background is from film and animation and that's what I still do as a day job. I run my own company doing animation and stuff for more marketing and things, but also a few things like the VFX job for an independent film or something like that. It's quite a wide range of things. I'd actually made a short film. back in 2012 and I spent a couple years on it and I was really happy with it. But it just didn't quite go as far as I'd hoped it would. It wasn't as successful as I'd hoped. I realised it's because film is so over-subscribed. There's so many people wanting to work in film and people that already do and there's no way for the industry to really expand in any way to kind of let all those new people who want to be involved in. And I kind of realized what I'm really interested in is storytelling. It's not necessarily film, storytelling. And I thought maybe VR is a really, I mean, I saw the Oculus Rift, and I was like, wow, that's amazing. And I got one anyway, because I was interested. And then I saw a couple of demos, one especially for Blocked In, which is this little experience where there's Tetris blocks, and you're in a little diorama. And I saw that, and I thought, actually, the sort of things I'd done before with 3D modeling, the skills work completely within that and I could do that and I thought that seems exciting because I could then instead of having to go through filmmaking which is expensive and time consuming you need lots of people to be involved I thought I'll go and make VR experiences because it's such a new field open wide so many interesting possibilities seemed like a good one and then yeah I spent the main thing I had to do though was learn how to do things properly, how to model correctly and how to texture correctly to keep things running efficiently but also just being any good. I've still got quite a lot to learn but hopefully it's good enough at the moment.
[00:04:52.037] Kent Bye: Yeah, and I think virtual reality is a new enough medium that it's still a little bit of the Wild West where people can experiment and do a lot of things with it. One thing that I wonder is in terms of the copyright issues and, you know, is this something that you would argue is a fair use of taking this and sort of repurposing it into another medium or was Studio Ghibli in contact with you at all in terms of like, hey, you know, like you didn't ask for permission to do this or how did that sort of play out?
[00:05:18.347] Nick Pittom: So originally I didn't care. I just thought I'll just do it because it's not gonna be like a big enough deal that I think I'll get shut down. I thought even if I got shut down at least I've shown I can do stuff. I could show screenshots of it or something like that and people might be interested. In the long run though I think because they are such small experiences they're not an expansive thing. I'm not pretending they're more than they are and I'm not making money from it. I think it's flown under the radar enough that people don't care. And that's great, but in the long term, I've spoken to Oculus and I've spoken to a couple of other people, and taking someone's IP that's not yours and making something, that's fine, and they seem genuinely cool, but to actually then progress further, to get support, to make bigger things, unless you get that, you can't really do anything. It seemed a bit pointless keeping to putting all my energies into stuff I can't directly benefit from. Not like money's important in and of itself, but I'd like to be able to do VR full-time. Which means I have to generate money somehow and if I'm spending all my time on things that can't in any way Or and I can't get support from people of any kind it just seemed a road that I didn't need to travel down But hopefully in the future. I'd love to do an adaptation or work on adaptations of things like animes or whatever so there's potentially an Avenue to go down, but currently there's not any opportunities for that and
[00:06:38.946] Kent Bye: Yeah, I know in John Carmack's recent talk at a college in Texas that he had given at SMU, he was talking about his process of learning where he would clone a lot of things and just copy it. And I think that's an important part of this learning in a new medium is to take something that you know works in a previous medium and just sort of clone it and reproduce it. So it seems like in some ways that's your process and your path to be able to do that.
[00:07:03.583] Nick Pittom: Yeah, absolutely, because I wanted to go, there's so many problems to solve with VR, especially at the time when there's, when even everyone who's professionally involved in VR don't really know the answers. Trying to figure that out, as well as trying to do those other creative problems of good design, good ideas, it just makes sense to just sidestep problems you don't need to solve. And of course, you know, even on the cynical side, taking something that's already popular, you know, people are gonna wanna check it out. And that's good, you're riding on some coattails a little bit, so there's definitely benefits from different angles on that.
[00:07:37.165] Kent Bye: So what were some of the lessons that you got from your first boiler room experience that then you took into the My Neighbor Totoro?
[00:07:44.292] Nick Pittom: So for each of the demos I've done, on the Ghibli stuff at least, I learned specifically, even though something might be awesome as a film, it doesn't necessarily make it a good VR experience. And even though Totoro, it's good, it's just not the best VR experience. There are plenty of other VR experiences that are in and of themselves better on what you're doing, what you're playing, what you're enjoying actively. And as a means to explore the film, it's neat. You get to look around, you get to explore it in a way. But it's not the best way to experience the story itself or the characters of that film. Watching the original is the best way to do that. So it's this weird halfway house where it's not really excelling to copy a scene from a film like that directly into VR. And much like games, converting a game, any game, like Skyrim to VR is possible and it's neat, but the best games are going to be games created specifically for VR. And I think even storytelling experiences like I'm hopefully going to be able to make. The best storytelling experiences for VR will be made for VR because you're going to take into account what works for the pacing, or you're going to take into account the fact that you're in an environment, there's going to be certain types of storytelling that work so much better. But I still do believe you can tell a good traditional story with, you know, characters and arcs and scenes and you just got to kind of try and find a way to take the lessons of film and take what's relevant and hopefully apply new lessons to make it work for VR.
[00:09:15.184] Kent Bye: And it sounds like your kick-started project of My American Pie VR experience is sort of a joke in some ways of making either the most absurd or crazy mishmash of all sorts of disparate ideas. I think there's a challenge there in terms of taking input from the audience and then taking it all into account and then transforming that into something coherent. or something that works as a whole piece. And so as you're going through this process, what is your underlying intention for what you want to experiment with or get out of or give people in terms of this experience?
[00:09:50.514] Nick Pittom: So for the Kickstarter demo, I was at the time painfully aware that I needed to come up with something within a day. And I thought, well, again, the problem is you want to put time into something creative and put the effort in to make something good. And if you can't put that time in, you don't put that time in, it's not going to be good. So I thought rather than worrying too much about a specific idea, I would literally open up the doors and say to everyone, look, just give me an idea and I'll put it into a game. And I thought that I can either make a good game that happens to involve all that, that'd be great, or I can make a stupid game that happens to involve all that. And that'd be fun on a different experience, you know, it'd just be random. But I think since then I've come up with a plot device where you've been in a horrible accident, your brain's been uploaded to a computer and it's got corrupted. and your doctor is telling you to find this particular piece of information, and if you can get to that, then they can piece together your mind again somehow. And that means you can have lots of crazy random stuff happen. It's a bit like a fever dream or something. And yeah, at the end of all that, it'll hopefully, yeah, be a good excuse to have lots of random elements. And hopefully, you know, try and make an actual enjoyable Because I'd like it to be something that people could play, even if they don't know about the Kickstarter. So I probably won't call it Redaport's Big Crazy Stupid VR Adventure. I might give it an actual name, and the Kickstarter can remain called that or something. But hopefully, yeah, it'll be a good game. I'm looking forward to actually starting on it. I've got a few ideas I'm excited to try out.
[00:11:16.118] Kent Bye: Yeah, it kind of reminds me of, like, Memento, where, you know, it's a film that's told backwards, linearly. But it also reminds me of Sightline, which is using the change blindness to be able to do cut scenes, where if you look to the left and you look back to the right, then you're in a completely new environment. And it sounds like that could be an interesting mechanism to switch between all these different scenes. Have you thought about how you're going to tie together all these sort of random parts into what the user interface is in terms of accessing these scenes and memories?
[00:11:46.632] Nick Pittom: Not specifically, no. These are still problems that I am trying to solve. But I've got some ideas. Sightline is such a great demo. and such a great, I guess, experience. Like, there's a lot of head movement. You do have to move your head around. And I think that prolonged sort of head movement works great for their game. I don't think it's the sort of thing you'd want in every game. But equally, I don't think I've got the technical capability to make something of that technical kind. They've done a really great job in kind of making it a seamless transition. It's such a good idea, though, for that to be a transition between a dream world. So, yeah. Yeah, thank you for that idea. That might be a good one to use.
[00:12:26.593] Kent Bye: Great. And maybe you could tell me a bit more about this new project that you're working on. Is it the Crystal Rift?
[00:12:33.238] Nick Pittom: So, I met John Hibbons, who actually had made Crystal Rift up to this point, at a meet-up at Bossa Studios in London. They do Surgeon Simulator. And he came up to me and said, you know, I really like your demos, you know, I like what you've done with the art style and everything like that. And he showed me his demo, and I was really impressed with literally how well it ran, how good it was, like, comfortably for VR. I mean, I don't think originally that was... He was trying to match those sort of dungeon crawlers from the old days. But it just so happens I watched really, really well in VR, and I really liked the way he added atmosphere, but there were also some kind of challenges and gameplay and stuff. But I thought what I could bring to that game is look at the way that the environment could be used for storytelling, look at narrative elements to add for that, but also the actual content and experience of the overall game that I could help bring something to. But yeah, he's done a really amazing job technically on that game. It's really nice. We've worked on a new alpha that's come out, which is updating some elements and throwing around some ideas. But we're really going to get stuck into it now, create the full experience, hopefully go green light, Because I've not done, that's the other thing, I've not done a professional game of any kind and my developing is all very much self-taught or watching tutorials. So to be able to work on something that's professionally coded and is competently made and put together, that I thought was a really good experience for me to have. There's no point in me just winging it and hoping I eventually, you know, don't code things wrong. It's nice to be working with someone who knows what they're doing. So it just seems like a nice... And also it's exciting to be involved in more than one thing. To have a few different avenues to go down so you can kind of explore what you enjoy doing most as well.
[00:14:21.030] Kent Bye: And finally, what do you see as the ultimate potential for virtual reality and where you want to kind of dive in and help push it forward?
[00:14:29.262] Nick Pittom: I think the potential is so wide open. I think there's so many experiences you could have. I love storytelling, and I think there's so many possibilities. Not just to have film-like experiences, but interactive ones as well, that are truly like, choose your own film direction, be in the film if you want. It's just one of those things. It's so wide open in that side. And then games, in a way, there's restrictions in games now that were not there before that people have to pay attention to. But these sort of restrictions open up new doors and new avenues that people wouldn't have thought about before. So, yeah, yeah. No, I think the potential is just so wide open. It's kind of difficult, maybe impossible to predict and maybe foolish to predict where it's going to end up. But I don't know. I think there's plenty to get excited about for years to come.
[00:15:20.058] Kent Bye: Is there anything else that's left unsaid that you'd like to say?
[00:15:23.580] Nick Pittom: It's just been great coming to Kinect and meeting literally and seeing all these people that you've seen before or kind of caught their work online or even your own voices of VR thing. But yeah, you know, because I listened to your podcast, I listened to Kyle's podcast, I've watched Bruce, you know, and meeting all you guys and all these developers and stuff and having developers, you know, like Eve Valkyrie, the guys from that, knew my demos. which is bizarre. It's like, it's kind of cool. It's a bit, you know, like for nerds, it's like rock and roll or something, you know, you know, and in the film industry that would be like, I don't know, some famous director going, Oh, Hey, I've, I've seen you off film or something. So it's a nice feeling to have, but it's cool. Yeah. Just an exciting thing to come around and they do a really good breakfast.
[00:16:07.623] Kent Bye: Great. Thank you.
[00:16:08.823] Nick Pittom: Brilliant. Thanks man. That was really good. I really appreciate that. And yeah, I'll, I'll look forward to listening to all of your interviews when I get back. Awesome. Thanks.
