One of the most compelling applications of VR will be the educational experiences that will be possible, and David Whelan’s Immersive VR Education has been making some big innovations in this space. They’ve created the VR documentary about the Apollo 11 moon landing, which has won awards at the Future of Storytelling, Unity’s Vision Summit, and the Festival of International Virtual & Augmented Reality Stories. They also just released the alpha version of Lecture VR, which aims to be the Powerpoint of VR complete with 4D effects, networked learning environments, and opportunities to directly apply science to problems within a virtual space. I had a chance to catch up with David at Unity’s Vision Summit where we talked about Apollo 11, Lecture VR, and the future of immersive education.
LISTEN TO THE VOICES OF VR PODCAST
Here’s the launch trailer for Lecture VR Alpha 0.1, which was released yesterday
Here’s the Tears of Joy of a father & son watching Apollo 11 that David mentions.
And here’s the Apollo 11 Trailer:
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast.
[00:00:11.887] David Whelan: My name is David Whelan, I am the CEO of Immersive Virtual Reality Education and we are the people behind Apollo 11, VR and Lecture VR. Apollo 11 is a VR documentary and it's based all around the events of the moon landing. So you go back to 1969, you become Neil Armstrong, you get to sit on top of a Saturn V rocket, go to space, land on the moon and you get to plant a flag in an hour-long journey and it's all heightened with music. the emotion is heightened with music. We're trying to bring emotion into education. And if you have an emotional response to something, you're going to remember that. So we all have watched movies, thousands and thousands of movies, but we never remember the movies that made us laugh or make us cry because we have an emotional response to it. So we find in education, if you have an emotional response, whatever that emotion is, you're going to retain whatever knowledge is coming because you had such a reaction to it. We've also built our education platform, LectureVR, so if you've ever read the book Ready Player One, it's kind of like the education system from Ready Player One, so any educator can go onto our system, upload their content, so it could be a PowerPoint presentation or whatever they like, that'll be streamed in on a screen inside the virtual space. then they choose the location, so if they're talking about marine biology, why hold it in the classroom? You can actually have the class on the seabed, for example, and you can choose from our 4D effects, you can have a whale swim through the centre of the class, and the students can interact with that. And it's a social learning environment, so it's networked, so you as an educator could be in America, your students could be from all over the world. What we do is we record the tracking data of the educator. So the educator, if they're wearing a headset and they've hand tracking, we record all that data. So literally when the class is finished, they can sit down from the point of view of the students and replay that class over and over again. And then that can be streamed again indefinitely on the web. So it's a fully immersive learning, teaching environment. And we're trying to make it really intuitive. So you can kind of think of us as a VR PowerPoint that's also networked that other people can join into.
[00:02:08.814] Kent Bye: Awesome. So yeah, let's first dive into a little bit more of the Apollo 11 and then go into the lecture VR. So I think the last time I talked to you was at SAVRCon back in 2015. And since then, and maybe even before that, you've managed to gather a lot of different awards for this Apollo 11 experience. So maybe you can kind of go through the different awards that it's already received.
[00:02:28.463] David Whelan: Yes, so there was the Firevirus award which was in Toronto. We won the Public Choice award. It's my personal favourite because it wasn't chosen by a panel of judges where the Apollo 11 demo was shown on the floor and people voted on it. And any Public Choice award I think is more valuable than maybe the judges awards. But then we did win some other judges awards. So we won the Time Warner Future of Storytelling award. in New York at a really massive event for us because we're just a small start-up. I got to meet President Al Gore at the event which was a great honour. We had the highlight of New York at this event. We've also won Best VR Film Interactive Story here at the Vision Awards which we received last night. and at the VR Fest in Las Vegas we also won Best VR Film, our Best Computer Graphic VR Film, so we're after winning quite a few awards. I think really it's the story narrative because in the Apollo 11 experience we use all the real NASA audio. So when you're in the cockpit you really hear Neil and Buzz talking and then we've heightened that down with epic music and it really gets a reaction to people because they've maybe heard this audio before but when they're sitting down inside the cockpit and they're actually looking out the window and they can see the Earth from say 800 miles up that really gets the hair standing up on the back of their necks and when you hear JFK's speech set to music as well that really gets a really good emotion so we're really delighted with the progress of Apollo 11.
[00:03:49.231] Kent Bye: Yeah, so there's this whole question of being able to add interaction and be able to tell a story as well. So at the future of storytelling, it's sort of an event that's trying to look at what's next and being able to tell stories. And it seems like this is an experience that has an emotional thread, but also seems to be resonating with audiences. So for you and your experience of actually creating one of these more interactive stories, what do you think are some of the key components that you're taking away as design principles?
[00:04:19.520] David Whelan: Okay, so the way I've designed Apollo 11 is we always start with the audio. So the best way to get an emotional response from somebody is music. Everybody loves music. They have their favorite types of music. They have music that they don't like. But when you hear a really inspiring piece of music, that will have an emotional response to you. So when I sit down when I was designing the storyboard for Apollo 11. I was just listening to music constantly. And when I heard a track, I usually get pictures in my head. I think this track could be really good for the launch sequence. Then I'll match that up with the audio, the actual launch audio from NASA. And we see, is that a good marriage? And once that's a good fit, the way I picture it is, when I close my eyes and I listen to the audio, I visualize what's happening. And the visualization of what's happening when you open your eyes inside our VR experience, that's exactly what you see around you in 360. So if you can hear the audio of Neil Armstrong walking on the moon, you can visualize in your head what this looks like. So when I've created the storyboard, I sent it off to Drash, who's the lead developer on Apollo 11. He's an absolutely fantastic developer. I can't rate him highly enough.
[00:05:19.763] Kent Bye: He also did the Titans of Space, which is kind of a favourite educational experience as well.
[00:05:24.449] David Whelan: Exactly, the Titans of Space was one of my very first VR experiences and it's great to actually get to work with the guy on Apollo 11. So, literally I sent him the storyboard and he comes back then with the visualisations of it and then we just tweak all the timings to make sure that it's all absolutely perfect. And then we've done Apollo 11 in stages, so it's kind of stage one is you're hearing the JFK speech, stage two you're on the launch pad, stage three and then we're just following the mission through, you know, so it's kind of what way can we design this where we get the most emotional impact? And then scale is obviously a massive thing, so like the best thing about VR is you can see the scale of objects. So when you're standing on the launch pad and you're looking up, that Saturn V rocket is 360 feet tall. But you're looking up, it really looks like it's 360 feet tall. Then when you're out in space, we get the Apollo craft moving past you and you can see the scale of the Apollo craft compared to the moon, which is behind as the backdrop. And you can actually see how big the moon is, you know, and you really get a sense that these guys done this almost 50 years ago. on pretty much a computer that has less calculating power than most toys. Like if you get a talking toy, for example, now in the shop, you know, something that you squeeze the belly in, I might say like 10 or 15 phrases, that probably has more power in it than the computer I had in Apollo and you're just going, my God, these guys had some balls to do this 50 years ago, you know? So that's pretty much how we designed Apollo 11. what can we get the best emotional impact out of the experience while being truthful to the story and making sure that we get all the facts right. Like even the cockpit interiors, every switch, every dial is absolutely perfect. All the tags are absolutely the way they're supposed to be. So we're trying to be as authentic as possible. And for the landing sequence, when you're in the lander, when you look out the window, we've taken all the photographs that NASA have taken of the landing site, and they've stitched it all together. And we've used that. So when you're looking out the window, it's the real moon you're looking at. You see every crater as Neil Armstrong seen it when he was landing.
[00:07:12.797] Kent Bye: Wow, that's pretty amazing. So you actually recreated all that path as well?
[00:07:16.478] David Whelan: Yeah, we recreated the path exactly as it was recorded by NASA as well to make sure it's authentic. But then we have, that's kind of the passive journey through the experience. So you could sit down through the journey and choose the passive journey where it's a 40 minute experience where you sit down, you don't have any interaction, but you're looking around and you're kind of in awe and wonder. But then you can do the interactive journey where you take control. So you control the docking sequence, you control the landing, the parallel landing, and you actually, you're Neil Armstrong on the moon and you get to walk and you deploy the experiments. So there's a little bit of interactivity with that. We feel that Apollo 11 is likely going to be a lot of people's first VR experience. Because when people buy these headsets, like the Oculus headset is $600. So a lot of parents who are in their 40s and 50s are going to buy these headsets for their teenage kids. But if you spend $700 on something, you're going to want to try it out. But a lot of these parents have never played video games before. So if you put a controller in their hands, they're going to stress out and sweat inside the Rift, whereas they're going to see Apollo 11. It's like $10. There's a journey there. It's like a VR documentary, like a VR movie. You feel like it's in a good place for them. They can click on that and they can experience it. There's actually been a video on YouTube of a father and son sharing the experience. It went viral. The father actually got a really emotional response and he teared up inside the experience. It's a great video. I think if anybody looks up at Dad's reaction to Apollo 11 VR, you'll find it on YouTube.
[00:08:36.084] Kent Bye: And you've also had some astronauts that have actually walked on the moon do it as well, right?
[00:08:39.846] David Whelan: Yeah, so we had Al Duke. He was the youngest man ever to walk on the moon. He was also on Capcom for Apollo 11. So when Neil Armstrong was talking back to Earth, he was talking to Charlie Duke. And Charlie Duke had that famous saying. So when they landed Eagle, Charlie Duke was the guy who was saying, you have a bunch of guys about to turn blue. That's Charlie Duke's voice. So when he was inside the experience, he could hear his own voice talking back to him as Neil Armstrong took his first step. And he was standing literally five feet away from him. So we did post this on YouTube as well and he had a huge smile on his face and he really felt the presence of the astronauts inside the cockpit. So as they were taking off, as he turned, we have a triggered event. So if you turn at a certain moment, the astronaut sitting next to you will give you the thumbs up as you're taking off. And Charlie Duke turned around and he gave the thumbs up back to the astronaut. It was one of the best moments I've ever seen, you know, the best reaction I've ever seen. And he absolutely loved the experience. When he came out, he said, look, this is exactly how people are going to remember history in the future. It was a great honour to meet him. He's actually one of my childhood heroes. We're also going to meet Al Wharton. He's in the Guinness Book of Records for being the loneliest person that ever lived. So he was command module pilot for, I think it's Apollo 15. He's coming to Ireland in July. So we're going to have a day with him. We're going to put him in the experience. He didn't actually get to walk on the moon, but he will in our experience. So that's going to be quite a trip for him.
[00:09:57.227] Kent Bye: Yeah, it sounds like, you know, you've done a lot of research, it sounds like, in terms of trying to recreate everything. Like, is like, for example, this thumbs up, is that something that the astronauts actually did?
[00:10:07.496] David Whelan: We don't know that for sure, but I'd like to think that they did give a thumbs up, you know, but they were probably stuck right in our seats. We were probably hitting about 6 or 7 G when this astronaut was giving this particular thumbs up, so they were probably just really but we do do a lot of research so we're trying to be as authentic as possible but sometimes we do take a little bit of creative license so because this we want this experience to be inspirational during the launch there was a shroud over the command module so all the windows were blocked during takeoff it was to protect the windows from being hit by debris or ice because the whole ship was actually it was super cooled just before takeoff So we took off that shroud so people could see out the window, especially like kids, like they want to look out the window and see the Earth get smaller and smaller and smaller and then obviously then when they're in space then they want to look out and see the Earth. So there's a couple of little tweaks like that that we've kind of just to make it more impactful we've changed it but we might release an update where it's totally authentic. But we often get people that say I want to do the whole experience as a simulator. I want to sit in the capsule for two days, so you can't really please everybody. We're trying to make something that will have broad appeal, but we're also trying to be very, very true to the original mission. We're trying to follow it as closely as we can.
[00:11:14.993] Kent Bye: So let's move on to the lecture VR. And when I took a look at this trailer video, I got the sense that this really could be kind of like one of the first PowerPoint for VR. So you have the capability to also put up slides in 2D. But perhaps more interestingly, you are creating different ways to do 3D interactions with people. So maybe talk a bit about how you imagine somebody using this tool.
[00:11:41.561] David Whelan: Okay, so this tool, PowerPoint is a really good analogy. So we're trying to make it so, so easy for educators to go on to this system. So as an educator, they can do this in 2D, like on true webpages, they choose the location that they hold the class. So as I was saying, if they're talking about Mars, let's hold the class on the Mars surface. And then we have a 4D effects system. So they can load in these effects the same as a PowerPoint slide. So as they're giving a lecture, they can say, right, I want a Curiosity rover to appear. So they just bring up a menu in virtual space, they click on it, Curiosity rover appears, and they can manipulate that and they can invite their students over and they can manipulate and work with the rover as well. You know, so it's a way for educators to create their own immersive virtual lessons, you know, so it's kind of VR PowerPoint is a good analogy, but then we also have tools that are networked. So in the video you can see two users putting together a skeleton, but those two users, one user is in America, one user is here, one user is in Ireland. So I literally pick up a bone off the table and I can hand it to somebody who's 2,000 miles away, but they only feel like they're two feet away, and we can interact and collaborate with tools. What we want to do is, with education, let's say we're teaching mathematics. Mathematics is very difficult for a lot of students, especially equations. We have a demo, it's not shown in the video, if you want to learn about ballistics or ballistics trajectory. there's a complicated mathematical formula to calculate ballistics trajectory. So what we've done is we put the user on a dock, there's a cannon right next to the user, there's a whiteboard there and the calculations are there. So we have a teacher appears, shows how you use a calculation and then you have to calculate on the flight the angle that the cannon needs to go to hit a ship that's like two miles out at sea and you can see the ship. and they need to sink the ship so that when the ship sinks it's going to be a coral reef that's the whole point of this so it's kind of showing a real world application for mathematics whereas if you give somebody a formula or you say pi and we're all going to learn about pi in a classroom setting if they don't have a real world application for these mathematics they're not going to remember it you know they're going to try cram it into their head, but as soon as the class is over, maybe like two or three hours later, that formula just completely evaporates. Whereas if they have a real world use for it, it will stick with them. Like our whole motto is learn through experience. We're trying to give students experiences that they remember. So it's like, say, riding a bike. So if I was teaching somebody to ride a bike, the last thing I'll do is put a load of text in front of them and show them some videos. You get them to ride a bike and that's how they'll remember it. In virtual reality, we can get people to do tasks that would otherwise be dangerous or impossible to do in the real world. And by them actually doing it, they're going to retain that a lot better and they're going to remember it.
[00:14:14.328] Kent Bye: Yeah, in some of the interviews I've done about people talking about VR in education, a lot of people tend to have the first thought of recreating a classroom and sitting in a classroom and teaching. I think that with VR, it has the capability to allow you to take a field trip anywhere in the world. But I guess the challenge in terms of trying to recreate that feeling of a field trip is that it takes a lot of 3D art, a lot of assets, and so I guess one of the biggest things would be like, is this something with like 360 degree photos, like photo spheres, or is this something that you imagine having to actually recreate with like 3D art to try to, you know, recreate these models of these places historically? Or where do you see that going in terms of actually setting the context of trying to go to a specific location, and that could set the context for the lesson?
[00:15:01.946] David Whelan: Well, there's a few different ways you can do this. So we feel recreating an environment in 3D space is obviously the best way, because you can walk around that environment. Whereas if you use a 360-degree photosphere, the users are stuck in one location. And if they move away from center points, it doesn't look right. It's not a great experience. We do see, like, eventually we can have a plug-in where a teacher can call up an address bar, they type in whatever address that they want, and then we can load in all the Google Street View data around that user, so they can visit literally anywhere in the world, but that's like a 360 degree photosphere. With our Mars environment, it's a 3D rendered environment, so they can walk around, if they see mountains off in the distance, they can walk to those mountains, they can get up on those mountains, they can go in that crater. So we feel like recreating environments is a lot better. So eventually, we're going to have thousands and thousands of environments. We'll allow a facility for, say, developers to upload their own objects and tools into the system. And then if they obviously want to charge for their environments and stuff, like as premium features, they can do that if they want. But we're going to give all these tools available to teachers for free so that they can put up their educational content on it. So they can choose whatever location they want. They choose the tools that they want to use inside the environment.
[00:16:08.146] Kent Bye: It sounds like LectureVR is an amazing tool. It's almost like creating PowerPoint, but yet, having a PowerPoint application, you still need somebody to make the presentation and still actually make the content of the presentation and then actually deliver it, and then have an audience of people. And so, to me it seems like one of the big challenges, as VR is just getting underway and starting with the gaming community, How do you foresee pushing this out, this lecture VR? Are there certain partners, or are there people that you have in mind that are gonna be in there actually using it from the start?
[00:16:39.960] David Whelan: Yeah, so at the start, we're releasing the alpha in a few weeks' time. We have a meeting room environment, and we feel that a lot of developers actually here at this conference, they have teams that are really spread out geographically. So you could have an artist who's in New York, you could have, say, a developer who's in Florida. And they, at the moment, they just do Skype and they try to share their screens or whatever. So in a virtual reality space, they can have a whiteboard. We have a virtual reality whiteboard inside the system. So I can go up and draw on the whiteboard and somebody in China can go up and draw on the whiteboard as well. And at the end of the meeting, they click on the screen grab icon, which is on the whiteboard, and that saves a JPEG into their pictures folder, or it will send an email to everybody who was at that meeting of the whiteboard session. That's why we're releasing it on the DK2. It's going to be a useful tool for developers, especially if they're really spread out geographically. And they can have private meetings, obviously, they can password protect. But in the future, where we see it is, if you've ever done a course online, so if you've ever done a Coursera course or an edX course, that's like distance learning. they provide lots of text content and lots of video content. That video content has to be produced. So to produce a high-quality video, you need a sound guy, you need somebody to do the editing, then you need a teacher in there to present. So it takes, let's say for an hour-long lecture, that could represent maybe 80 hours worth of work just to get that hour-long lecture done. And it might take, say, five or six people to actually get this lecture done. And that's what they currently do on Coursera and edX to get really good content. So universities like Stanford and MIT, they're like huge users of these systems, whereas what we're providing is a tool that a single educator can go in, they can give a class live, everything is recorded, so as they give the class live, we're recording all their movements, recording their audio, and then they can bring in the effects on the fly, and that's all recorded as well, so that class can be replayed over and over again. So literally, instead of using five people that take 80 hours to do something, you can use one person that takes an hour to give a really good presentation. So if you're really confident to give a presentation and you're not going to forget your words or forget your lines, you can literally do a class within an hour and have it published up on the system.
[00:18:43.415] Kent Bye: Is that using like a Kinect-based sensor to, you know, people set up a depth sensor camera like the Kinect and be able to record their movements that then you, you know, somehow clean up and translate into an avatar?
[00:18:54.187] David Whelan: Yeah, our original Alpha used a Kinect to record the movements and obviously then the teacher, the presenter, had a headset on him. But what we've been using lately are the Vive controllers and the headset and walking around within the space. The only thing that it doesn't track are the feet. In the particular video that's been released, the pre-Alpha video, we used a perception neuron and we just put two neuron sensors on the feet themselves. But we used the Vive controllers and the headset for all the other tracking data. so it would be nice say if I could just do maybe like two lighthouse sensors just maybe on the feet that would be great for us you know some kind of strap or whatever because you really get the presence of the teacher when they're walking around and they're just moving their hands and moving their heads so but we will provide options so if you have a Kinect that will record your movements if you only have say an Oculus Rift and no hand controls obviously the head will just move and we'll pre-animate maybe some movements with the hands so maybe the hands are clasped in front and they kind of move now and again our gesture now and again, but pretty much whatever tracking system is available, we're going to support that.
[00:19:51.499] Kent Bye: And so because you've been creating these tools to give these types of lectures, has there been any educational experience that you've had within VR that really stands out and kind of like the story of that?
[00:20:02.731] David Whelan: Titans of Space actually was the first time, like that really set it up. When I tried Titans of Space, I didn't know, like I'm a huge space fan but like there's so many moons that I didn't know about and I went through Titans of Space three or four times and I really learned a lot from doing Titans of Space and this was three and a half years ago now actually and that kind of really resonated with me and I said this is like huge and then I've three kids myself and I've put them into Titans of Space and now they're experts on the solar system they can tell me the moons and all the planets and stuff a lot of stuff that I didn't even know about you know so That was one experience that really resonated with me. But then you have Frucious, who's making some great educational content as well. His NEOS demonstration, I thought it was really, really good, because it shows the size of objects in relation to other objects, to the whole universe down to a tiny atom or a quark. So I think he's doing some really great work as well. But there's very few educational experiences available in VR at the moment, which is a real big shame, because any time somebody makes an educational experience, it really does well. There was another guy he made from ashes. You know, I think as a lone developer in Australia, I thought that was a very good experience. Very well done and a lot of thought put into it.
[00:21:06.027] Kent Bye: And finally, what do you think is the ultimate potential of virtual reality and what that might be able to enable?
[00:21:13.795] David Whelan: With virtual reality, one of the key problems with education, especially in developing countries like China and in India, is everybody knows that education is the best path out of poverty. So if you're highly educated, you can get a good job, you can have a good life. So in China, there's thousands upon thousands of hugely qualified students coming out of schools wanting to go to university. But because there's so few universities and so few teachers at these universities, they can't get into those universities. So there's huge problems with access to great education. Whereas we feel a platform like LectureVR, they can get a college education online from distance learning. All they need is an internet connection. and a HMD, hopefully they'll be very inexpensive in the next three to five years, which we see that will happen. So that's kind of the big, big plan that we're going towards. But it's all baby steps at the moment. Right now, we're just building a community, getting feedback from, like, we will approach MIT, we will approach Stanford and say, hey, guys, look, there's this new platform. Would you like to put some content on it? We're going to help you put the content on the system. And we'd like feedback. What kind of tools do you want us to build? And we'll build that tool set for you.
[00:22:22.310] Kent Bye: Awesome. Well, thank you so much. No problem. Thanks. Thanks a lot. And thank you for listening! If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash voicesofvr.
