Cerevrum Inc is building an ambitious educational platform starting with training people to become better public speakers with Speech Center VR. The basic mechanic is that you stand in a variety of different virtual rooms in front of an animated crowd of virtual listeners as you give a presentation. The app is designed to help people get over their fears of public speaking, but there are many other educational learning opportunities from a number of upcoming courses featuring public speaking coaches.
I had a chance to catch up with CEO Natasha Floksy and COO Olga Peshé to talk about designing their two educational applications Speech Center VR and their brain training application Cerevrum
LISTEN TO THE VOICES OF VR PODCAST
Natasha has an art degree, and there’s a strong design aesthetic is imbued within every dimension of Speech Center VR from the different rooms to the user interface to the highly customizable avatar system, which is one of the more impressive aspects of the experience. At the moment, you are a disembodied ghost, and so you never fully appreciate your own selected digital identity. But there is a wide array of identity choices, along with many different features and functionality within this app.
You can download Speech Center VR for free, upload a presentation PDF, and record yourself talking to a room full of virtual strangers. There are also interactive social components as well just in case you wanted to hold an intimate meetup there. There are a number of in-app purchases for getting a chance do some practice training within a variety of other different public speaking engagements. There’s actually a surprising amount of functionality included within the experience, including a supplemental eye training application to help to improve your vision.
There’s a number of small improvements that could be made including having a monitor for the presenter, improving the social behaviors of the virtual audience to be a little less uncanny, and having a little bit more intuitive way to advance slides more than swiping down on the side of the Gear VR headset. But overall, Cerevrum Inc. has built a robust educational platform with a lot of room to grow into many specific domains.
Donate to the Voices of VR Podcast Patreon
Music: Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to the Voices of VR Podcast. So, when I was at SIGGRAPH this year, I had a chance to attend the press conference for NVIDIA, where they were announcing a number of different new GPU cards, as well as some new offerings for virtual reality. So I had a chance to catch up with Bob Petty. He's the general manager for the ProViz business unit at NVIDIA. So we talked about some of these new cards and how they're really designed for professionals, whether in the special effects or virtual reality industry. And in the press conference, they talked a lot about artificial intelligence and how the NVIDIA GPUs have been gaining a lot more popularity in the deep learning fields. and how he sees some of that artificial intelligent applications fusing into the process of content creation within virtual reality. So we'll be talking about some of these trajectories of VR and AI and how they'll eventually be kind of fusing together and how he sees people will be using some of these GPUs either locally on their own machine or distributed into the cloud, as well as digital light fields and the future of graphics that is moving more towards interactive real-time ray tracing with these digital light fields. So, that's what we'll be covering on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. This is a paid sponsored ad by the Intel Core i7 processor. If you're going to be playing the best VR experiences, then you're going to need a high end PC. So Intel asked me to talk about my process for why I decided to go with the Intel Core i7 processor. I figured that the computational resources needed for VR are only going to get bigger. I researched online, compared CPU benchmark scores and read reviews over at Amazon and Newegg. What I found is that the i7 is the best of what's out there today. So future-proof your VR PC and go with the Intel Core i7 processor. So this interview with Bob happened at the SIGGRAPH conference that was happening in Anaheim, California from July 24th to 28th. So with that, let's go ahead and dive right in.
[00:02:15.090] Bob Pette: I'm Bob Petty, GM of the ProViz business unit at NVIDIA that encompasses the Quadro product line, our grid cloud graphics virtualization product line, and the rendering software unit, the IRAE, our optics, all the toolkits for high quality rendering.
[00:02:33.841] Kent Bye: And so we're here at SIGGRAPH and you made a number of new announcements of new cards So maybe could kind of run through what the cards are and what the primary use cases for those are Sure, our flagship card has always been the 6000 series from Quadro.
[00:02:47.026] Bob Pette: So the Quadro P6000 Pascal 6000 series largest framebuffer the desktop card fastest GPU out there most cores over 3800 cores and The design goal was not just to have the title of the fastest GPU, right? The design goal was to give developers and users what they needed for seamless VR, both for the creation of VR and the consumption of VR. And specifically for the Quadro line, we focus on the professionals. Our GeForce line, the 1080s, are targeted at gamers and consumers. The 1080 is similar to our P5000 that we announced, but I'll come back to that. So the use cases for the Quadro P6000 are architectural walkthroughs, in addition to the design of actual buildings, cities, cars, the consumption we think is the biggest growth area for VR. And so we want to make it easy for people to create content. from professional applications, whether those be Maya, Max, Goetia, SolidWorks, create as real as possible content with either photoreal rendering, easy ingest of 360 videos. The pro use case, we believe, for people to use it on a regular basis for collaboration, design, has to be as real as it gets. And so the P6000 with the large framebuffer, multi-cores are great for photoreal. That ties into one of our other announcements, the IRAE VR package, which is then the future interactive rendering, interactive ray tracing and VR will continue to dovetail. What we focused on at this show was the creation of those photo real scenes and being able to view them not just the normal simple panosphere, but be able to look at within a volume, have a different look, create a different view, turn lights off and on, change materials. Not quite fully interactive ray tracing on the fly, but that's where we're headed. So, a lot of cores, a lot of frame buffer. The 5000 series, again, targeted at the pro user, if you think of it kind of as a good, better, best, the 5000 and the 6000 are high-end cards. Certainly, performance-wise, it's about maybe a 30% difference. Depending on your dataset size, the 5000's got a 16GB framebuffer versus 24GB. Depending on your dataset size, the amount of computation you're also doing, whether you're augmenting it, bringing in live video, the developer or end user can pick the card of their choice. So, primarily the Quadraline designers, artists, creatives, but also the consumers of people that are going to buy their car in VR, design it, design their house, buy a condo that's yet to be built. And you don't want that looking close to what they get, you want that looking exactly like they get. And that requires a lot of computation and visualization at the same time.
[00:05:27.206] Kent Bye: And so at the press conference, you mentioned that NVIDIA was one of the creators of GPU processing. And so I'm just curious if you could talk about when that happened, as well as if we're kind of at the point where a lot of the software architectures for a lot of these tools have been changed in order to really take advantage of that GPU processing.
[00:05:45.296] Bob Pette: Yeah, so NVIDIA invented the GPU, if you will, back in mid-90s. I used to work at Silicon Graphics, and we had the geometry engines used to create big flight simulators and the like. That's where OpenGL kind of got its origin as IrisGL. NVIDIA found a very creative way to use that technology and create a GPU that could do not just visualization, but computing, because at the end of the day, visualization is computing. You're computing eye points, you're computing vectors, polygons. And they made it a true programmable unit that developers could use to do both compute and visualization at the same time. That's continued to morph as we look at the growth of things like artificial intelligence and deep learning. the GPU is perfectly suited to begin to understand our world better than perhaps humans can in some cases. At a minimum, maybe help humans understand the virtual world as best as it can. And it's not just drawing, it's compute and drawing. And we think that will revolutionize the whole design process. So if somebody is manufacturing an engine or a part or a car, There's a lot of repetitive things that artists or designers do that can be automated, whether they're drawing a flange or a tie rod or whatever. Once the computer, the GPU, senses that they're drawing this given the context of everything else around, the GPU can actually present options. I notice you're drawing a tie rod for this Mustang. Here's all the possibilities in the inventory. And so we think AI and shape recognition, object recognition, will help shorten that design process, improve the quality. Deep learning as well will help and that's understanding over time differences in ray tracing and materials, look for errors, decide what is truly good enough when something is truly resolved because this ray tracing can run forever. Deep learning can be used for that, it can be used for material creation for virtual reality. It also can be used in the design process and in the manufacturing of those designs that we talked about. picture robots with GPUs inside, constantly looking at, say, an assembly line. They'll be able, over time, as they put more and more of these components together, look for optimizations. They'll know when something failed, when something was rejected, and they'll be able to continue to improve that manufacturing process. You can use AI in the VR space for object detection. Think about military troops out there in the battlefield trying to figure out what that object is. Rather than have them interpret it, you want the GPU to search its vast database and immediately decide whether that's friend or foe. A lot of different use cases, all dependent on the programmability of the GPU, and that's why we, at NVIDIA, we're not just focused on games, it's our majority of our current business, but we're focused on the use cases where we know there's still headroom and user demand to shorten the design process, shorten the time to insight, shorten the time to discovery, and that often is not just a computational or a visualization problem, it's both.
[00:08:47.455] Kent Bye: Yeah, and in the process of doing the Voices of VR for the last two years and doing nearly 500 interviews now, artificial intelligence has been one of those topics that has been emerging and getting a lot of attention and attraction in interviews that I've done on that. And so it's actually kind of inspired me to go start the Voices of AI, where I went to the International Joint Conference for Artificial Intelligence and did like 60 interviews in seven days. And so in the process of talking to people, I was really asking, like, Do you foresee people deploying these artificial intelligence agents that are learning on the fly and being trained up on consumer hardware? And a lot of people are saying, you know, it's so computationally intense that you have to kind of do it beforehand before you actually deploy. Whereas you can deploy like the weighted network, but yet some of the initial kind of tuning that you have to do with the gradients of these neural nets really kind of require vast cloud computing rather than just kind of relying on something like somebody's phone, for example. Now, with VR, you have the capability of having these high-end GPUs with anything from a 970 to a 1080, or if you get some of these more professional GPUs in there. I could see people who are doing VR development getting a Docker container to be able to set up an environment to very quickly integrate with this parallel GPU processing, kind of tinker around with rather than paying a cloud service to do that GPU processing to kind of do it on their own computer. And so I'm just curious if you've thought about the different specific use cases of how you see people using AI within their own computer.
[00:10:19.340] Bob Pette: Yeah, no, that's a great question. And I think at the end of the day, if I look 20 years out in the future, I think our view is that more and more processing and visualization will come from the cloud. But it's not going to be a switch that flips. And you have people today who've got three or four GPUs in a workstation that are using that as a supercomputing device for AI, for HPC, for deep learning, as well as visualization. You mentioned the containers. I think one of the core elements of I think local processing and cloud processing kind of growing together is kind of a common deployment scheme or common development scheme. So, good toolkits, applications that can run locally, or in hybrid mode, or in the cloud, and containers are a great way to do that. I don't think it will be one or the other in terms of where the processing happens. And I think the cloud is going to be necessary with the Internet of Things, with all these smart cameras, smart UAVs, and smart devices that will include sensor information in them as they're made. You're going to need a network that extends well beyond just some local data center. let's envision that you start putting sensors into every car that's built. Not just the ones you have today like LiDAR, radar, cameras, but maybe like stress and strain sensors on the chassis, on the body, and those are talking to the cloud. That's contributing and improving the training and the learning process on how to continue to optimize that manufacturing of that car or to optimize the visuals in a camera. And so, those things are going to absolutely require cloud, both mainly for the connectivity, but also just for the vast amount of training and computation that has to happen. But certainly, in an individual level, you'll see more and more AI just go into the local workflow, if you will, so that a designer can truly benefit from that object detection, shape detection, pattern recognition. and you'll improve quality in the locale. That then needs to feed into some greater system that improves the software. So it's harder for artists or creative to make mistakes because the software is learned. And now the next gen of software comes out that includes AI and it's a pattern. But you want to have enough compute power on the desktop to be able to do true AI while you're building, while you're playing, while you're using, and not completely depend on the cloud.
[00:12:35.093] Kent Bye: And so when you look at here at SIGGRAPH, there's a lot of visual effects industry that have their kind of established pipeline for doing compositing and visual effects within a 2D medium. And it seems like that workflow and tool set is pretty well defined with a lot of big players, Nuke and other tools that are out there. But when you introduce something like digital light fields, it seems like something that's volumetric data that is a completely different paradigm in a lot of ways that has to start to rethink a lot of the tool set for how people are working with these volumetric digital light fields and to me it feels like that's sort of like the emerging format that I think a lot of the most photorealistic experiences in VR are going to be these type of either direct light fields or synthesis of light fields with other compositive light fields to do all sorts of virtual worlds. And so, maybe you could talk about NVIDIA's vision for what that pipeline and workflow is going to look like.
[00:13:30.060] Bob Pette: Yeah, that's a great observation. I do think light fields are how people want to view, whether that's video-created panospheres or rendered panospheres or point cloud panospheres. The one issue you have with them is that the camera is at a certain height and a certain angle, and if you rotate your head it doesn't look real. We all know about all that. And that's where light fields really contribute to having the right parallax, the right thing happen as you move around. It is so computationally intensive that it's not something that's readily done today, and what we're demoing in the booth is something that took hours and hours and hours on multiple GPUs, and that gives us maybe a meter cubed area to move around in. still allows you to look around a column or to bend down and look up and get the right view, but it's truly, truly computationally intensive. That'll get better over time as people have access to more compute, as algorithms get optimized, but it also has to be easy for them to implement. The compute will take care of itself. either access to the cloud, access to a data center, more GPUs in the system. They don't have to worry about how the light field gets created from a compute standpoint. What we're working on with iRay is making it easy via iRay plugins for all of the apps, all of the creative apps, whether it's Max, Maya, Katia, SolidWorks, NX. If they're using the iRay plugin within the workflow, just as they would normally place a camera to render, place a series of cameras or a volume of cameras without really having to go through and manually put a camera at every three inches or every four inches. It can't be that complex or nobody will ever do it. And so what we're working on is the ability to define a volume of where you want the cameras, where you want to look out. We'll figure out how many cameras really need to be in that volume. and do the computation and return this light field blob, VR blob if you will, for the iRay VR viewer. So the key is it's got to be in the workflow and so we're targeting the most often used apps. It's the same way they place a camera today to do simple pano rendering, they'll be able to place that light field camera. And just, you know, with a little bit of extra input of length, width, height, or a path of where someone wants to walk, which is a common feature in simple camera rendering, they'll be placing a volume instead of a point. And we really want to get it to the point where it's that easy. Instead of a picture, instead of just a small... circle, you're putting down a rectangle or a cube, and then iRay will do the rest. Connect to a local GPU, it takes a certain amount of time. Connect it to one of our DGX supercomputers or the cloud, takes a different amount of time. Less worried about where the computation happens and more worried about making it easy for people to access that computation, because that's the way people want to look at VR scenes. It's nice to do a simple panorama. You know, if I was looking at a baseball game in VR, I'd want to, like, get close to where the runner was getting tagged out, or close to the batter, and I wouldn't want to be restricted because there's one camera. I'd want to have a cube of cameras all pointed at home plate, or all pointed around the base, so that I could walk around the base. So I could really get there and yell at the ump, you know, because he really missed that tag. and that's going to require light fields, right? And the technology exists and our goal with iRayVR is to make it easy within the workflows to do that.
[00:16:54.199] Kent Bye: Now I know that in the last week or so the Kronos group has come out saying that they want to support this common interchange format for immersive media just as the images on the web have the JPEG or there's different files of MP4s and standards for how to encode video that they want to create this GLTF format to have a full range of anything that could include like a full scene in like a Unity export or perhaps even light fields that you're able to take as a input and be able to read and see with a variety of different virtual reality head mounted displays. And so I'm just curious from NVIDIA's perspective if you're thinking about how to take and output the IRAE into a GLTF format so that it'd be compatible with all the different VR headsets.
[00:17:38.262] Bob Pette: Yeah, so we definitely want broad adoption and certainly support standards, but we don't want that to stifle innovation. So we'll definitely work to influence the standards so that it can incorporate what we can deliver, and the standards shouldn't be based on the lowest common denominator, if you will, right? But we don't want to do something that's one-off, that is too hard for people to interchange between applications or viewers. At the same time, we don't want to basically not do light fields because everybody else can only do panospheres, right? So it's a balance and I think that you start first with what's possible and what people need and then you start influencing the spec and I'm all for a standard spec as long as it's inclusive enough that people can continue to innovate and not be stifled by that.
[00:18:22.326] Kent Bye: I'm just curious from your perspective to kind of reflect on seeing this virtuality industry really grow over the last couple years and you know how you kind of make sense of some of the big milestones that you've seen as back from 1968, from when Ivan Sellerin first created the Sword of Damocles, into the peak in the 90s, into now consumer VR in 2016.
[00:18:43.339] Bob Pette: Yeah, so good or bad, I guess I was there for most of the time frames you just mentioned. I haven't been at Silicon Graphics for 21 years. VR for us was commonplace, and at NVIDIA, VR was commonplace, but it was on a much larger scale, right? You have the same issues that people are facing today with how long somebody can be in a VR, a virtual environment. The biggest issue in the step function change was the commoditization of the headset. And it's funny because people have tried 3D glasses and stereo displays. But I think, you know, when Oculus first kind of hit the scene, a lot of developers who didn't have the opportunity to program in VR, in a virtual world, in large venues and enterprises, started tinkering around and realized the power of VR. And that just started a whole wave of bringing back what has been around, like you said, for decades. That's the most obvious change and what brought about this rapid pace of innovation around VR and trying to solve some of the same problems that we were solving decades ago, eye tracking and intent and how to collaborate and communicate in VR and the like. I think the next piece is the capability of the PC or the computer in delivering a true VR experience and that will continue to be the case to innovate what people can do. So back in the 90s, you know, virtual environment was a cave, reality center, even headsets connected to a computer that cost maybe a hundred thousand bucks. Now we've got VR ready GPUs at a few hundred dollars and so you can take all the intelligent people out there who can afford a couple hundred dollar GPU and put their creativity and contribute to this VR development on mass versus having a few scientists at the largest oil companies and the largest automotives just doing most of the VR development. So I think obviously the headset kind of woke people up to it's not just for the high-end. And then the availability of GPUs that could process what would take a supercomputer back in the 90s that could do that processing in real time so that people could immediately see the effects of their work and not have to go to a cave or a large reality center to see that virtual experience.
[00:20:56.566] Kent Bye: Great. And finally, what do you see as kind of the ultimate potential of virtual reality, and what am I able to enable?
[00:21:04.752] Bob Pette: So for me, and I think for NVIDIA in general, people just naturally want to collaborate. They want to talk. VR shouldn't get in the way of people looking each other in the eye and seeing facial expressions and working together at a whiteboard or working together on a design, being up all night. It should allow us to transcend distance. It should enable us to not have to fly 15 people from around the world to make a design decision. And so the experience has to be good enough that everybody can stay in the environment. It's got to be real enough. You know, audio needs to come from the right direction. The visuals we already talked about in terms of photoreal. Haptics are really important. You're not going to get somebody drawing on the whiteboard if it doesn't feel like they're really drawing on a whiteboard. You're not going to get somebody that's doing virtual training for maintenance if they're not really getting the right level of torque on a part. They're not going to really want to use it. I think the challenge is to make VR more than just a visually compelling thing, but truly mimic life. And that means sight, intent, voice, touch, feeling. I think you've got to get beyond having to have controllers in your hand. It could be gestures, it could be eye-tracking UIs, it could be voice commands. but it needs to feel more like you and I would be talking now, and we should be able to do that over distance, and I'm seeing your avatar, and it looks just like you, you're seeing me, it looks just like me, the voice is coming from where you are, if we're holding something together, it both feels like we're holding something together, and that's all possible, and a lot of the work we're doing with VRWorks, we announced the stitching SDK, previously the audio SDK, we've got haptics and physics, We're really trying to simulate the real world. And I think you'll see more and more both games and certainly professional apps where that ability for people to communicate and collaborate is going to determine how fast VR takes off in their industry. If they still have to take off a headset and get on a plane and fly to see somebody to ask them questions, we've probably failed. But we're already seeing today where it's making a huge difference and people aren't having to get on a plane to do that.
[00:23:08.438] Kent Bye: Awesome. Well, thank you so much, Bob. Yeah, thank you very much. Appreciate it. So that was Bob Petty. He is the GM for the ProViz business unit for NVIDIA. And he was talking a lot about the new GPUs that were announced at SIGGRAPH and some of their applications within the professional graphics industry and where he sees GPUs going with this kind of fusion between artificial intelligence and virtual reality moving forward. So I have a number of different takeaways from this interview is that first of all, it seems like that these digital light fields are something that are really super computationally intensive. So in the demos that I was able to see at SIGGRAPH, Bob mentions that some of these renderings took two to three hours of just churning through and processing all the different information that is there. And it sounds like that some of the calculations that are happening within these ray tracing in order to kind of simulate the light rays moving around and give this physically based and photorealistic type of rendering. So it does take a lot of time to process on the back end before you go in. And so it isn't at the point of being able to be dynamic in real time and interactive at this point. However, in some of the applications that they were able to show, you're able to select a clock between four different times and By choosing a different clock, you're seeing what the sun would look like in that room given that time. The point is that if you're able to capture a full scene with that reflective properties, then you can dynamically change the light. But this demo did take a few minutes just to even load up and to see. And so I want to ask Bob about how he sees whether or not some of this processing is going to happen in the cloud or on your computer. And he seemed to think that there's going to be some combination of both. So one of the things that Bob said that in the future, that one of the directions that they're headed is to be able to do interactive ray tracing on the fly. They're not there yet, but that's kind of where they're heading. And not sure if that would require doing some cloud processing distributed approach to be able to really do all those calculations, or if the GPUs are going to be local within these workstations are going to be good enough. Now, one thing that I noticed within SIGGRAPH is that the photorealistic type of scenarios where you're really rendering out these scenes to get them to be this super high fidelity, one of the industries that came up a lot was the architectural visualizations. And so be able to render out what a space might look like and to have it as photorealistic as possible. So I like to think of the GPU as this massively parallel processing compute power as well. And so it seems like artificial intelligence is being able to use a lot of that, especially with this deep learning. So you're essentially able to model the interactions of thousands of different neurons interacting with each other at the same time. And I think that NVIDIA is putting in a lot of different marketing efforts and initiatives to be able to really go after the artificial intelligence market. So it was really interesting for me to start to hear how some of the AI and VR worlds are going to start to come together. Especially when you start to think about how you can use some of the feature extraction capabilities of machine learning to be able to kind of visually detect what you're doing and then kind of be a smart assistant to be able to say we're detecting that you're working on this specific type of object and because of that we can more intelligently give you a subset of other related objects that may be useful with whatever workflow that you're doing. So in a lot of ways this GPU technology is going to be able to enable lots of different artificial intelligent applications within your computer. But you know in asking Bob whether or not he saw in the future with all these VR developers having access to these GPUs whether or not they're going to be able to really take advantage of that to be able to train up different AI. And it sounds like there's going to be a little bit of a combination where there's still going to be a lot of different AI applications that really need like cloud computing to be able to really train things up. But one example that he gave was the cars as they're going out and driving, they're able to still be learning on the fly and perhaps take some of the data that they're gathering locally and sending up to the cloud and be able to kind of do this massively parallel processing vision brain that is able to grow and evolve over time. So when I was at the International Joint Conference for Artificial Intelligence in New York City, I saw David Silver from DeepMind talk about his guerrilla architecture, which is essentially the general reinforcement learning architecture, it was basically able to take an input from one distributed node and be able to feed it back into this global brain. So you can just imagine this one giant brain and all these different autonomous agents out there gathering data that's feeding back into this global brain, and that global brain is then being copied and sent back out to all these agents. And so It's essentially like all the learning that's happening from all the individual agents being able to improve everyone. And so I think that was some of the thing that Bob was talking about where he kind of sees like this distributed global brain approach that's going to be happening in addition to being able to do some processing locally on these machines as well. And I think with these virtual reality machines, as well as self-driving cars, there's going to be these GPUs out there. And with that, it's going to be a little bit like a lot of new capabilities that we hadn't even really thought of. And so you can kind of think of these self-driving cars as this way of getting this global brain out there in order to do a lot of distributed processing. So I will be diving into a lot more details of the AI implications within the Voices of AI, hopefully launching within the next couple of months. There's a couple of other conferences that I'll be covering as well. So I'll be diving into the AI portion of the Voices of AI here in much more detail. I think there's a lot of really interesting things that are happening in that as time goes on, there's going to be more and more of a fusion between the worlds of AI and VR, but I just wanted to feature some of the thoughts here of Bob Petty, who I think that NVIDIA is really one of the companies that are really at that cross section between virtual reality and artificial intelligence and being able to enable both of these emerging technologies. So that's all that I had for today. I just wanted to thank you for listening. And if you enjoy the podcast, then please do spread the word, tell your friends. And if you'd like to tell the world, then go leave a review on iTunes and some of your thoughts about what you've been gaining from the show. And if you'd like to support the podcast financially, then become a donor. Go to patreon.com slash Voices of VR.
