Andy Beall is the CEO of Worldviz, and has been doing VR since 1992. I had a chance to catch up with him at SIGGRAPH where he told me about the type of work that Worldviz has been doing in the VR space since 1992, including creating warehouse-scale tracking spaces that go up to 30 by 30 meters. Andy has also studying the psychological effects of VR, and the first time he tried the Oculus Rift he experienced the sensation of vection, which he had only theoretically studied before. So he shares with me what vection is, and the differences between vertical and horizontal vection and why it’s important in VR. Andy also has some pretty amazing stories of running around a warehouse wearing an Oculus Rift and jumping over virtual hurdles.
LISTEN TO THE VOICES OF VR PODCAST
Donate to the Fatality & Summer Trip
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to The Voices of VR Podcast. So Andy Beal has been involved within virtual reality since 1992, and he's currently the CEO of WorldViz, which is a company that started in 2002 working on bringing virtual reality technologies to academia primarily, but also other marketing and warehouse scale applications in the engineering realm. So I had a chance to catch up with Andy at SIGGRAPH, where we talked about WorldViz and what they're currently working on, a little bit about the history of VR, as well as some of his first experiences with the Oculus Rift and really pushing the limits of large-scale tracking to VR, as well as the concept of vection, which is an elusive concept that he had been studying for a long time but never actually had a chance to experience it until the first time he put on the Oculus Rift. So, that's what we'll be covering on today's episode of the Voices of VR podcast. But first, a quick word from our sponsor. This is a paid sponsored ad by the Intel Core i7 processor. If you're going to be playing the best VR experiences, then you're going to need a high-end PC. So Intel asked me to talk about my process for why I decided to go with the Intel Core i7 processor. I figured that the computational resources needed for VR are only going to get bigger. I researched online, compared CPU benchmark scores, and read reviews over at Amazon and Newegg. What I found is that the i7 is the best of what's out there today. So future proof your VR PC and go with the Intel Core i7 processor. So this interview with Andy happened at the SIGGRAPH conference, which was happening in Anaheim, California from July 24th to 28th. So with that, let's go ahead and dive right in.
[00:01:58.780] Andy Beall: My name is Andy Beal and I have been working, playing in VR for almost 25 years actually. Currently with a company called WorldViz, co-founded by two others in 2002. Started in 92 during my doctoral thesis as a way to build virtual experiments for studying people.
[00:02:18.330] Kent Bye: Great, so maybe you could talk about how you went from getting your doctoral thesis into VR and then eventually starting WorldViz in 2002. What was happening there in those 10 years?
[00:02:28.532] Andy Beall: Sure. So my doctorate thesis is an area of experimental psych. And I sometimes explain this to people as I'm not the kind of psychologist that's going to help you with your problems with your brother or your sister. I'm the kind that figures out how everyday humans can drive a car, how everyday humans can fly an airplane. Ended up doing my thesis on control flight. Today, self-driving cars are actually starting to occur. We're seeing them. But 20 years ago, that wasn't the case. We really didn't have a clue how to build a machine that could do a complex task like that. Instead, we had humans and animals that can do it, but we didn't know how they did it yet. So I think of my area of psych as reverse engineering humans in an ethical manner. And what you need is you really need a really good stimulus, you need a good visual display, you need a good acoustic display. If you really want to get complicated, haptics, other senses, to expose somebody and start to probe their mind, figure out what makes them tick. So for me, and actually for our company, the foundry of our company was using VR as a scientific tool. And our initial customer base were all scientists. So when we launched our company in 2002, We were catering exclusively to university research labs, people that barely had heard of VR, dying to get their hands on it, wanted to focus on their research, get better publications, advance their career, etc.
[00:03:45.299] Kent Bye: And so you were primarily producing some virtual reality HMD technology then? Or what else were you providing?
[00:03:51.062] Andy Beall: We were primarily producing software that would glue together various other HMDs. So I actually did in 94 build my own head-mount display. So way back using Sony Watchman TVs. The reason was there was like only one other offering back in the early 90s where you could buy something. And I was, from a research point of view, and personally as a grad student, way too poor to buy one of these commercial devices. So I built my own out of Watchmen TVs. The company has supported the dozens of headsets that have been available prior to sort of the recent revolution. And now, you know, we're supporting hundreds of different headsets through software, through the software that we make. So we make a simulation engine similar to a game engine, but focused on VR, focused on what's needed for doing immersion and presence. We connect to not just the displays, but also the input hardware. That's just as important for the equation, right? The tracking devices. So we actually make a tracking device ourselves. So that's the piece of hardware that we get involved in. We call it warehouse scale optical tracking for allowing people to walk around one-to-one scale, a really large environment, like we're talking 30 by 30 meters, something on that order, and make that easy enough so that a professional without a full-time staff can run that, meaning an architect can run it. or an aircraft designer. It doesn't need to have a couple of FTEs that are just going to maintain this mocap studio.
[00:05:15.035] Kent Bye: And so maybe you could describe to me what was happening in 2002. It seemed to be a little bit like the winter of VR, where there's a lot of excitement in the early 90s, and then things kind of died down until around 2012. But starting your company 10 years before that, what was the kind of environment like to start a company and really bootstrap it in that way?
[00:05:34.062] Andy Beall: Yeah, there's definitely a couple of winters, I call them like the dark ages of VR. So, you know, during that winter, if you will, there had been a steady growth of user base using VR. So, you know, the shiny objects of like Sony jumping in and then abandoning after a couple of years. Canon kind of dipping their toes in it and kind of pulling back for a decade or two. a couple movies that I think did more to hype and to harm almost people's expectations and distortions. But all throughout that time in the research community, in the scientific community, there was just a monotonic growth, just a steady slow growth of the use of that. So, like I was referring to, through my graduate career, I created a bunch of tools. I was giving them away for free. I was having more and more people saying, Andy, can I get access to doing VR? Because I was, with my lab mates and colleagues, showing results that were finally publication quality, you know, meeting peer review, using the tech. So in 2002, I was getting enough requests that one of my co-founders was seeing the market potential of that. So even in 2002, there were enough inquiries, enough people obtaining government grants that would allow them to buy the gear necessary. They needed a company that could back that with a software platform that was fairly hardware agnostic, so they could pick and choose the pieces that would fit their research program. There was not a one-size-fits-all, as I still believe there's not a one-size-fits-all today. And we provided support. So we knew it well enough and I knew the do's and don'ts and the pitfalls of VR, the ways to avoid getting people sick. But really, when is the research question of such that it warrants the use of this technology versus traditional tech? And that was a value and it's a value we give today, I think, to our customers. We talk people out of things sometimes. At the end of the day, it's not going to serve the customers need or the researchers need you know We're wasting both people's time. So I think you know, we gained a lot of respect in the industry giving honest You know expert consultation on how to use this based on a lot of experience and if we go back to 1968 look at Ivan Sutherland and the sort of Damocles it's something that was a
[00:07:41.955] Kent Bye: supported by the military and DARPA putting money into this research and it seems like the military has been a part of funding and supporting VR within the research community since that time. But yet some of the applications of VR and going to conferences like IEEE VR have a chance to talk to people who were in industry to talk about like the aerospace and automobile industries that have been using VR since near the very beginning and then big rush of excitement in the 90s but yet then continuing on then there's a little bit of an opaque not a lot of transparency in terms of who's using the VR technologies because it's either for you know their proprietary reasons or it could be something that's from the military so I'm just trying to get a sense from your perspective what types of applications that people were doing with VR from your perspective during these kind of winter times.
[00:08:32.690] Andy Beall: Alright, let me get to that. But first, I like the historical note you made about Ivan Sutherland and the Sword of Damocles. I think that's rightly citing the beginning of a near-to-eye display with head tracking. I also like to cite, though, going back another 20-some years to the Link flight simulator. And that's really what I give credit as the grandfather of VR, is flight simulation. And I also like it because it points out that the original VR, so VR is interactive simulation. It's not just watching something, it's not just experiencing something, it's interacting with it. And there's some sort of a database or some sort of a knowledge system. that makes it interesting. The flight simulator back then had no digital display. It was only your avionic panels. And the goal was to teach people how to fly through a cloud, trusting their instruments, okay? And actually, if I can, it goes back to a very important sort of like human control interaction point. And that's that when flight was first became sort of massively used during World War I, They discovered that people, if they flew into a cloud, had about a 90 second expected lifetime. Because they would go into what's called the graveyard spiral. They would start to tip the plane. They would go into a steepish bank, not realize that they're in a bank, glance at their instrument panel, see their attitude indicator, the artificial horizon. look like that they're in a bank, they would correct it, but they would correct it at a rotation speed that their vestibular system would register it. Now they are confused, but absolutely in a state of belief that they're in an opposite bank of equal magnitude. Now the instrument's pointing out that they're level. Now they have even more reason to think that the instruments are failing. and they actually go into an oscillation, similar to like a pilot-induced oscillation, and literally it's about 90 seconds, two minutes, and they're in an uncontrollable state of flight. So this flight simulator saved lives, and it was just getting you to be able to understand the complex relationship between these different dials. And that same sense of flight simulation, Microsoft had a great flight simulator for a number of years, it truly was a powerful form of virtual reality. It just wasn't sensory immersion in the sense of what popular VR is today. So, the Soda Damocles was the first sort of headset, and I think it gave us that sense of a very natural interface into a natural-seeming world, although the graphics back from the Ivan Sutherland day were, you know, interesting. Your question on the customer base, so, you know, we've honestly been very little involved in military. We have worked with military contractors, but not on military projects. We don't have security clearance. We've never really pursued that line. But for most of our history, I would say about half of our customers' projects are essentially in the public domain. They're government-funded, non-military funded. So these have led to publications. So if you search by our name or by a product's names, like Google Scholar, you'll find literally hundreds and hundreds of scholarly articles that are published using our technology. It goes into areas within the research realm that include some of the things I was talking about, human control of machinery, like driving, flying, also just human interaction with other humans. So since mid to late 90s, there's been a real interest by social psychologists to utilize VR with virtual humans to study interpersonal communication. And it's been a hugely growing field. In the spectrum of problems that our customers work with, right now, just to do a snapshot, the easy, kind of the low-hanging fruit, is architecture construction, because that's large projects are already digitized, the assets are already on digital machines, albeit in all sorts of formats, which is still one of the big hurdles today, but doing a large-scale, one-to-one product design. So I like to say, you know, like designing a hospital operating room is great for VR, because The drawings are all there. It's easy to go and try. And it pulls on the sense of one-to-one large-scale tracking. So you need to move around. You walk through it. You push things around. You place virtual humans all over the place. You can almost simulate a theater, what it would be like during a heart bypass. That kind of product visualization. So not like modeling a shampoo bottle. That really has no place for immersive VR particularly. Next, I would say along the scale. with clients that we're working with on a day-to-day basis is marketing, using this tech just for either PR for a new movie, PR for a new product line, PR for a new hotel location. That's very real, and we've been doing that for years. The next, I would say, is training. There, again, in manufacturing, you have companies that are building like a giant tractor. They've got the designs already in digital format. That part's easy. Training involves adding some interactivity to it, so that's where it's a little harder. And then into our kind of more mainstay area, and that's just the full research market. That can be anything.
[00:13:23.183] Kent Bye: And maybe you could kind of recount some of the big highlights from the last couple of years since this modern consumer VR revolution that's been happening, and your perspective of how you kind of make sense of what's been changing and evolving within the VR ecosystem there.
[00:13:36.689] Andy Beall: Okay, sure. I'll give you one of my first reactions on trying Oculus for the very first time. It had just come out, the first units in public. I was at a show in Orlando. Everybody was standing in line. It was a seated demo. It was kind of like a Doom-like space. You're moving around with a gamepad. I tried it and within seconds I stood up and the guy running the booth tried to push me back down in my seat and I said, no, I need to stand. And I was just zooming back and forth against a wall. And I was swaying back and forth, almost falling. And I said, You have no idea. This is the first time I've ever had, and the term is vection. I had an illusory sense of self-motion. And I've actually studied this and tried to create that in VR. I've never had it happen before. It wasn't the resolution, because the Oculus wasn't killer resolution. It might be the latency. It might be the lightweight aspect of the headset. But it was a wonderful sensation. I was literally just trying to get myself to fall over. And that was an achievement. And I went back to our booth and I told everybody, you guys, Oculus is changing things. I don't quite know what it is. I knew what the recipe was. It was, you know, the convergence of the cell phone screens and the gyros and a good idea with using GPUs for the distortion. But something about that composition led to what we started exploring back in our offices, and that was we used a simple fanny pack and a wireless link. And we have a couple videos on our website where we have people running around a warehouse with an Oculus. And this was back in, I guess, 2013. And the eye-opener for me was, not only am I getting some sort of perceptual phenomenological experiences that I've never experienced in VR before, But I'm running. So we have virtual hurdles. I mean, I can run virtual hurdles. I can, you know, leap over virtual pits. Right at, I would say, the edge of being able to do sports in VR, okay? And not just sitting on your sofa playing Madden football. I mean, I'm actually running, okay? I'm able to keep track of where other objects, other players are in VR. And whether you're a sports fan or whether you're a sports trainer, the fact that those kind of motions and the complex interplay between multiple live people, from a perception motor research point of view, or just this is a whole different area of experience, leads to my final thing that excites me most right now, and that's the social VR. We've had customers doing social, meaning like collaborative virtual environments, you know, eight, 10 years ago. But doing it day-to-day now, as we do in our offices, has gained and seen hundreds of customers now who come in, and instead of me being on the outside and showing them and grabbing their shoulders and pointing things, I'm in there with them. I'm part of the experience, and I'm live kind of as their host. It's a completely different experience for the customer and for me. And I think it's the, in some sense, you know, VR is, most people are just touching the tip of the iceberg, I think. This is the tip of that tip, in a sense, is what the social collaborative side is gonna be, so.
[00:16:34.144] Kent Bye: Yeah, it's interesting as you talk about the vection, because my understanding of vection is that, for me, it's like a motion sickness trigger, so that if I have the sense of moving forward, but my body isn't actually moving, even if it's optical flow flowing through the high textures. But what is it specifically that's mysterious or something that was new? Do you think it was the field of view, or what specifically, like how do you describe what that vection actually is?
[00:17:01.267] Andy Beall: So, it's interesting you say that you find that as like a precursor to getting motion sick or cyber sick. Yeah. Have you done the shower curtain vexation illusion? No, I haven't. What's that? The shower curtain vexation illusion is where you sit on a stool basically and you're surrounded by a circular ring above you that has like a black and white striped shower curtain all around you. And this one works for everybody. It's just a real physical shower curtain, okay? And you start that shower curtain spinning around you at a fairly slow rate, and you absolutely feel like you're spinning on the stool and that the shower curtain's stationary, okay? And that's, I don't know of anyone that I think gets sick from that. If you ramp it up reasonably, it's a nice static sensation, okay? That one works particularly well because the axis of rotation is vertical, it's aligned parallel to gravity, so you actually have no static vestibular cue conflict. Now if you do this shower curtain illusion by staring into a drum and the axis of rotation is in the horizontal plane, There, you're constantly conflicting with your gravity sensors in your head, your down sensors, because you're trying to go up. And there's actually an interesting perceptual phenomenon. You'll feel like your feet are swinging up, and at about 90 degrees, most people suddenly get swung back down, and then they go back up and back and forth. And that can be sickness invoking, all right? So why I think we're seeing that in headsets, so the only kind of infection I had achieved in a headset prior was with these older displays lying on my back, staring straight up, so that I was not having a gravity conflict. And even then it was quite weak, okay? I think field of view, but field of view is not it. There's no singular answer. There have been a number of people that have studied vection. You can actually get a very strong sense of vection looking through a small, almost like peephole, okay? It has a lot to do with your belief state. You have to believe that you're looking through onto, call it like an allospheric frame of reference, a large geographical frame of reference. So, the Oculus, I honestly do think the low inertial characteristics, the low impedance of head motion, the light weightness of that display, I had a lot to do with that.
[00:19:09.154] Kent Bye: I see. Wow. So, but do you think it's connected to being a trigger to motion sickness then, for creating this vestibular disconnect?
[00:19:16.480] Andy Beall: I think vection could be a way to detect it and it could be a trigger. You can get vection in different axes and like I was speaking with the conflict, if you're getting vection in a motion direction that's going to be conflicting with, we're stuck in terrestrial environments with gravity pointed in a constant direction. If you are doing barrel rolls in VR, you're almost guaranteed to get sick, okay? If you're doing rotations about a vertical axis, much, much less likely, but still you're likely to get sick because the onsets and offsets of that is actually going to be in conflict with your vestibular semicircular canals. So, in virtual environments, this goes back to research from the late 90s, if you want to navigate a large virtual space, say with a joystick, only use the joystick to go fore and aft. physically rotate your body left and right so that you're getting a cue with your vestibular system the same as what you're seeing, all right?
[00:20:15.737] Kent Bye: Yeah, yeah, that's for sure. Well, for WorldVis, starting in 2002, you've kind of gone through that winter and now it's going through this transition with the consumer VR and proliferation of different tools and services that people are providing. And so, just curious how WorldVis is pivoting or adapting in order to still maintain your business within this new time period.
[00:20:39.324] Andy Beall: So, you know, in many ways, WorldViz has always been about bringing professional VR into the grasp of somebody who's not a VR developer. And I don't see that really changing right now. I see, you know, there are other providers of VR tools. You know, the game engines are becoming extremely popular right now, extremely capable. They may be more popular, but they're not necessarily more accessible to a researcher or an aircraft designer or an architect to get into building a real simulation. It is better than it was 10 years ago, but I think there is still going to be a need for somebody who can simplify that and speak to the professional without requiring any real background expertise and kind of protect people from themselves, so to speak, on using the wrong hardware, using the hardware the wrong way. I think what we're seeing right now, and a bit of a direction we're going, is trying to ride almost on top of, say, the game engines. And there's an interesting sort of dynamic right now, a certain pressure, I would say, by the game engine companies as they're changing their modeling. to make it possible for a company like us to continue with our own game, our own simulation engine rather, but also embrace other engines for when the situation with the client merits that. And what we are working on is providing a framework or a structure that allow you to sort of write virtual experience in a code that's almost like a meta code. that almost is an abstraction. So we've always kind of embraced a method of abstraction to keep our clients, our customers from having to get into the C++ or the OpenGL or the DirectX. We've always believed that, yes, you can always get more power, you can squeeze extra oomph if you drop down, but is it worth your while? If you're trying to do rapid prototyping, if you're running a research lab, you're better off probably focusing on your research and doing faster cycles rather than squeezing out more polygons per second. Leave that to the game titles. So I don't see that recipe for success changing right now. I think in some sense there's even more available to a company like us that wants to bring it all together and make it accessible to, in some sense, a layperson.
[00:22:49.323] Kent Bye: Great. And finally, what do you see as kind of the ultimate potential of virtual reality, and what it might be able to enable? Oh, wow.
[00:22:57.809] Andy Beall: OK. You know, I sometimes quote Andy Van Dam, a computer scientist, who says, virtual reality is not visual reality. So in many ways, at the very least, we have four other senses that I think are really undertapped. Acoustic is the next easiest one to go after, but our other senses Who knows what will happen if we have a good haptic stimulator, olfactory and taste. But the other is VR doesn't need to just simulate reality. And I think we get stuck on that. We try and make the more and more photorealistic render, the more and more physics-based truism of some sort of a situation in real life, rather than trying to explore what's more impossible. And it's kind of the impossible, which is some of the ways that goes back to my roots, that I use VR as a way to test situations that physically could never occur in the real world, And yet, when you take these scientific questions and you put them in situations, you create a stimulus that's physically impossible. You start to tweak and probe the mind in ways that really, to me, VR had some of the greatest possibilities originally. And I think we can still see that. And so, to me, it's just, you know, a comment would be to not get stuck in just the reality of VR.
[00:24:13.653] Kent Bye: Awesome. Is there anything else that's left unsaid that you'd like to say?
[00:24:17.676] Andy Beall: You know, another comment I had when I first tried the Oculus and had some of those aha experiences myself were having an immediate sort of elation and also immediate sort of fear of like, I've got two young kids and this could be the addiction of this generation. You know, honestly, not worried about that. I think there's more constructive, positive possibilities with this. And there's always going to be that fear of, is this going to be the media that is the downfall? I don't see it as anywhere of that sort.
[00:24:45.537] Kent Bye: Awesome. Well, thank you so much. All right. You're welcome. Thank you. So that was Andy Beal. He's the CEO of WorldViz, and he's been working in virtual reality since 1992. So I have a number of different takeaways about this interview is that first of all, that was some pretty amazing stories to hear his reaction to the Oculus Rift. And it's something that I didn't quite realize the concept of vexation was something that was so elusive. I just thought it was something that had been all pervasive within VR, but it is something that is relatively new. And I think one of the most striking things about that to me, I think is that it's still kind of unknown as to all the different factors that is playing into this feeling of vexation which is essentially the feeling that you're moving but you're not moving and so there's a little bit of a disconnect there which is kind of like a optical illusion that you are physically moving but you're not actually moving Also that there's a difference between the vertical and horizontal vection. And I don't think I've ever necessarily experienced the vertical vection. I've certainly experienced some vection within VR that started to make me feel a little bit motion sick. Anytime you do like barrel rolls, like he said, or just going through roller coasters. I think a lot of people kind of experienced that. But the thing that was really striking to me is that his kind of theory that eviction has to do a lot with your belief state. So the belief that you're actually in a virtual world and that you feel like you're moving but you're actually not. And I think that belief state, he thinks, is kind of tied to the fact that using this HMD with the Oculus Rift, because it was so lightweight and just using a mobile phone and It just had a different form factor from a lot of the other HMDs, which in the past have been a lot more heavy, and you can definitely feel the impact of moving your head around. Here, just being able to move your head around, I think, was able to have that low inertial impedance, which just helped trick the mind into believing that you were actually in that virtual world. And so it's also interesting to hear a bit more about his perspective, the history of VR and the different applications that they were being used for, mostly in academia. But just a slight comment on that is just that historically, I know that the military has been a huge funder of a lot of research within VR. And it's entirely likely that some of the academic institutions and places that WorldVis has been collaborating with were in turn doing research that was for the military context and training. So it sounds like WorldViz is doing some of their own custom tracking solutions. And so it was really, really fascinating to hear Andy talk about being in VR and running around, jumping over virtual hurdles, like literally just running around in these huge spaces. I think it's probably one of the first times that I've heard somebody actually doing that. I know that there's been some other people that have been experimenting with different VR backpacks and whatnot. Just to kind of roam around within a very large space, I think that there is going to be some people that want to do that and do more kind of virtual sports or laser tag or stuff like that. So I think with the advent of either VR backpacks or some wireless VR, it's going to be a lot easier to do that. So I've never had a chance to actually experience the vertical shower curtain vection illusion, where he says that there's no vestibular cue conflict within the vertical plane, but If you stare into a drum and the axis of rotation is in the horizontal plane, then your gravity sensors or your down sensors will be triggered in a way that triggers motion sickness. So I would love to see some people actually do that within a VR gathering at some point just to experience it. I don't necessarily have a shower curtain that fits the specs or have it in a circle in that way. But I think it would be fun just to kind of experience it and give people a visceral experience of what that actually feels like. So if anybody feels inspired to bring a shower curtain to the next VR gathering, I'd love to try it out. So that's all that I have for today. I just wanted to thank you for listening to the Voices of VR podcast. And if you'd like to support the podcast, then spread the word, tell your friends, and become a donor at patreon.com slash Voices of VR.
