Professor Eric Hodgson explores some of the psychological impacts of virtual reality when it comes to spatial perception & memory, and effects like change blindness. He also talks about how to walk in infinite VR spaces within limited physical spaces with a technique called redirected walking. Finally, he talks about insights from IEEE VR community on this consumer VR revolution, compares the split between the old VR community to the new VR community, 3D user interfaces, data visualizations, and some examples for how corporations and the military are using VR.
Eric Hodgson is the director of the Smale Visualization Center & also works with the Armstrong Institute for Interactive Media Studies at the Miami University of Ohio. He runs the HIVE (huge immersive virtual environment), which is a gym-sized VR Lab. For more information on Eric, here’s some of his VR research publications.
Topics
- 0:00 Intro
- 1:04 Redirected Walking
- 3:50 Factors that go into spatial perception & visual dominance
- 5:26 Why was the HIVE lab built?
- 8:10 How our spatial memory works
- 10:49 Change blindness for changes that are out of the field of view
- 12:12 Navigating geometrically impossible spaces & how we mentally fill in gaps
- 13:29 Movement speeds in virtual environments – natural vs. VR movement speeds
- 14:38 Why VR movement speeds are 2x normal walking speeds. Field-of-view?
- 15:42 IEEEVR community perspective on the rise of consumer VR
- 17:20 The divide between Old VR vs New VR communities
- 19:03 Future of Professional VR companies & high-end VR markets
- 20:41 How the military is using VR
- 22:42 How corporations are using VR
- 25:04 Why now? The timing of this VR revolution. Previous low-end VR headsets
- 27:01 Cutting-edge demos and prototypes at IEEE VR conference
- 30:03 3DUI conference, and user interface research in VR environments
- 30:58 Highlights of the future of telepresence keynote from the IEEE VR conference
- 34:03 Where VR is headed within five years & what gets you excited about what’s to come
- 35:41 Data visualization within 3D VR environments
Links:
- “Redirected Walking – Playing with your perception’s limits” blog post with a great overview of the technique by Sébastien “Cb” Kuntz
- Video of Reverend Kyle visiting Eric’s VR lab and walking in the virtual grocery store.
- Comparing the four approaches to generalized redirected walking
- Selective attention test
- IEEE VR 2014 Conference
- Older VR companies that a lot of people within the consumer VR world haven’t heard of: Vicon, Polhemus, Haption, Advanced Realtime Tracking (ART), Worldviz, Sensics, NVIS, and CyberGlove Systems.
- Slide from Henry Fuchs’ IEEE VR keynote on Why Oculus make this $2 Billion dollar breakthrough?
Music: “Fatality” by Tigoolio
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:11.916] Eric Hodgson: I'm Eric Hodgson. I am a professor at Miami University in Ohio, and I direct the Smale Interactive Visualization Center. And I also help run the Hive, which is a huge immersive virtual environment. It's a gym-sized VR lab where we can walk around kind of wear the VR system and walk around over a pretty large tracking area. And I spend most of my time either doing research on spatial perception and navigation and kind of usability of VR and teaching in the Interactive Media Studies program. You know, students from all over, different majors across the university who want to learn 3D modeling or learn how to use VR tech for, you know, marketing or architecture, engineering, fine arts, kind of runs the whole gamut.
[00:01:02.742] Kent Bye: Great. And it sounds like there's one technique that you've been researching, which is a redirected walking. Maybe you could tell me a bit about like, what is redirected walking and why did it come about?
[00:01:13.729] Eric Hodgson: So redirected walking is essentially tricking people into walking in circles without them realizing it. So they can just not run out of space in the physical environment. So there's this problem of, you know, you put on your headsets and You can see this beautiful virtual world, but you have no idea where you are in the actual physical world. And then you start walking, and you walk into a wall, and you break your headset, and you hurt yourself. And it's just a bad situation. So it's kind of like the old Star Trek holodeck for people who are old enough to remember the Star Trek holodeck, where they go into this tiny little room, and the walls change into some training environment. And then they can seemingly walk for miles and never hit the edge of the room. So it's that kind of technique. Essentially, we just kind of tweak the virtual environment as people are going through it. So a lot of the research we've done lately has been in a virtual grocery store. So for example, as people might walk down an aisle, it looks like they're moving in a straight line, but we can take the aisle and kind of slowly rotate the whole store in one direction or the other. And it's slow enough that people don't notice that it's happening, but to physically get to the end of the aisle, they have to kind of walk along a curved arc. And you can end up doubling back and heading back the way you came. And you can do additional things like amplifying turns. So if somebody stops to look around, or if they get to the end of an aisle and try and make a 90 degree turn to the right, you can multiply the optic flow. So even doubling it, so they look like they turn 90 degrees, but they actually turn 180. And again, just kind of go back the way they came. So you do all these little tricks to extend the amount of space people have to work with. And we really got into the redirected walking research out of necessity. We were studying, my colleagues and I, studying navigation. And it was always kind of hard to study room-sized navigation. You have the typical VR setup where you have a small lab room and tracking cameras on the wall. And we'd ask people to navigate. It's like, OK, take two steps forward. Now stop. Turn. Take two more steps. Now stop. You're going to hit the wall. Point back to where you started. it didn't really feel like real navigation. And so we're always looking for ways to navigate in larger and larger spaces. So, you know, that's what prompted us to put together this gym size hive, the lab, and it's like, okay, now we can study navigation in really big rooms, but we're still kind of limited. So we got into the redirected walking research to let people walk over, you know, city size virtual environments and, you know, just really large spaces.
[00:03:49.265] Kent Bye: So what it sounds like is that the visual field is really dominating any sort of internal mechanism in your body of determining what degree you're turning. Is that true? Or maybe you could talk a bit about how you're able to achieve this with psychological tricks in some ways.
[00:04:05.353] Eric Hodgson: Yeah. You know, our sense of space is really a combination of a lot of different sensory systems, you know, 3D positional audio. You know, you can hear things coming from different directions. You get a lot of information from your inner ear of your balance and being upright and acceleration as you're moving you get a lot of information from your body you know your muscles and joints and and even kind of internal signals so you know if I'm trying to walk in a certain direction or at a certain speed like I know that we call that efferent copy where you kind of know what your intentions are and where you're supposed to end up so it's a very multi-sensory kind of experience and with the body-based senses you know your muscles and your inner ear there's a a little bit of slop to it. So, you know, if you close your eyes and you try and walk in a straight line, you know, more than likely you're going to veer a little bit one way or the other, even though it feels like you're moving straight. So that's kind of the thing we're taking advantage of. In the real world, you know, you use your vision to correct that little bit of slop and hone in on things. You know, if you lose sight of something for a couple seconds, it's fine because you still generally move in the right direction, the right distance, but then you visually hone in on it to make sure you're accurate. So we can use that visually honing in to make it feel like your body is moving in the right direction, then visually adjust and adjust and adjust.
[00:05:26.375] Kent Bye: And maybe you could tell me more about what is the context of which the Hive environment was being developed? What's the application of such a huge virtual environment? Where's the funding come from? And how are people applying this type of technology?
[00:05:41.567] Eric Hodgson: It was really built as a research tool, and specifically for the navigation research. It's always been a collaboration between myself and a couple other folks at Miami, some in the psychology department, some in the computer science department, and now with the interactive media studies folks. Just a tool that would let us look at visual perception and spatial perception. And for the computer scientists, it was a good way to study motion tracking technologies. you know, look at developing new motion trackers and algorithms for the redirected walking, steering and, you know, human computer interface issues and that kind of thing. But it's one of those things where it kind of lives in the lab and it hasn't gotten a lot of legs commercially. But, you know, we've had some good conversations with folks and seen ways that you could use this kind of thing. You know, gaming, I think it'd be really interesting to do, you know, outdoor competitive you know, virtual environments, there's a lot of safety issues we'd have to work out. But, you know, a lot of games right now, the movement speed is very fixed, the coordination is fixed. You know, it's more about how fast can you push the buttons than anything else. But you can imagine an actual game mechanic being how fast can I physically move, or how efficient of a route can I plot, or how coordinated am I, you know, to physically do things. And I always thought that would be very interesting. And we've seen things on the industrial side, too. I remember a couple years ago, I was talking to someone, they had this problem where they're training repair technicians in a very large oil refinery, where basically you have almost a square mile that's just a maze of pipes. And they had developed a virtual simulation in a cave where the repair technician would have to navigate through this you know, mile-long building finds the right pipes and then make a repair, diagnose a problem. And they always thought it was really unrealistic that in real life, you know, they're carrying a heavy bag of tools and physically walking for a mile to get where they're going. So they're under a lot of physical fatigue and stress. And in the training environment, they're just pushing a joystick and they just fly right through the factory, you know, with a couple of ounces in their hands. So we had looked at maybe doing a large-scale walking, physically carrying a load, and making it much more realistic to the type of environment that they have to do their real repairs.
[00:08:13.148] Kent Bye: And it sounds like you've also done a lot of research into, like, spatial memory, and maybe you could talk a bit about how does our spatial memory work? I know that, you know, I've heard that remembering physical spaces is one thing that our minds are really great at, and I'm not quite sure why we remember physical spaces so distinctly, but I know that there's different memory techniques where you could start to memorize a list of things based upon location, and that's something that the Greeks used called loci. So I'm just curious, like, how does our spatial memory work and why is it so potent?
[00:08:48.095] Eric Hodgson: Yeah, I mean, you know, space is important to everything we do. We have to be able to get around, we have to be able to find where our food is and where our home is and our work, you know, it's kind of inherent in everything. And we have a lot of biology and physiology that's kind of hardwired to do spatial processing and spatial memory and spatial navigation. You know, you mentioned the the memory mnemonics, the method of loci where people kind of leverage that spatial thinking to almost use it as hooks to hang on. I think it was originally a theater technique where I can remember my lines and kind of look around the theater and remember what I'm supposed to say when and where. I mean, the field of spatial cognition, spatial memory is really pretty deep and very interesting. The more I get into it, the more you find that, you know, really people are just very flexible and can use a lot of sources of information to stay oriented and to get reoriented when they're lost and to find their way around. There's a lot of debates on how internal memories are represented, whether it's polar coordinates, you know, distances and directions versus more, you know, Euclidean coordinates, north and south and, you know, left, right, forward, backward. And it seems like people can do, you know, both in different situations and basically take advantage of any information that's out there to keep themselves oriented. It's also been interesting coming to the VR space because you see a lot of really kind of impoverished situations, impoverished information from a sensory perspective. You know, like the little mini maps that are in games, it's kind of a crutch, right? Because visually I'm moving through this really expansive environment, but physically I'm sitting still and that causes a lot of conflict. And, you know, if you take that mini map away, people get lost really quickly and have trouble finding objectives and getting through levels. So you almost have to, give them a map of how to get from point A to point B or they can't do it. And I'm really excited about VR because you can start reintroducing a lot of the sensory information that we've lost in traditional video gaming.
[00:10:50.060] Kent Bye: Yeah, I'm really curious about some of the other ways that our brain can be tricked in VR spaces. I know I've seen one thing where you walk into a room and you kind of switch the door position so that when you come out, you may not even notice. So I'm just curious about some of the other things that are so subtle that our mind doesn't really pick up on, but little tricks you could do in virtual reality to use that to your game design advantage.
[00:11:16.331] Eric Hodgson: Right. So yeah, that's an effect called change blindness, where you can make a very large change to the environment as long as it's out of the field of view at the moment. And it actually works really well in real life, too. There's a lot of famous examples of you have people watching a video of a basketball game and tell them to count the number of passes. And they're focused on the ball, and they're watching the movement of the ball and counting the passes. And then they get to the end of the video, and you ask them, hey, did you notice the giant furry gorilla dancing across the basketball court? And they say, what are you talking about? There was no giant furry gorilla. And then you roll back the tape, and there's actually a giant furry gorilla running around in the court dancing during the basketball game. And they don't notice because they're so focused on the ball. And yeah, you can do this in VR. You can do it in games where they might remember that there's a door behind them, but they don't remember which wall it's on. So you can flip it from one to the other and make them exit the room a different direction than they came in.
[00:12:12.394] Kent Bye: Have you applied any of those techniques or any other psychological tricks into the virtual reality space?
[00:12:18.603] Eric Hodgson: We did some research a couple years ago. It was a lot of fun on geometrically impossible spaces. So we had people navigating a series of hallways. It's a little bit like trying to walk around in an M.C. Escher painting, where locally, if you look at just the area right around where you're in, it all makes sense. But if you take a step back and look at the global arrangement of the space, it's just all warped and twisted, and it doesn't really work. So people would go around these series of hallways, and essentially, they'd complete three sides of a square and end up back where they started. So it was like there was a 90 degree turn and a segment of the environment that was just missing. And we had people wandering around these hallways and trying to find things and they'd go a couple circuits in each direction and at the end we'd have them try and draw a map of where they went. And it was always really interesting to try and see how people would mentally fill in the gaps in the space and add extra 90 degree angles to kind of bend them in and make them fit. So I mean people are really pretty tolerant of you know non-vertical environments and impossible environments as long as everything kind of locally makes sense while they're in it. And another thing that really stands out I think has gotten a lot of use is movement speed in virtual environments and in games. There's an interesting talk at the IEEE VR conference last month on walking in place techniques. So if you don't have something like an Omni or a virtualizer to run in place and have these treadmills under you, you can do walking in place and stepping and have the environment move. And the gist of the research was, how high do we have to crank up the movement to where people who are walking in place feel like they're walking in the virtual environment? And it turned out. it was at least, like, double natural walking speed. Like, if you give them actual, you know, what is it, like, three meters a second is kind of normal walking speed. If you move the camera at three meters per second, it's, like, laughably slow. But if you double it, you know, six meters a second, seven meters a second, people are like, okay, this looks natural, this is right. And, you know, if you think about it, that's, like, almost sprinting speed. And you find that in a lot of video games where the slow walking speed is actually, like, Usain Bolt sprinting down the 100 meter track.
[00:14:38.849] Kent Bye: Why do you think that is? I mean, why is it that in virtual spaces, you have to move the camera at twice the speed of how you normally walk?
[00:14:47.695] Eric Hodgson: I'm not sure that's really well understood. Part of it could be the field of view, where, you know, the speed at which things kind of move at the edge of vision is different when the field of view is narrower. There's been a long history of documenting how people underestimate distances in virtual spaces, sometimes by as much as a factor of half. So if they think they're moving half as far, you need to move them twice as fast. It's not really well understood, but it is pretty well documented that you have to move people faster than normal through these virtual environments. And that's actually one thing I like about doing stuff in the Hive where people are physically walking, because we really don't run into that. We don't have to double the movement speed Because people get such good feedback on how their body's moving and how fast their body's moving, the optic flow matches really well. And they can walk at a one-to-one scale.
[00:15:43.307] Kent Bye: Yeah, I'm curious, since you are in that IEEE VR community, and over the last couple of years with the Oculus Rift, there's been this whole renaissance of virtual reality. And I'm just curious, from your perspective, what you've seen has changed.
[00:15:58.597] Eric Hodgson: Yeah, I think this year, It was a really interesting conference because it was kind of, I think it was like a week after Facebook had purchased Oculus and, you know, everybody was kind of taking a little bit of back and, you know, stepping back and thinking about the big picture. And, you know, Henry Fuchs gave a very nice keynote on, it's kind of the responsibility of the people in this room to make sure that the VR revolution doesn't fail this time. Because, you know, it's been about every 10 years that consumer VR will be like the big thing and then it'll kind of die off. And this is probably the most momentum that it's had and probably the best chance of success. And it's kind of, we need to get out of the labs. We need to get this stuff into the hands of the public and not just be kind of sitting in our ivory towers doing our own thing and not worrying about it. It is, I think it was a really good conference and people were really contemplating, you know, I think one of the big questions that came up was, Take your system like the Hive we have at Miami or some of the other systems they're developing at different labs around the country and imagine what it would be like if you had a million users and what would you have to fix in the next six months if you were launching and you had a million users and stop worrying about tiny little issues and let's make sure we do this thing right and make an impact. One of the things that's kind of struck me over the last six months is there have been all these changes in the VR world, is really the growing divide between the new VR and the old VR. So that's something, you know, as I looked at the people who are exhibiting at GDC versus the people who are exhibiting at IEEE VR, and, you know, you read up a lot of these articles that are coming out online about the coming VR wave and the 3D revolution, and the people who are exhibiting their motion tracking and display systems At IEEE, you know, it's companies like Vicon, and Polyhemus, and ART, and WorldViz, and Enviz, and Haption, and, you know, it's these companies that, again, have been around for decades, and when you read the 3D press and the VR press, it's like nobody seems to know who these companies are, or even aware that they exist, and, you know, you look more at the GDC, and it's Virtuix, and Oculus, You know, it's all these new companies that are just really getting off the ground. They're really getting all the press. And, you know, I think there's this kind of struggle of some of these older companies that are going to have to change and adapt and really rethink their product lines and their markets. It's been kind of interesting, both on the economic business side of, you know, how they're going to change and on the kind of reporting side and awareness side of, you know, even throughout Reverend Kyle, we were talking And I mentioned some of these companies, like who is that? Because I consider him to be pretty up to date, he's put a lot of time and energy researching VR and thinking about it. So someone who's even kind of that tuned in to be kind of unaware of this whole industry that's out there and all these companies.
[00:19:04.011] Kent Bye: Yeah, the thing I've noticed is that there's been a distinction between, you know, calling this VR phase consumer VR and everything up to this point has been more of like academic or military or corporate VR that has been, you know, on the scale of tens of thousands of dollars. So at the consumer level, that's not feasible. So do you think that those companies are going to be able to adapt and be able to produce anything for the consumer VR market?
[00:19:31.214] Eric Hodgson: I think some of them will. It'll take some redesign, I think, and changing their pipeline. It's always a little hard when you have so much inertia of a couple decades of business practices, and suddenly they're out the window. But I also think there's still going to be those high-end markets that they can cater to, and they have better experience catering to. I know Sensex, in particular, is really looking at military-grade headsets, where you know, it's important to the military that a headset can hold up to rough handling and drops and sand and extreme temperatures and water and, you know, so they can build something that's a little more higher quality and more durable and have that. But, you know, I do think it'll be really pretty different markets where consumers aren't going to buy a, you know, even if it's 5,000 or 10,000 instead of 50,000 or 100,000, you know, it's cheaper, but it's still kind of out of the consumer price range. And the consumer manufacturers aren't really going to want to maybe put the time and effort into developing a $10,000 headset for a niche market. So you may see a complete division going forward of companies that don't really compete against each other but basically make the same product.
[00:20:42.226] Kent Bye: Yeah, I'm really curious. I don't know anything about the military applications other than, I imagine, flight simulators. But what kind of applications are the military using VR for?
[00:20:54.052] Eric Hodgson: It's really pretty diverse. You have things like dismounted infantry where you might have a squad of soldiers who are doing like a running in place navigation wearing headsets and you know a little bit of hand tracking and that kind of thing or gun tracking more likely. You have tank simulators and flight simulators and drone simulators, naval ship kind of things. I actually have seen papers on teaching young naval cadets how to navigate a ship that, you know, maybe you're in waters where there's other ships around and it takes a really long time to adjust your course and change the direction of these very large ships. So learning kind of the rules of the road out in open water and who needs to change course when and how to do this collision avoidance. Medical training simulation in the military is important. You know, they do work on kind of physical simulators where you have a mannequin that has a lot of sophisticated sensory information and sensors built in and spurt blood and that kind of thing. You also have more of the visual simulators where you can do visual medical training simulators. It really kind of runs the gamut. I know NASA, I've talked to some of the NASA folks who do a lot of 3D visualization, looking at data or getting underground information that they need to visualize or even things like building a virtual environment of the launch tower for the shuttle to try and map out, you know, where to put surveillance cameras to see different parts of the shuttle and even piping feeds from those cameras into the virtual environment so they can easily navigate and switch between the camera feeds. And I mean, just fascinating stuff out there as far as everybody I talk to, you know, it doesn't seem to matter what field they're in, they all have a use for it and can find some way to leverage this.
[00:22:42.860] Kent Bye: Yeah, I mean, I've also seen Ford doing car simulations. And what else is going on in the corporate world that you've seen?
[00:22:49.345] Eric Hodgson: I have a colleague and good friend at Boeing who does a lot of work with VR and 3D. And the one project that always fascinated me that he was telling me about was they have to design these planes down to every last nut, bolt, and wire digitally. You don't want to build a physical prototype of a plane and find a problem and have to scrap it. And, you know, at one point he was taking essentially video game characters, avatars, that were, you know, tall, short, fat, skinny, you know, different sized avatars. And, you know, he wasn't putting them in the seats, like you might think, or putting them in the cockpit to, you know, see the dimensions of the cockpit. He was actually simulating the repair technician. So he would try and cram these virtual bodies up into the crawl spaces under the fuselage to see if they would fit. So if you have a fuel filter or an air filter that you need to change every so often, you don't want to have hours of downtime on the tarmac where you have to pull off the side of the plane to get to the air filter. So you have to make sure that the crawl spaces are going to handle a decent-sized repair tech with a tool belt on. So that just always fascinated me. And even talking about getting into the AR stuff, where you can send somebody out to a plane, and they just look through their iPhone at the engine, and it'll highlight, change this part. or put part numbers or repair schedules or costs. You can overlay all kinds of information onto the plane to reduce errors and make sure they're getting the right parts. I've also seen some stuff with Disney as far as previewing their rides before they build them. So almost like a roller coaster tycoon kind of thing. Disney actually has a group that digitally builds their rides and builds the scenery. they can go in a cave or a headset and ride the rides and make sure that they don't have any gaps where you can see the beams or the rigging, or perfect the special effects, or ride it from the perspective of a five-year-old versus an adult, or change this and that, and really hone it and get it perfected before they go out and break ground.
[00:25:00.690] Kent Bye: Oh, wow. That's really fascinating. Yeah, there was one little point as to why now and it said that, you know, from the academic community, there was no real incentive in pushing VR or at least, you know, there's no guarantee that it was going to yield a dissertation paper or it was just a different currency within the academic world versus the more startup world, which I thought that was really interesting.
[00:25:25.995] Eric Hodgson: Right, right. And I think one of the big effects lately has been, you know, really the cell phone industry, you know, head-mounted displays have been around for forever. or at least 60 years worth of forever. Ian Sutherland was tinkering away in his lab building head-mounted displays in the early 60s. And there's the sensorama that came out in the late 50s, 1960-ish. So I mean, there have been people doing this kind of research for at least 60 years. But it's so expensive to custom engineer a micro display, a small screen with a high pixel density. you know, suddenly they're everywhere and that, you know, there's millions of units and you can just grab one dirt cheap and put it in a pair of ski goggles and make a headset. And the same with the motion tracking, you know, there's such sophisticated motion tracking built into all these cell phones. And again, you have millions of units. The cost per chip is just dropped through the floor. And, you know, suddenly you can build motion trackers for dirt cheap, whereas, you know, five years ago it would cost thousands of dollars. And, you know, economically the time is really right to make cheap devices that are actually worth wearing and worth using. I remember a couple years ago, I tried a headset that was under $1,000 and I was getting so sick and the visuals were horrible. I wouldn't even consider a headset that was under $10,000 because you just knew it was going to be terrible quality and they had to cut so many corners to get the price down that low and that's not really true anymore.
[00:27:02.427] Kent Bye: I'm curious about some of the other big insights or talks that you saw at the IEEE VR conference that really stuck with you.
[00:27:10.892] Eric Hodgson: One thing I love about the IEEE conference is it's really kind of before the cutting edge. This stuff that comes out at CES or GDC, it's a little more consumer ready. There's a lot of people who demo prototype systems and custom hardware. at IEEE and a lot of times it's, you know, just held together with duct tape or, you know, I've seen systems built out of cardboard and, you know, just kind of strapped together really at the conceptual stage and, you know, maybe do a little bit of beta testing or, you know, test on a few people. And it's really a very research-oriented environment. And Yvon Boger had a write-up on his blog and he talked about it's definitely on the research side of research and development. I think that's very true. And one of the best examples I can give of that is, I think it was last year, there was a pair of graduate students who were showing a system that was a scent display. Again, it was built out of cardboard and duct tape and some old computer fans that they had just strapped onto the side of a computer monitor. And they had a little Arduino controller or something that was pumping perfume into this thing with the fans. They would have a picture of a peach that was on the computer screen, and the peach would move around to different places. And they would just dial on the microcontroller up and down to change the fan speeds, and it would blow the perfume around so that the only place it smelled like peaches was right in front of the picture, and the rest of the screen couldn't smell anything. And it wasn't like they're launching a Kickstarter, or they have this commercial business plan. It was literally build out of cardboard and duct tape. It was right on the ground floor. So even a couple years ago, you know, they're showing the FOV to go project that kind of predated the Oculus, where you had literally an iPhone and a cardboard cutout to hold it. And you hold the thing up to your face and do the little split screen app on the iPhone to show the 3D visuals. So it's kind of neat to kind of see these things. And then a year or two later, they're starting to go mainstream, or they're starting to show up at GDC. As far as the research talk, I'm a little biased towards the spatial and perceptual stuff. distance estimation and usability. But it really kind of varies. I've seen some really interesting stuff on data visualization and interface design. So if you're throwing up thousands of data points into a 3D space and you want to have a hand tracker to select individual data points, how do you pick the right one and not hit the 300 that are right next to it? Or if you're picking up an apple in an apple barrel at a virtual grocery store, how do you make sure you're hitting this one and not that one? There's a good poster on a prototype design for a new improved cave. How can we do multi-projector blending on a curved cave surface? A lot of interesting conceptual work, I think.
[00:30:03.821] Kent Bye: And there's also a 3D UI conference that's happening at the same time. Were you able to attend any of that or see some of the best demos come out of that?
[00:30:13.646] Eric Hodgson: Yeah, that was one I actually had a chance to attend last year. I missed it this year. It's a very similar. conference, a little more focused again on the usability and the interface, which I think is really great. You know, it's kind of a paradigm shift when you get rid of the mouse and keyboard and, you know, a lot of the traditional user interface elements, the heads up display and get into the virtual world. I think it's going to be a really ripe area of research. But yeah, they did have a lot of demos, you know, Tactile Haptics was there showing off their controllers, Betty Moeller and her group showing off some of the user interface and virtual applications for actually in-flight entertainment and in-flight business teleconferencing and that kind of thing. It was really, really phenomenal, I thought.
[00:30:59.120] Kent Bye: Yeah, in terms of telepresence, I know there was a whole keynote on telepresence. Maybe you could tell me a bit about what was some of the main points that you took away from using virtual reality as a telepresence tool.
[00:31:10.850] Eric Hodgson: One of the things I took away is that telepresence is much harder than it sounds. It's actually very difficult to reliably overlay a virtual human, especially a moving virtual human, onto the physical world. And a lot of it has to do with the technicalities of rendering on a see-through display. And a lot of the same issues as VR, just the latency is a real killer, and video processing. And if I had a video feed of you and I was trying to feed it into a headset on my end, there's a lot of bandwidth that has to go through and a lot of processing you know, cut you out of the background and project you onto my physical environments. And so there's a lot of very difficult technical problems there. And it's one of those things that it holds a lot of promise. And, you know, five or 10 years down the road, maybe we'll see a good telepresence system. But a lot of the keynote was about kind of, we've been on the cusp for a long time, but we're still not there. And there's a lot of large hurdles. And I thought there was some interesting elements too on looking at some of the kind of the Hollywood special effects. So I forget which news network it was, but they did kind of a green screen illusion to make it look like they were doing telepresence. So they had a reporter in a remote location, and they kind of beamed in the image so it looked like she was standing in the newsroom and having a face-to-face conversation with the main anchor. And in reality, the anchor was just staring off into blank space and couldn't see her. He was just pretending that he was having a telepresence conversation. you know, she's in a green screen room with a lot of very sophisticated cameras, you know, getting beamed in and, you know, just some discussion on whether that's a service to our field or a disservice or making it look like we've made a lot more progress than we actually have or, you know, this is a thing that's possible when really it's all just smoke and mirrors and they're not actually having a telepresence conversation. So, yeah, I thought that was a very interesting discussion. A lot of the telepresence stuff I saw presented was actually based on using the Kinects. And they were doing crazy stuff like a PC can only handle 10 Kinects at the same time. So we would mount like eight in this room and two in that room all pointing, you know. So they would use an array of like 10 Kinects to capture a room from a number of different perspectives and heights. And they would track which objects are moving and which ones are stationary and like reconstruct the whole room and the people in the room.
[00:33:42.172] Kent Bye: Whoa. Yeah. I think with V2, it's going to get a lot better.
[00:33:46.456] Eric Hodgson: Yeah. That's what they're saying. Like we can't wait for V2. Yeah. I mean the, the resolution's better. The depth is better. The tracking is better. I think they can use more than 10 and they're really excited about that.
[00:34:01.184] Kent Bye: Whoa, that's great. I'm curious about your perspective of where you see the virtual reality going in like five years. What would be sort of your vision of what would be possible with all the technology and tools?
[00:34:16.247] Eric Hodgson: I think what's possible is more that we're going to do a lot of the stuff that we've been doing, but it's going to be cheaper and more accessible and more people will be doing it. So, you know, I do see it taking off in certain segments of the gaming world. I'm not sure if it's going to be ubiquitous, but things like racing simulators, flight simulators, horror games, or even just apps to kick back and relax on the beach or swim through a coral reef or float through space. I can see a lot of good gaming and entertainment applications there. But the thing that really gets me excited is the ripple effect of that. We've been working for a while trying to develop medical training simulators, or do obviously the research environments. And a lot of the teaching I do is how to use this technology, video game technology in general, to do things that aren't games, and data visualization, and all this kind of stuff. And again, that's stuff people have been doing in labs and in Fortune 500 R&D departments for decades. But it's always been so specialized and so niche, and just a handful of people doing it. The thing that excites me is there's so many new people getting into the field and so many new startups and it's suddenly accessible for much more of the population than it was five years ago.
[00:35:41.716] Kent Bye: You mentioned data visualization and I'm really curious about what you've done with data visualization and working in 3D spaces and what kind of applications you've seen where doing a data visualization in 2D space just didn't work but in 3D space it made total sense and you get all sorts of new insights.
[00:36:00.426] Eric Hodgson: Right. I haven't built a lot of them myself, although we do teach some of the techniques and do simpler versions. But it can be a very powerful technique to use the 3D space for data visualization. And sometimes that's taking a multivariate data set and plotting it in three dimensions and maybe using color as a fourth dimension or time as a fifth dimension. You can layer in different dimensions onto the data set. But I think a lot of it Two is sometimes you have data that's kind of spatially rich or, you know, data that has a location associated with it. So, for example, you know, our psychology department has a brain imaging center. And, you know, they put on these little sensors, almost like a hairnet around your head. And, you know, they're getting a lot of readings on electrical activity under the scalp. And, you know, each of those sensors has a 3D location. But when they display the data, it's either a spreadsheet of sensor one, here's the activation over time, or they'll do a 2D heat map. You lose a lot of the spatial information when you flatten it out or you just put it into a column of numbers or a sine wave of activation over time. So it's always interesting to take that and put it back into a 3D space and show a 3D brain and where it's lighting up and when. There's another really interesting example that I went to a talk about a couple years ago, where, again, it was one of these oil refineries. And they had a lot of maintenance records and repair schedules and specs and information on all these piping systems. And somebody sat down with it and took that out of notebooks and spreadsheets and everything and did a 3D recreation of the plant and color-coded all the pipes. So things that would fail. frequently were biased more towards the warm colors and things that were really reliable and didn't wear out very often were more, you know, blues and greens. And then there was another dimension of if it does fail, how bad is it going to be? And, you know, so things that were really catastrophic were just like bright red and, you know, things that didn't really matter were kind of on the other end of the spectrum. So then when you look at the 3D model of the plant, anything that was red is like, it's likely to fail, and when it goes, it's going to be bad. And what they found was all of the red stuff in the plant was in like one spot. And the people who had worked in that plant and done the repairs for 10, 20 years, that was something that they had never realized, like that they needed to treat this part of the plant with extra attention, that they need to be more careful there. All the information was there, they had it, they had actually generated a lot of the information, but It was a new revelation to them once they looked at the 3D model and actually put the data where it went. And I think that can be a very powerful technique.
[00:39:02.990] Kent Bye: Great. Well, thank you so much, Eric, for joining me today.
[00:39:05.551] Eric Hodgson: No problem.
