I recently traveled to southern France to cover the biggest gathering of virtual reality academics in the world, the IEEE VR & 3DUI conferences. I was able to record over 15 hours worth of interviews and talk to over 50 attendees, which was a little over 10% of the 520 attendees.
In this podcast and video, I give a brief overview of some of the highlights of the coverage that I’ll be releasing over the next 3-4 months. The video includes photos of the more than 100 academic posters that were shown as a part of the IEEE VR and 3DUI conferences.
It’s worth noting that lack of coverage coming out of the IEEE VR conference last year was part of the reason why I started the Voices of VR podcast in the first place. I celebrated my 100th podcast interview with an interview with Sébastien Kuntz, and gave a bit of backstory that’s worth repeating again:
I first discovered Sébastien’s work during the IEEE VR conference last year because he was tweeting about different presentations talking about the academic community’s response to the Facebook acquisition. Here’s a couple of examples of his tweets that captivated my attention:
#IEEEVR S. Ellis: the first HMD was invented by Galileo ! pic.twitter.com/eAO1tk6y5v
— Sebastien 'Cb' Kuntz (@SebKuntz) March 31, 2014
#IEEEVR Why did Oculus make this 2B$ breakthrough ? pic.twitter.com/NH1DpkolhS
— Sebastien 'Cb' Kuntz (@SebKuntz) March 31, 2014
I wanted to hear more from Sébastien and attendees at IEEE VR, but there weren’t any consumer VR publications covering what was happening in academia or with VR researchers. In fact, there was hardly any coverage from any publication of last year’s IEEE VR conference beyond tweets from attendees, with the most prolific being the ones from Sébastien.
Because of this lack of coverage, I decided to start my own podcast. I reached out to interview a couple of other attendees of the IEEE VR conference including Eric Hodgson and Jason Jerald. I also really wanted to hear more from Oliver “Doc_Ok” Kreylos who was a respected commenter on the /r/oculus subreddit, and also happened to be working in VR within an academic context.
So with that, I hope that you enjoy my exclusive coverage of the IEEE VR conference over the next 3-6 months.
I’ll also be attending the SVVRCon conference on May 18th and 19th, and I start to mix that into the IEEE VR coverage as well.
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, this is Kent Bye of The Voices of VR Podcast, and I wanted to give a brief little overview of my trip to Southern France to cover the IEEE VR Conference, which is the biggest gathering of virtual reality academics and industry experts from around the world. So I've been to a number of different consumer VR conferences and this is the first kind of old-school VR veterans you could say gathering that I went to and there's a distinct difference between the energy of consumer VR which is just filled with vibrancy and passion and The IEEE VR is much more cautious and very pragmatic in terms of their approach to virtual reality. These are people that have been involved in VR for, you know, 15, 20, 25, and up to like 45 years. So this is a group of people that are clearly compelled to be involved in VR, but that initial enthusiasm and passion is a little bit more subdued. And the other big difference between the consumer VR and this more academic industry approach is that the mindset is pretty different in terms of the goals and intentions and motivations. With the academics, they tend to want to make very small incremental advancements and to add a intellectual contribution, and they actually put a number to it. They said they had to make a 30% intellectual contribution, which basically means that they have to add at least 30% new information or adding to this body of knowledge. And so for consumer VR, it's much more about tying together all of these little incremental insights and to put together a cohesive experience and to have more of an integration and a holistic approach to it. And so In some ways, there was some debate last year at the IEEE conference in terms of why was it that the consumer VR was able to come out with something like the Oculus Rift and have this big breakthrough with it being purchased by Facebook for $2 billion. Why didn't something like that come from the academic community? And there was actually a lot of things that have been, you know, leading up to the Oculus Rift that I think that was kind of tied into the story of Palmer Luckey and his time there and Mark Bolles' lab there at USC ICT. But overall, I'd say that the intentions and motivations are a little bit different. So, That all said, what were some of the big takeaways for me? Well, I've never covered a conference like this where you are handed this USB stick that is just filled with nearly four gigabytes of information, and that's 216 papers ranging from long papers, short papers, posters, demo write-ups, 3D UI, conference proceedings, and then workshops that were held. The week was basically split into kind of two sections. The first two days were a lot about 3D user interfaces, and the last three days were all about the IEEE VR lectures and posters. It's kind of broken up into like long papers or more extended papers that are really contributing new information to the field of VR. And then there's the short papers, which is still contributions, but maybe not as fully formed as the long papers. And then if it's something that's more of an exploratory idea or not a full study, or a lot of the graduate students, you know, PhD students would have posters talking about some early explorations and kind of like early hypotheses that are being tested. And I think about the IEEE community is that they're very much about asking a lot of questions. And so a lot of ways, they'll both be pushing the field of VR forward, but they're also trying to bring up a lot of these open research questions. So you'll often in the process of having a conversation with these people, you'll ask them a question, they'll be like, well, you know, I suspect that's this, but at the same time, we don't really have the data. And so that's an interesting research question that is worth exploring. And so A lot of these types of conversations when people are there are really kind of like trying to define what the next sort of research that they're going to be doing. One thing that Sebastian Kuhn said is that, you know, consumer VR is a lot about being what's happening right now and what's happening at the IEEE VR is looking at things in terms of five to ten years down the road. So, some of the highlights I'd say. One thing for sure is the Uncanny Valley discussion in terms of not only does it apply to the avatars within a virtual space, but also in almost every dimension of VR. For example, locomotion. And what I learned is that with the Uncanny Valley, as people are familiar, is that if it's a low fidelity, things are okay. Once it gets into the the middle fidelity and the range of trying to get to this high fidelity that's just a huge drop off in terms of it just starts to look creepy or just doesn't work as well. And so the takeaway is that it doesn't just apply to avatars, but also to locomotion, to sound, to haptics, to almost every dimension that you're thinking about your VR experience is that you're really making a design decision in terms of If you try to get to the highest fidelity, for example, if you try to get finger tracking, then you have to go as far as to actually trying to mimic haptic feedback when you're actually grabbing objects. In the absence of that, then it's just going to not feel as good. And so maybe it's better to do something with a low fidelity where you're holding a controller that maybe gives some lower level haptic feedback. The other thing about haptics is that when you get into this thinking about a generalized haptic solution, that's like so far down the road that people aren't even seriously considering that. And so either you do something that's extremely high fidelity. So for example, they have these machines that ranged in cost about 85,000 euros to be able to very specifically mimic the force distribution of trying to mimic giving some sort of shot to somebody into their organs and mimicking that force distribution as you're pushing down, or with space agencies using these haptic feedbacks to do training simulations, it's something that's very specific to the use case. And so a lot of the haptic solutions tend to be rigid body simulations to a very specific use case. So that in mind, you start to think about like what problem you're trying to solve in VR. And I'd say that this group of people are definitely trying to solve a lot of problems. What else? So I think some of the more fascinating research that's happening there is the work of Mel Slater, and he's been doing work about presence in terms of identifying the place illusion and the plausibility illusion. And there was a poster there actually of a PhD student who is actually trying to empirically measure these components of presence. And one of the things that he said is that it really depends on the sense of immersion that you're having combined with a sense of cohesiveness. And so the immersion is feeling like you're in another place, and the cohesion is like this everything makes sense and it's a logic that the world is adhering to it's believable and that there's a sense that you can actually interact with it and now those interactions are kind of behaving in a way that also feel real and That's the component of presence that if it's broken then it's hard to recover from because it's almost like a house of cards that once it's broken then it's hard to recover from but Also, what they found is that, you know, you really have to have those two components of immersiveness and the cohesion in order to really get that sense of presence. And in the absence of one or the other, then there wasn't really as high of presence scores. And so those are really kind of being proven out to be the two key components of presence. And so some of the work that Mel Slater is doing in terms of virtual body ownership illusions, one of the things that he said during his lecture is that when you look down, you pretty much 100% of the time of your entire life, when you look down, you see your body, you're like, okay, yep, that's me, that's my body. There's never been a time when you look down and be like, nope, that's not me. And so when you're in VR, and you look down, you see your body, your mind is just kind of hardwired to accept it. And it's this weird kind of like disconnect between your rational mind and kind of maybe knowing you're in a VR space, but there's just some part of your very low-level subconscious perception that just believes that you're in this other body. And so you can start to do all sorts of interesting things. So they're really interested in exploring like what's it mean to kind of change the gender or the race to somebody in VR and to see what kind of lasting impacts that may have on your sense of self. So they do different word association tests and essentially find that there is some connection between having an impact of your own sense of racial bias when you are embodying the avatars of people of different color than yourself. And so lots of really interesting studies in that that are still trying to determine kind of long-term effects of that. But in the short term, it's pretty interesting and, you know, brings up a lot of ethical questions in terms of what's it mean to be able to have these different experiences in VR where it may be impacting your sense of self and what you're identifying with. And so what type of experiences are you giving someone in VR and what kind of long lasting impacts may that have? So there's a lot of ethics standards that academic VR researchers have had to abide by for many years. And in terms of consumer VR, that's basically all out the window. It's the wild west in terms of, you know, there's no limit that anybody's sort of overwatching you and saying, nope, you got to stop and take a break now. And, you know, so it's really unresearched in terms of this consumer VR and in terms of people having these more lengthy experiences and what the impact of that might be. But I think overall, there's a lot of excitement from academic VR community in terms of what's happening in consumer VR. So one of the really interesting things about the conference was that if you look at the sources, you know, the citation of these different papers are ranging everything from psychology to neuroscience to SIGGRAPH to robotics to artificial intelligence. And it's really quite a broad range of disciplines that are being integrated with VR. And so that's one of the things that I find really interesting is that you have people who are collaborating with neuroscience experts and they're bringing those insights and putting them and applying them to VR and so the principles of neuroplasticity are really interesting to me in terms of let's say for example you have a stroke and you can't move your right arm but yet When you go into VR, you can have this experience where you can see your right arm moving and that starts to actually physically rewire your brain to help you actually physically move your arm again. And so VR has the capability to kind of inject these input signals into our perceptions that start to rewire our brains in a way that can actually accelerate our ability to heal. And that's being applied to both stroke rehabilitation, but also people who have prosthetics. And so you get the sense of wearing this thing in your arm and then going into VR environments and kind of starting to see what it would look like to start to have the capability to use a limb that you may have lost. So, lots of really interesting things in terms of medical applications. Lots of stuff in terms of training and looking at what are the different ways to optimize the learning process, adding other capabilities in terms of new ways of locomoting through virtual reality spaces. There was a lot of interesting little workshops that were happening. There was a whole sort of sub-branch of, you know, molecular science visualizations. So how are molecular sciences using the ability of having immersive environments to be able to visualize these really complicated 3D molecular interactions and to do these simulations and to add sound to them and to Just hear about what type of things that these scientists are being able to do within VR and people who are coming from this molecular science background and coming in and using these technologies to be able to kind of push their own science forward in terms of like getting a better understanding as to what's happening with these interactions. You know, I think that one thing that's really interesting to me is just the history of virtual reality and just being able to talk to people that have been involved in VR for 45 years. So like Henry Fuchs, who's just a few years after the, sort of dynamically said, track down Ivan Sutherland and, you know, had been interested in VR for a long, long time. And just to hear from his perspective in terms of like, what was actually happening back in 1970 with VR and, hear how the flight simulations of the airlines, you know, were a big part in terms of funding a lot of this, you know, need to improve what at that point was just essentially like physically building models with like tiny cameras in them and then having a way to kind of like give that to a pilot. And, you know, if they accidentally crash into a tower, then that camera rig would actually physically crash into the model of the tower that they would have to go back and then rebuild. It was just a really not ideal way of doing a flight simulation. With computer graphics, they just saw the potential and really started to fund a lot of the progress of VR since the early days, as well as, of course, a lot of the military training applications that have always been funded. But really in the 90s things really exploded in terms of expanding out to other Industries in terms of the automobile industry and Germany actually seems to be a really big part of the history of VR in the 90s and up until today in terms of Really, the early adopters and innovators of VR have been really sustaining the field for many, many years. A lot of people think that VR died, but it's been alive and well in this community for many, many years, in large part because of the economics of the cost of let's say building a prototype car which could range from half a million to a million dollars and you know you spend a million dollars on a lot of really high-end VR equipment and you're able to produce an infinite amount of models that you can actually physically sit in and get a really strong sense of the ergonomic nature of these cars and so it's convinced me that virtuality is not gonna ever really go away because it's just so compelling for these companies, especially when you think about architectural visualization. More and more, the architecture firms are starting to find the compelling nature of being able to actually use VR to explore their designs before they actually build them. So one of the threads that was happening there was doing collaboration within VR and one of the papers that won the 3D UI poster contest was someone who was looking at how can you kind of mediate social interactions within a virtual reality space to kind of contradict the tendency for somebody to kind of dominate the conversation. So if you're having kind of an asymmetrical interactions within a virtual reality space, then how can you counter that and give a little bit more of a balance in terms of trying to hear from everybody? And so there was some interesting ideas in terms of doing a digital moderator to kind of pass the talking stick based upon an algorithm to kind of more evenly distribute the process of a conversation. So, for example, you could only talk if you had the talking stick, and you would have a little thing that would buzz in your hand, and if it would buzz, then you could talk, and if there was no buzz there, then you were to just kind of listen to other people. And so it's just kind of interesting to think about, you know, the process of collaboration face-to-face versus how could it change the nature of our collaboration if we have it mediated through a VR environment where there's kind of like this invisible moderator that's digitally able to kind of facilitate a process that we may not be able to do without feeling rude of interrupting people or telling people that they shouldn't be speaking. So just the idea that you can start to implement algorithms in terms of how people are actually collaborating, interacting in social groups was something that was kind of like an interesting insight in terms of one of the open problems that are out there in terms of collaboration into VR. So again, I guess in conclusion, I'd say that VR has been a part of this community for many, many years, and that they're really the innovators and early adopters. I guess a lot of people that are doing VR now in the consumer VR kind of see themselves in that same type of innovator and early adopter role. But in the larger context and history of VR, there's been people that have been doing this for a long, long time. And we're really at the point of going into the mainstream and crossing that chasm, something that's been seated there for many, many years, going into more of a consumer market into the mainstream. And so that's where we're at in terms of the larger trajectory of the field of VR. I think that there's a lot of amazing insights that they have in terms of things that are still applicable. There's this kind of debate and question as to whether or not some of the research that's happening in the VR community is still valid because there's all these huge breakthroughs in technology. And if they're operating on sort of old technology, then are the insights still applicable? And there are some legitimate questions around that. But yet, at the same time, we're still human beings, and there's still very fundamental capabilities of our psychology and our physiology that are pretty consistent. And I also wouldn't underestimate some of the systems in terms of the tracking technologies. Some of these things are very sophisticated, even more sophisticated than high resolution than a lot of things that are out there right now. I did think that there's a wealth of knowledge and information. And I did over 15 hours of interviews talking to over 50 people and right around 11% of all the different attendees of the entire conference. And each person that was there was really staggering in terms of having like 520 attendees. I'd say at least three quarters of those people had some sort of talk or paper or poster or demo. You know, just about everybody there you could, you know, pull aside in the hallway and start talking and asking them questions and get their insights into VR and they would have something, you know, very sophisticated to kind of tell you. And so you get the sense that there's a lot of Very smart people that are working on this for a long, long time and just a lot of really exciting stuff. I'm really excited to share out over 15 hours worth of interviews over the next three to four months. And so, yeah, look for that on the Voices of VR podcast. And yeah, thanks for listening in and kind of giving a sneak peek of some of my insights from the IEEE VR. Thanks for listening.
