#363: Time Perception Research & Manipulating Time Dilation Effects in VR

bruderAt Oculus Connect 2, SVVR’s Karl Krantz told me that he accidentally spent 12 straight hours in VR and only thought that 3 hours had passed. I then started hearing a lot more time dilation stories from Owlchemy Labs’ Alex Schwartz and Devin Reimer as well as Fantastic Contraption’s Sarah Northway. These time perception underestimation anecdotes were both fascinating and really scary, and they found that there some of the likely causes were achieving the flow state, having a deep sense of presence, as was as fun the fact that ‘time flies when you’re having fun.’

University of Hamberg’s Dr. Gerd Bruder has done some research into the issue of time perception in virtual environments, and he’s discovered some fascinating results that he presented at the IEEE VR conference in March. He found that there are environmental cues — such as the movement of the sun — that can change subject’s perception of time when it’s artificially manipulated in VR. These environmental time estimation cues are called “zeitgebers”, and they are one of the many factors that impact our time perception. Some other correlating factors are cognitive load, the level of flow that a VR experience generates.

I had a chance to catch up with Gerd at the IEEE VR where he shared some of the other insights and results into why we’re vastly underestimating our time that we’re spending in VR.


Here’s the paper by Christian Schatzschneider, Gerd Bruder, & Frank Steinicke titled
“Who turned the clock? Effects of Manipulated Zeitgebers, Cognitive Load and Immersion on Time Estimation.” This research starts to explain some the factors that directly impact time dilation, but there is still a lot that we don’t know about it yet. What this research shows is that there are a lot of zeitgebers cues that can be manipulated and studied within VR environments.


For more information on this topic, then be sure to check out ResearchVR’s podcast on “Time Perception and Dilation in VR.”

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

Rough Transcript

[00:00:05.412] Kent Bye: The Voices of VR Podcast. My name is Kent Bye, and welcome to the Voices of VR podcast. When I was at Oculus Connect 2, I kept hearing from different developers about going into VR and coming out and vastly underestimating the amount of time they were in VR. For example, Carl Kranz said that he spent 12 straight hours in VR, but when he got out, he only thought that he was in there for three hours. No breaks for using the restroom or eating or drinking or anything like that. And so there's something about the immersive quality of VR, of getting into the flow, as well as something that's unique to the virtual environment that alters our time perception. And so today's interview is with Gerd Bruder, who is a professor at the Human-Computer Interaction Group at the University of Hamburg. And so he's been investigating and researching this issue of time perception and has actually done some really interesting studies on that, which we'll be talking about on today's episode. But first, a quick word from our sponsor. Today's episode is brought to you by the Virtual World Society. The Virtual World Society wants to use VR to change the world. So they are interested in bridging the gap between communities in need with researchers, with creative communities, as well with community of providers who could help deliver these VR experiences to the communities. If you're interested in getting more involved in virtual reality and want to help make a difference in the world, then sign up at virtualworldsociety.org and start to get more involved. Check out the Virtual World Society booth at the Augmented World Expo, June 1st and 2nd. So this interview with Gerd Bruder happened at the IEEE VR Academic Conference that happened in Greenville, South Carolina in March. And so Gerd is probably one of the most prolific academics in the VR research community with over 100 different publications. And so He's investigated all sorts of different areas about perception in VR. And just a quick note before we dive in is that there is this phenomena within the research VR community where they found that when you're in VR you actually underestimate the distances that you see. And so something about the virtual world is causing us to not exactly see the spatial relationship that is being represented and how we're actually perceiving it. So take that in consideration as we're talking about the space and time and speed and all these different dimensions that are really tied in together. So with that, let's go ahead and dive right in.

[00:02:33.530] Gerd Bruder: My name is Gert Bruder. I'm working at the HCI group at the University of Hamburg. And I've been in VR for around 10 years by now. So I started in reactive walking. But after I got my PhD, I kind of expanded my field of research to general motion perception, space perception, interaction techniques, etc. So nowadays I'm pretty much doing everything. In particular, time perception has been around for a while now. But actually, I haven't really been focusing on time perception in the past. It kind of came to me that after I was doing spatial perception and I was doing spatial-temporal perception, like moving through the virtual environment, that there's also this temporal aspect in there. And all of them are connected. I mean, mathematically, you can describe it in a well-defined relation. Like our speed is defined as distance divided by time. So the speed of motion, distance, and time are per definition connected. At some point I just realized that all of those areas are really interesting for research and so in the past we kind of dived a little bit more into the field of time perception as a standalone factor in virtual environments. Yes, that's currently what I'm doing but I'm really not limiting my research to this one direction.

[00:03:53.129] Kent Bye: Yeah, so just anecdotally, in the process of doing a lot of different interviews with either game developers or people who are playing these experiences, I've had a series of different people kind of bring this topic up to me. There was one person who got in and played a tower defense game and was playing for 12 straight hours without any breaks or pauses or stops. When he got out, he just thought it was three hours. He didn't mean to, and that was the thing that was a little scary. He just got so immersed in having so much fun. And then the Alchemy Labs, Devin Reimer and Alex Schwartz, were starting to do these exit surveys for people playing Job Simulator. And so after they were playing, they would ask them, well, how long did you think that you were playing? And so one person who was in there for about 50 minutes said, oh, it was probably about 10 minutes. And Devin was like, he just started laughing. And he was like, no, really, how long do you think you were in there? And they were like, no, I think it was about 10 minutes. And so they just consistently started to see this mismatch between how long people were actually in there and how long they perceived that they were in there. And it was just like a factor of 4 to 5x. And to me, when someone's engaged and having fun and doing something, time flies when you're having fun. What I'm curious is, what are the other dimensions of the virtual environment that may be altering our perception of time?

[00:05:13.344] Gerd Bruder: That's actually a good point. In the field of psychology, it's kind of a well-established result that, depending on your cognitive load, if you raise the cognitive load to quite a high level, then you will probably underestimate the duration in which you have been doing this. And the same applies to flow. If you have a computer game with a very high level of flow, which pretty much all the computer games try to achieve, then you typically get similar results. have very strong underestimation of the time they think they spent in this game. And in particular when we are thinking about virtual reality interfaces, which of course also currently go very much in the direction of gaming, then in the future we will really have a lot of problems with people putting on a head-mounted display and losing track of time because of low or just a high cognitive load if you are going into professional fields like where you're doing some architectural design or something else which raises quite a high cognitive load. So pretty much all the applications in VR at the moment, they either raise very high flow or very high cognitive load, except for some relaxation stuff or vacation stuff or something like that. But pretty much all of them, they really compress time, so to speak. And at the moment, there are a couple of things that most people, they don't really think about when they are implementing virtual environments, which, however, will probably have an effect on time estimation and time compression in particular. So one of the factors that we've been evaluating in this paper here was just the movement of the Sun, which is really like the first thing that came to our mind. Although we use the Sun for telling time in the real world, there are also a lot of other factors in the real world which used to tell time, which essentially can either lead to underestimation or overestimation if you don't present them correctly as in the real world. Like you moderate them a little bit or you change them because of game dynamics or whatever. then you will probably see an effect on time estimation as well. And so far, I must say that, yeah, really we have no idea what's really with all those factors that exist in the real world in which you can modulate. So we evaluated one of the factors, but I must say the thousands of other factors, they are also there in the virtual environment and their impact on time estimation, it's unknown. So I'm really hoping that in the future we will see more experiments, more studies, more papers being published in this field so that we really can understand what's going on there and in particular understand the factors which contribute to those effects.

[00:07:41.488] Kent Bye: Yeah, we kind of, in the process of talking to different developers, we kind of talked about flow as well as just having fun. I don't know if that's a dimension as well, like when you're having fun that you get this time compression effect. But in terms of flow and measuring it scientifically, how are they able to show that when you're in the flow, you have an effect of time compression whenever you're trying to estimate the amount of time?

[00:08:03.999] Gerd Bruder: Well, I'm not an expert on flow. It's more like in the gaming direction. But I think they have some questionnaires and some other measures that they use to measure flow and assess the effect of flow. But I'm pretty sure that it's kind of comparable to cognitive load, which we have very good studies in psychology to compare the level of cognitive flow of some tasks that you're doing, something in the virtual environment, some gaming environment or whatever. compared to a dual tasking method to some other tasks. So if the other task goes down, then you can say, OK, this is one task. It probably has a very high cognitive load, or probably flow.

[00:08:40.572] Kent Bye: Yeah. And so to measure cognitive load, then you have someone do a specific task, and then you have them do the same task while they're doing a virtual environment? Or what are some of the examples of the tasks that you're using to be able to extrapolate this cognitive load dimension?

[00:08:54.951] Gerd Bruder: So, in the past we always had a look at the field of psychology, to be honest, and we always took those tasks that are really controllable, like very high cognitive load in terms of either spatial or working memory, which pretty much are the two big dimensions of cognitive load. For that we typically use simple tasks like we're showing different letters one after the other like A, B, C, B and then each of those letters appears and then you have to compare it to the letter which came like two or three or four letters before the current letter. So you always have to memorize the last letters and compare them so it's really inducing an extremely high cognitive load in a controllable fashion. So if you want to have an even higher cognitive load then you just take the 15 previously memorized letters. The same goes for a spatial task. You can just compare spatial dimensions which you have to memorize and then compare in your mind. The idea behind a spatial task is that you can't verbalize them. So if you can't verbalize them then would probably be the same as a verbal task. But if you can't verbalize them, then it's typically a spatial task, which only requires spatial working memory. So like rotations, comparing rotations, positions, something like that. In the past, we found that those tasks, they really provide us with a controllable amount of cognitive load. So whenever we introduce such a task, we measure the errors or just the performance of our users and have a concurrent task in the virtual environment, like interacting with objects, moving through space or something like that, and we see a drop in the performance in the concurrent tasks, then we know, okay, this primary task, it has a very strong effect at this particular moment, either verbal or spatial working memory or something like that.

[00:10:34.933] Kent Bye: And so in your study that you were starting to look at this time perception What were some of the findings or conclusions that you came from what you did in your experiment?

[00:10:43.787] Gerd Bruder: Yeah, so our main conclusion is that yeah, we evaluated the movement of the Sun over the sky and we found that in Yeah, although it's a pretty subtle effect that the movement of the sun has on the scene, it still had a very strong effect on time estimation, in particular if you had no task in the virtual environment. So we had this virtual island scene where you could stay on this tropical island for a certain duration and when the sun was not moving, we saw that something apparently fell off for our participants. So they thought that they were in the virtual environment 25%, I think, longer than they actually were. this environment. So definitely there's a very strong effect of just the movement of the Sun. So just those very subtle side givers in virtual environments. So this was like one of the major results which we found here. The other result was kind of in line with the literature I would say in the field of psychology and that just raising cognitive load will have an effect on time estimation. It's not really a new result but In particular, in my talk, I try to raise a little bit the awareness of the developers in the field of virtual reality that cognitive load is not only affected by the task that you're trying to accomplish in the virtual environment, but also very much by the user interface. So in the past, we've seen very good studies showing that just by using a different user interface, you will probably end up with a different cognitive load. And since we know that cognitive load is one of the major factors affecting time perception, we now know that actually using a better user interface, a more natural user interface like 3D grasping with your own hands or something like that or just walking through space might actually reduce cognitive load and then reduce the impact of this factor on time estimation. So make it more natural, so to speak.

[00:12:25.652] Kent Bye: I see. So if people are using like an Xbox controller, then that could be a level of abstraction that is adding more cognitive load and could perhaps change someone's perception of time as they're going through a game.

[00:12:36.482] Gerd Bruder: Yeah, maybe. I mean, the literature in this field is a little bit conflicting. So game controllers, if you're proficient with a game controller, so like you've played games since your early childhood, then you're typically very good at this. So it doesn't raise so much of a cognitive load. But on the other hand, if you have a very natural user interface like walking around, touching with your hands, etc. Then again, you also are trained since early childhood on walking around and touching objects. So again, very low cognitive load. The problems we are having are mainly in between. So when you have something like a semi-natural interface where you have to steer something with a strange degree of freedom, etc. Then you typically see a very high cognitive load and then the performance goes down and then also time estimation will probably be totally off.

[00:13:21.662] Kent Bye: Yeah, the thing that I wonder is that there's a number of different experiences that have the physics all in slow motion, you know, and I just imagine that if you change the physics of how gravity works in a simulated environment, that that could change the way that people perceive time.

[00:13:38.400] Gerd Bruder: Oh yeah, definitely. Like I said at the beginning, everything is related, right? So if you're moving through space, you're like covering a distance over time. And if distance compression effect, like a distance underestimation or something like that, that we typically find in virtual environments, then it will probably have an effect on time. And if you have an effect on time, like due to cognitive load or something like that, you can expect that there might be an effect on your spatial perception as well. So I guess in the future we really need such comparative evaluation which truthfully haven't been done yet. So as far as I know no one really evaluated the different components and their interaction effects. So there's been a lot of work being done in the individual field but not really comparing the interactions. So I would wish that for the future that we could get an understanding of those interactions.

[00:14:25.529] Kent Bye: And so let me see if I understand this correctly in some of the results that you had from your experiment, which is that when you stop the movement of the sun, when people estimated the time, they had 25% longer time. So they thought they were in there for longer because the sun wasn't moving. Is that correct? Exactly.

[00:14:41.850] Gerd Bruder: That's pretty much what we found. I mean, there are applications for that, right? I mean, just think about a vacation. If you can spend like an hour with a head-mounted display on a tropical island and it feels like two hours or one and a half hour or whatever, then probably, yeah, I guess there are applications for that. On the other hand, I guess there are also ways that you can make cognitive loads work for you, or just those manipulations of those side givers, which also, I guess, can go in the other direction, that you can really think that it was shorter than it actually was. Think about a virtual classroom. You have pupils coming in in the morning and they're leaving in the afternoon. And wouldn't it be great that afterwards they just leave the classroom and think, oh, it was only one hour. It felt like one hour, but it was actually like six or eight hours or something like that. So I'm not sure how far we can push this, to be honest. I guess 25% is already quite a lot. Maybe we can go up to 50%. I really have no idea how far we can push this. But I'm excited about this possibility.

[00:15:47.075] Kent Bye: When you're in a real world that you're able to kind of look down this hallway that we're standing in here and and get a sense of it and you have a mental map in your mind as to what's happening but yet when you're in virtual reality your expectations of what is real and what's happening may be happening very dynamically and you know you may be in an environment that's changing you know moment to moment and so I could imagine that our brains have been wired to be in environments where we don't expect that the entire world around us is just going to suddenly shift. And so I'm curious what kind of impact that might have in terms of time perception.

[00:16:19.816] Gerd Bruder: Oh yeah, that's a good question. I mean, we've seen all of those change blindness studies, etc., done by Evan Zuma, etc., where you're looking somewhere and then behind you something shifts. you look around and you don't see any difference and you still think it's a coherent virtual environment. So I guess if you add something like that, like brief moments of teleportation which you can't perceive due to change blindness or some related perceptual illusion, then it will probably have an effect on time estimation as well. Like imagine you are seeing a long corridor and you want to reach the end and you walk a couple of meters and then you don't pay attention anymore and then you're at the end. then you will probably think, oh wow, I must have walked a long time. Whereas we could probably use some illusions to just do it in a couple of seconds, right? So I guess definitely there might be some interesting work being done in this direction.

[00:17:12.676] Kent Bye: And so what are some of the biggest open questions that you're looking at that's really driving forward your research?

[00:17:18.440] Gerd Bruder: Yeah, so time perception is like one quite interesting field at the moment because, well, we kind of established the first protocols in this work here and so I guess we'll stick to this research a little bit and try to come up with some novel ideas or maybe some applications in this area. I must say, for myself, I'm currently very much interested in different applications. So, basic research is nice, in particular in time perception, but at some point you want to have something that really can be applied to realistic applications. And at the moment we, of course, have the classical VR applications like architectural design or, I don't know, construction or something like that. But we also have this huge interest from the gaming community in VR. And I think from all the applications at the moment, this is probably the application that is the most promising and probably worth a lot of research being done in this field. So I'm really hoping that in the future I and other researchers here will kind of come a little bit closer to the gaming community. It's like a GDC and etc. So to bring those communities a little bit closer to each other because I really believe that we can learn a lot from each other. So, of course, they can learn from us about perceptual stuff and VR hardware and all the effects, and we can learn about flow and all the things that you need to make VR experience even that more interesting for the users. So this is kind of, at the moment, one of the directions that I'm going in.

[00:18:48.341] Kent Bye: And so what are some of the things that you think that the game development community can really learn from what's been done over the last 20 years of academic research into VR?

[00:18:56.805] Gerd Bruder: Well, they can definitely learn about things that they better shouldn't do, like building head-mounted displays with a weight of 3 kilograms or stuff like that. But I think in particular in the last years we've established quite a lot of research in the field of spatial perception, which definitely will also have applications in gaming. Like you're seeing a target or like an enemy at 10 meters and you perceive them at a distance of 5 meters, which is occurring very often in VR to be honest. then of course it will have an effect on gameplay. Although the gaming community also came up with some solutions in desktop environments like making objects bigger or smaller than they actually would be to make them feel right. In particular in our community we can give some pointers on how to do that in a 3D environment where you're looking at it from an immersive perspective.

[00:19:47.062] Kent Bye: Are you a gamer, and are you playing any VR games?

[00:19:50.204] Gerd Bruder: Well, a little bit. So I'm not a professional gamer, so at the moment my position just doesn't allow that I take too much time off work. So publishing, publishing, publishing. But yeah, I mean, I've seen a couple of the VR games that came out, and I find them really exciting. Just a very immersive view onto a gaming environment. I'm really, really looking forward to more development in this area.

[00:20:15.302] Kent Bye: So what type of experiences do you want to have in VR then?

[00:20:18.785] Gerd Bruder: Oh, there are a lot of things that I would like to do but I just can't do in the real world like visiting Florida or something like that or just walking up the Mount Everest or stuff like that which on the one side in the real world I can't do because I can't travel there and spend so much time but in a VR environment we could compress it to a couple of minutes getting there and then walking up the Mount Everest. So I would really like to see something like that. So we can really, on the press of a button, have a vacation in a very nice place in the virtual world. So after work, instead of watching a movie, just put on your head-mounted display and enjoy the evening on a tropical island.

[00:20:59.274] Kent Bye: I would really like that. And finally, what do you see as the ultimate potential of virtual reality? And what am I able to enable?

[00:21:09.240] Gerd Bruder: Wow, the ultimate possibilities of VR. Yeah, I mean, you can apply it to so many things. Pretty much all the things you can do in the real world, to be honest. I mean, consider this idea of Second Life, although it's probably not the best 3D graphics, etc. It's kind of a compelling idea that you can push everything to a virtual dimension. And in particular, if you can bring your own body into the virtual world and you can interact and talk to other people in the virtual world, then there are really not any formal limits to this idea. Whether we want to pursue all those possibilities that we have there, that's a completely different question. And to be honest, I don't think that we should take too much away from the real world, which is nice in the real world, but we should focus on the things that we can do and we are better than in the real world. Yeah, that would kind of be my ultimate goal.

[00:22:00.798] Kent Bye: What are some of the things you can do better in VR than the real world?

[00:22:04.201] Gerd Bruder: Yeah, just like I said before, like traveling through space without having to actually move through space, like press of a button and be in California, press of a button and be back in Germany. Let me give you a little bit of background. So in my studies, I often see that when people walk around in my laboratory, so I give them a natural walking interface and they can walk around for as long as they want. And I play with real-life walking so they can cover really long distances, like one, two, three kilometers. Then they do that, yes, but if you give them a game controller at the same time and give them the choice, either sit down and use a game controller or walk around in the laboratory for as long as they want, then most of them, they just tend towards a game controller, because it's much more easy for them to use, so you don't have to walk around in the real world. I mean, I guess people are lazy, so you always go the way that imposes least effort on you. So you can do that in VR. You can't do it in the real world. And I think that's actually a nice possibility, right? It doesn't only pertain to walking, but I think some other things as well, like using automatic solvers, you see something and if you can implement an algorithm, then you can have it automatically solve for you. You can do that in virtual environments as well, like you're seeing a puzzle and then you can click a button and it's solved. You can't do that in the real world, because you just don't have this programmatic control over the real world. So a couple of perspectives that I think are really interesting in VR.

[00:23:34.861] Kent Bye: Nice. And I'm just curious if you have anything else that's left unsaid that you'd like to say.

[00:23:40.565] Gerd Bruder: Just that I really hope that our communities will grow closer. So currently I have this feeling that we are still a little bit apart, but I hope that just by bringing gamers to the VR conferences and bringing VR researchers to the gaming conferences that we can grow it into one big field. So this would really be my hope and that really all the communities and the researchers can benefit from each other and in the end that all the gamers will benefit from our knowledge and the combination of our knowledge, which I think is really at the moment not that much happening and I think if we combine everything then we can make it much more compelling for the gamers as well. So I'm really hoping that we can push this forward in this direction.

[00:24:27.000] Kent Bye: Okay, well thank you so much. Okay, thank you. So that was Gerd Bruder, who's a professor at the Human-Computer Interaction Group at the University of Hamburg. And so the research paper that was discussed here was called, Who Turned the Clock? Effects of Manipulated Zeitgebers, Cognitive Load, and Immersion on Time Estimation. So the Zeitgebers, these are the indications that are in the environment such as moving the sun. And to me it's really striking that you can just freeze the sun or speed up the sun and that will actually have an impact of our time estimation within VR experiences. And so Gerd is obviously really interested in exploring what kind of practical applications this might have. But if you are in a VR environment and you want to preserve this sense of time passage, then it might actually be worth modeling the sun as it's moving through the sky so that we have some sort of indications about how much time is passing. And also, just how there is this known effect within the VR community that we underestimate the spatial relationship, we underestimate how fast we're moving, and we're also have this difference in terms of how we're perceiving time and the amount of time that's passing and so It's kind of like walking into a virtual world. We really are having this completely different space-time relationship and connection and I don't really think that we fully know all the different factors that impact our time perception. But there's something that's different about being in VR than being in the real life where we're not fully modeling it to that reality. And so there's this time perception effect. And as Gerd said, we do know from the psychology research literature that cognitive load seems to be a key impact in terms of estimating our time. If you have high cognitive load, then that's going to change how much time you perceive as passing, as well as flow. So being in the flow state, which I think a lot of VR games are creating some really interesting experiences that are creating this type of flow. So this is an interesting point where you start to look at where the gaming and the research communities are coming together and trying to share information and knowledge, because really the gaming community is really experts in terms of creating immersive experiences that really give you this sense of presence and this sense of flow. And that's not necessarily the strength of the research community. And so when they're trying to measure the impact of immersion on issues like time perception, then I think that at some point, you're going to see a lot of the different researchers pull in these highly polished Vive or Oculus Rift experiences to be able to do research into presence and time perception. So I think there's a lot of really interesting open questions still yet to really fully investigate what is actually happening with our perception of time in VR. But the bottom line seems to be to be careful and to have some sort of external indications as how much time is passing. Otherwise you could go into VR for what you feel like maybe 10 or 15 minutes and end up being two or three hours. Now, that said, Gerd did say that he doesn't know how extreme the impact of manipulating these external Zeitgebers can have in terms of your perception of time. So that's still a lot of really interesting research questions yet to be explored. So with that, I am about to hop on an airplane and go to San Francisco to go to a number of different events, including the Rothenberg Ventures Founders Day, the Experiential Technology and Neurogaming Conference, as well as Google I.O. where I'll be doing a few interviews with some Google employees there about some of their big announcements that we're expecting to see this week. And so if you'd like to follow along my journey there, you could follow me at Kent Bye, K-E-N-T, B-Y-E, on Twitter, as well as now on Snapchat. I am experimenting with the new experiential age technologies that all the cool kids are using these days, and I have to admit I feel very old and confused, but I will say that Snapchat is a perfect example of a new paradigm thinking in terms of designing a user interface that is trying to capture the authentic moment of what's happening right now and to share it in an impermanent way where people see it and they let go of it and they don't try to hang on or accumulate it. With that, I've been kind of experimenting with the Snapchat medium and hope to do some different dispatches and storytelling and trying to share my own personal experience as I'm traveling to these different places in San Francisco. So if you're interested in that, please do tune in to my experiments and to Snapchat this week, and we'll see how it goes. And if it's successful and interesting enough for me, then perhaps I'll continue it. So with that, thanks for listening. And if you'd like to donate to all my initiatives here and helping to sustain and grow the Voices of VR podcast, then please consider becoming a donor at patreon.com slash Voices of VR.

More from this show