Mike Alger just finished up his master’s degree focusing on VR interaction design methods, and he’s released a couple of popular videos summarizing his insights into immersive usr interfaces. I caught up with Mike at the Seattle VR expo talking about how he’s been using his film background and motion graphics skills in order to transition into VR experience design.
LISTEN TO THE VOICES OF VR PODCAST
Mike says that interaction design is all about looking at the properties of a given technology as well as the properties of humans. The different types of motion tracked controllers versus hand-tracking will dictate the best user interface. For example, if you look the mouse and keyboard model of interaction for a desktop computer then you have two different types of input, and perhaps immersive professional applications will eventually take a similar approach of having one hand-on a 6 degree of freedom controller, and the other hand being tracked with a camera. But having a combination of technologies would dictate the range of interaction design possibilities. Mike had an example of this in one of this videos:
Mike uses his background in filmmaking and motion graphics in order to put together some prototypes and user interface ideas in a couple of popular videos called “VR Interface Design Manifesto” and “VR Interface Design Pre-Visualization Methods”
Mike is able to have some fast iterations by using After Effects and Cinema 4D, but he also advocates translating other ideas from web design such as the design grid. You can still do a lot of paper prototyping of interface ideas, but VR is a medium that has spatial, depth, and lighting considerations and so it’s helpful to see what it actually might look like in VR as well. Mike refers to Alex Chu’s presentation where he suggests using a “greybox prototyping method” by putting in textureless primitives in Unity to quickly test out lighting and spacing layouts.
Mike’s second video had a call to action that he was looking for a job as a VR experience designer, and he’s since taken a job within Google’s VR design department. Based upon the feedback he got on the video, he sees that there’s a huge demand for more designers to get into VR to help experiment, iterate, and figure out the rules for creating the most comfortable interactions.
I asked Mike about projecting the UI onto a sphere, and he said that this feels more comfortable because the UI elements are all equidistant from your eyes and that this helps to avoid the vergence accommodation conflict that can happen with objects being at different distances. Another tip that Mike gave was to look at how we are currently mimicking depth within 2D interactions with drop shadows, and to convert those visual cues into actual depth within VR. He’s also really interested in how to create non-fatiguing interfaces that prevent forcing users from doing too many repetitive motions that could strain muscles.
Mike says that being a filmmaker is like being an experience designer in a 2D medium, and that there’s still a lot to be learned about the language of VR. For example, Oculus Story Studio discovered that the distance of characters from the camera can affect the emotional impact of a scene. And just like any other communications medium, VR is going to have a lot of strengths and weaknesses and these specifics will be more clear as more VR creators experiment with different combination of factors.
Finally, Mike hopes that he can help make VR a socially acceptable way to experience immersive entertainment and education as well as do useful work in the enterprise. Since the recording of this interview, Mike has since taken a job as a VR designer at Google and I hope to see a lot more insights and lessons learned as he continues to iterate and explore his ideas about interaction design with VR.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast.
[00:00:11.958] Mike Alger: I'm Mike Alger. I just finished a master's degree where I was looking at virtual reality interaction design methods as they relate to user interface and user experience as a whole. Great. And so you just produced an 18-minute video that goes over a lot of these lessons learned about user interaction, user interfaces. So maybe talk a bit about what were some of the big conclusions that you came to in terms of the optimal way to do user interfaces within a VR environment. So it's kind of a method of thinking as it applies to looking at what the properties of the device are and what the properties of humans are to come to some sort of conclusion about what the best design methods are that we can be putting things in front of those humans for them to have a good experience or a bad one if that's what we're trying to design it for like if we're trying to have it be instilling some sort of feeling in them and getting them to perform some sort of action that we can look at what are the ways to use the properties of the device in whatever way. And so the things that I talk about in the video more specifically are if we're doing a interface for an operating system, which then I'm going to assume would be a sitting experience. And then if you define which head-mounted display you're using, you can use the properties of that and the properties of ergonomics to get some zones for content, for example. And then the other thing that I go through is Maybe if you're using, for example, hand tracking, then you can use the properties of that device and then use some affordances of buttons to kind of communicate what interaction you're supposed to have with the interface elements. It seems like the thing that I find really interesting is that your use of Adobe After Effects as kind of like a rapid prototyping tool and then using that to be able to create some visualizations. Maybe you could explain to me a little bit about how you use After Effects to do stuff in VR. Yeah, After Effects is kind of just a stopgap for me simply because the way I describe it to people is that everybody who is in this current wave of VR, maybe 90% of us were doing something else three years ago. And so a lot of people obviously come from game design, but I was doing video production, motion design, and motion graphics. And so my skill set was After Effects and video production workflows and so coming into VR saying I want to get into this I was thinking I should probably use my existing knowledge base and workflows to bring me up to speed into this new area and it seems like interface design was a natural you know We are already designing the sci-fi interfaces in After Effects for films, we may as well start using that to design the new ones. And it's a good rapid prototyping method because you can quickly keyframe some things, whereas coding the scripts that are going to do those same things in Unity or Unreal or whatever engine you're using is going to take significantly more time. kind of iterate on the design process much more quickly. The reason that I think that that is good to do is because it is kind of the same thing that we've done with other mediums like in film you have storyboarding and animatics as well and in web development you have web designers who will make a PSD for example first with the layers and maybe mock it up with some comments and that sort of thing. And it's a good method for prototyping something where you can iterate quickly on the aesthetics and that sort of stuff and maybe solve some quick problems before you implement those sorts of things. But you can also rapid prototype with greyboxing techniques in Unity and try to space things out correctly and I don't necessarily think that, like, After Effects is the best tool for it or anything. It was just kind of something that I was like, here's a quick way to get this. It makes sense to have foreground, midground, background coming from theater and film, those sorts of things. And it makes sense to pull in design grids from, like, print design for magazines and that sort of stuff and start mixing all of these different mediums and what we've learned from them to try to design what will be hopefully compelling experiences.
[00:04:15.846] Kent Bye: Yeah, you know, I've got a background in both filmmaking and working in Drupal open source content management system, working in web agencies that are getting wireframes and there's different methods to create either storyboards or to create a wireframe for a website that's kind of a high-level way of prototyping. it out before you actually build it.
[00:04:33.803] Mike Alger: And the challenge I see with VR is that because it's an immersive experience, you almost have to get in there and lay it out in VR. And it's harder to do paper prototypes in a way.
[00:04:44.447] Kent Bye: So when you do these types of ideations of trying to create things, are you doing a layer of paper prototyping?
[00:04:51.649] Mike Alger: Or are you just diving into VR and using After Effects as your stopgap? You know, I do have the Moleskine notebook that I've got my sketches of the different, for example, people love in the video the little wrist UI sort of thing, and I've got several iterations of hands with UIs in different places in my notebook, and then in Illustrator I have a bunch of different layers and things next to each other as I iterate through the different designs in that step of it, and then Yeah, it really ends up being the same as if you're doing an animation workflow, you're going to do those same sorts of things where you sketch it out first, you create the assets, then you animate the assets. This being a little different in that you would ideally be preparing those assets to use as art for a developer to then implement. Like if, for example, it is those buttons, that is a texture that's going to go onto a quad and you're going to be doing it at the right sizes and transparencies and optimizing those things for later because a lot of the same assets that you put into something like After Effects or whatever thing you want to use or just sticking it into Unity yourself are going to be things that you can use even as the assets for the final thing if it looks good. And what is this Unity grayboxing technique that you mentioned?
[00:06:07.409] Kent Bye: What's that involved?
[00:06:08.783] Mike Alger: Yeah, okay, so in the video I reference a presentation by Alex Chu when he was at Samsung. He did a presentation at the Samsung Developer Conference in 2014. It had some really great things that I sourced from, and one of the techniques that he talked about was basically just putting in primitives and scaling and sizing them, and without textures, they're all gray. And so there's a lot of gray boxes going on, I assume is where he's gotten it from, and so... It's just a quick method to test lighting and spacing in an environment where you can hit play on Unity, put the headset on, and check how things feel spatially so that those things that we just talked about of using paper, using Illustrator, using After Effects, is missing the whole point of VR, which is the spatial depth and understanding. And there's a lot of hierarchical understanding that you can build in to interfaces using that depth that was not previously possible. Yeah, gray boxing would be kind of the technique of just throwing that stuff up in there like texture lists and seeing how it feels I suppose and there's a point in your video where you had kind of like this whole Google Maps thing with things popping up and I looked at I was like wow that looks amazing Did you develop that in 3d or is that just sort of like a 2d? Kind of proof of concept of this is what it could look like if you were in 3d But you know watching a 2d video is a little hard to tell whether or not you actually built it out Yeah, I should mention that another reason to use After Effects for me is that it has that integration with Cinema 4D where you can go back and forth between the programs really quickly. So that stuff, the 3D things that are going on like the Vive controller or the map are things that I was doing in Cinema 4D and then referencing back and then rendering it out for the final thing. It's not built in a game engine, it was purely just a visual mock-up and I don't even think that all of those things that I showed were necessarily like the best ways to do it. I was simply trying to show some of the different interaction methods that people might not have thought of like pulling the indicator to the side or some of the radial menus that I was showing or the wrist UI or some of that sort of stuff, yeah. Yeah, I just think that right now the barrier to entry to kind of unleash your creativity in designing user interfaces, there's a lot of barriers for a lot of people to be able to get a sense of what it feels like. And so, yeah, I was just kind of surprised that for your research project for your master's that you were kind of actually doing a lot of implementation and prototyping and building stuff out and really exploring what actually feels the best. And so I guess there was a kind of a pitch at the end for you to now you're out looking for a job. And so what are you doing now and what's next? Yeah, so sneaky of me, right? Yeah, I mean, I imagine by the time that this podcast goes out, I'll probably be somewhere. I mean, right now, a lot of people did contact me from that. So yeah, I will go to one of those. It kind of will be a matter of determining what the company culture and location and projects and those sorts of things and it'll be a really tough decision because there's so, so, so much cool stuff happening in VR that you don't want to miss out on any of it, but I guess I do just need to work on something. So what I should say though is that based on the feedback that I got from that video and the number of companies and people who need this kind of design work to be done in their applications, If there's anybody who's interested in getting into user experience and user interface, there is currently a major need and a lot of opportunities for somebody who's interested in getting into that space. If you're worried about experience, none of us have it. I am guessing just as much as the next person and I'm glad that a lot of these ideas were resonating with people because maybe I'm headed down a correct path or something and I don't think it's even necessarily thinking that I'm getting these things right. But I think people are excited about the fact that anybody is trying stuff and so there's a lot of space to be trying things and there's a lot of companies that need it. So if anybody else is interested in getting into this position which I would call VR designer in the same way that other things have like a web designer that seems like we need VR designers for this sort of stuff. I would say now would be the time.
[00:10:33.201] Kent Bye: And so what are some of your experiences that you've seen that you think have like really excellent user interfaces or user experiences?
[00:10:42.688] Mike Alger: You know, it always depends on the input method. It's the ones that draw your attention to the correct place at the correct time and have clearly readable text that's not eye-straining. So, for example, Google Cardboard Design Labs is, you know, obviously based on the concepts of design, and so that one makes sense. I'm trying to think if there's any with controllers, and I feel like I haven't done enough things with controllers to really feel like there is one that's the best definitive thing on that yet.
[00:11:17.007] Kent Bye: Have you had a chance to try Tilt Brush or Medium yet?
[00:11:19.809] Mike Alger: Oculus Medium? I have not tried Medium, and Tilt Brush is a very good one. That's a good example. I'm glad that you brought that up. I'm sure that many other people have mentioned it on previous episodes as well. Yeah, that's a simple one as far as, like, swipe through the stuff, point at the things. It's the sort of thing where you almost hate to describe it to an audio listener because then you'll say, oh, you just have to see it. But then even to see it on a video, it's like, well, no, you just have to try it, don't you? And you hate to do that. But obviously, if you have the opportunity to try any of these things, you should. Yeah, the Tilt Brush interface is really great. It's smart. I think we'll see a lot of things emulating that. I think that if you're making an app that has choices for people to choose between tools, you should not just use Tilt Brush as your default, and that you should try other methods to be choosing tools. Because maybe there's other methods that are equally as good or better that we haven't figured out yet as well. It's a really great one, but don't let that stop you from trying out other things as well.
[00:12:28.570] Kent Bye: Yeah, and I think the, you know, just a design principle that I've seen that works really well is just, you know, projecting things onto a sphere, you know, not having things flat, but having things kind of almost equidistant away from you just gives it a little bit more of a immersive feeling, but also a little bit more comfortable, I found.
[00:12:45.298] Mike Alger: Yeah, and part of the reason for that is actually a comfort issue where your eyes, you know, you've got that convergence accommodation conflict that's going on with head-mounted displays currently, and if you have that interface curved around them at a distance that's equal to whatever the lenses are focused at, which is, you know, I mentioned in the video 1.3 meters for the DK2, but the CV1 and Vive are something more like 2, you'll probably want to check that first, but If it is curved around you at that distance from your head, that being your radius for the curve, then you avoid convergence combination conflict, where the eyes feel comfortable going all the way around, as opposed to if it's a flat plane, your eyes have to be converging at different distances at different points on that interface, and that'll cause a little bit of strain and discomfort over time. So that's one of the reasons, I should say, that if you simply wrap a UI 2D around the user, you do capitalize on this idea that increased real estate increases productivity, and that's great, but you miss out on that huge one, which is the spatial understanding and that z-depth. So, you know, if you want to start with that, I'd say the first things to start bringing your user interface into 3D would be look at the things in your 2D UI that are trying to mimic 3D elements. If you have like drop shadows or bevels on things, that's us in 2D trying to mimic 3D. So that drop shadow should actually be something that's pulled out in z-depth by maybe a few centimeters in the space. Or if you have tabbed hierarchies and those sorts of things, those tabs are an obvious place to be putting some z-depth into there. You'll find things in your existing 2D UIs that you are mimicking 3D space in a flat plane and you may as well just go with 3D with it, yeah. Yeah, and I think one of the things that you had mentioned in your video, but also that I've did some interviews at IEEE VR with the 3D UI conference chair and Rod Lindemann talking about how you really need to have like non-fatiguing interfaces. So, you know, not having yourself in an uncomfortable way. So, in terms of non-fatiguing design, what type of special considerations have you taken account with some of your recommendations that you were making in your video? You know it's kind of just in the video if that's what you want to talk about specifically I mentioned you shouldn't force somebody to be raising and lowering their arm on a regular basis because they're going to be getting tired of doing that pretty quick and then you also don't want to give them a surfer neck by having to turn around all the time with their necks if they're in a non-rotating chair. You know, you can obviously extrapolate the same things to other stuff, where if we can get rid of carpal tunnel, thank goodness, because, you know, that's not great. But then you want to also avoid causing somebody to keep repeating the same motions over and over. in the same way that there are ergonomic considerations in, for example, industrial actions of factory workers and those sorts of things where you try to avoid harming employees at factories by making sure that they're not having to perform physical tasks that repeatedly strain the same muscles or repeatedly stress on the same tendons and that sort of stuff because you're going to be causing physical wear and making people sore and that sort of stuff. You know, those things we haven't thought about with video games very much previously, a little bit with the Wii, but if you are designing interfaces for people to be using where they're performing some tasks on a regular basis, you want to avoid those sorts of things. When did you first know that you wanted to make this your thing, kind of doing user interface design for virtual reality? You know, I'm not even totally convinced that I want to do user interface design so much as I want to do user experience design in the larger sense of having started with film and then going into motion graphics, motion design, that sort of stuff. Yeah, interface was the starting point, but there are the larger implications of drawing attention using these cues of motion and color and detail and affecting a user's emotions using the equivalent of the long shot or the equivalent of the compression or expansion shots. Basically, re-figuring out the things that like Alfred Hitchcock or Orson Welles or people have figured out about film and we've talked about this several times in the VR space but that is user experience like being a filmmaker is a user experience designer for a 2D medium in its own way because you're trying to affect an audience by taking them through these emotions by placing a red thing here and we're gonna compose the shot there and we're gonna take this pacing with editing that is all manipulation techniques to get the audience to take some sort of path and for VR we can do the same sorts of things and see if they're even more effective and I loved the recent video talking about those same things in Henry where I'm sure you saw it with like doing a close-up shot and having Henry next to you really sad was actually uncomfortable but it was more effective to have him like a long shot in the distance sad and like that's isolating and So that is the beginning of those sorts of things to me. And so interface design is part of that because that involves getting people to not only be receiving information, but inputting information in a way that doesn't interfere with their experience. I mean, there's probably some points in game design where you want it to be frustrating and you may design an interface to be confusing or a puzzle or something like that. But the point being that it is a design process, that you're not just throwing something in because you saw it in another game or because it was a quick fix to get something, but really going through the user interaction testing process of making sure that it's the best way for people to understand things quickly and have the best experience. And finally, what do you see as the ultimate potential of virtual reality and what it might be able to enable? I knew this was coming my whole life. I feel like, man, the question, the big question. And I had an answer thought up for this because I've always listened and there's always, I'm like, what would I say to that question? And I had a really good one and I don't remember what it was. But, you know, it's going to be the same thing that a lot of people echo. In the sense that I don't see it as replacing these other mediums. I don't see VR killing 2D gaming in the way that video killed the radio star sort of stuff. It's another medium. It's a different medium. It has strengths and it has weaknesses in the same way that if you're designing for a partially see-through augmented reality device has strengths and weaknesses. You would want to do an IKEA app for augmented reality, but you'd want to do an architectural pre-visualization app for virtual reality. In my opinion, obviously there's different things for each of these. So the ultimate potential for virtual reality to me is, and I would like to help to get it to this point, is that it's a socially accepted medium that is providing another form of entertainment, education, and all the medical uses, enterprise uses as well. So, yeah, entertainment, education, enterprise as a useful medium for those things that is socially accepted and doesn't have a social stigma associated with it anymore, which I think we can get to through the creation of good experiences and useful ones in the medium, yeah. Great. Well, thank you. Thank you. And thank you for listening.
[00:21:05.055] Kent Bye: And thank you for listening. If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash Voices of VR.
