Reginé Gilbert is the author of the book Inclusive Design for a Digital World: Designing with Accessibility in Mind, and teaches at the New York University in the Tandon School of Engineering. She has been teaching UX for nearly a decade now, and is actively researching accessibility design heuristics for virtual and augmented reality. I’ll be getting into more details about XR heuristics in episode #1226 covering the XR Access Symposium poster session.
Gilbert’s book covers a lot of the best practices of designing Accessible Content for 2D content including the Web Content Accessibility Guidelines (WCAG), the Seven Principles of Universal Design (Equitable Use, Flexibility in Use, Simple and Intuitive Use, Perceptible Information, Tolerance for Error, Low Physical Effort, Size and Space for Approach and Use), and the Seven Inclusive Design Principles (Providing a comparable experience, Consider the situation, Be consistent, Give control, Offer choice, Prioritize content, Add value). We talk about how each of these frameworks starts to get applied to accessible XR design, as well as some of the pending work that has yet to be done. Hopefully this 15-part Voices of VR podcast series on XR Accessibility starts to map out some of that landscape of pending work and provide more context to folks like Gilbert who are advocating that XR Accessibility be integrated into the core experiential design processes. A full rough transcript is included down below on this episode, as well as now available on each of my 1200+ episodes of the Voices of VR podcast.
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye:, and welcome to the Voices of VR Podcast. It's a podcast that looks at the future of spatial computing. You can support the podcast at patreon.com slash Voices of VR. So this is the third of 15 of my series of XR Accessibility. Today's conversation is with Reginé Gilbert, who's written a book called Inclusive Design for a Digital World Designing with Accessibility in Mind. This is looking at all the different general accessibility practices for a lot of the different existing 2D media. There are a couple of chapters about XR, but I think it's a really great book to get a general background of accessibility as it applies to existing 2D modes of communication. A lot of the work that she's doing now is really on the cutting edge of trying to figure out some of those different heuristics for both virtual and augmented reality as you start to translate some of these design patterns that are from the web content accessibility guidelines as well as the seven principles of universal design as well as the seven principles of inclusivity. She's pulling together lots of different heuristics and I think it's a process right now that everybody's going through of trying to translate some of these different generalized principles into what's that look like in virtual augmented reality. So This is a conversation that I did back in March of 2022, right when I was in the throes of finishing up the 14-hour podcast series of looking at all the different ethical and moral dilemmas of XR for the IEEE Global Initiative on the Ethics of Extended Reality. So coming back to put this conversation in the context of this larger series of looking at accessibility, because I think it fits right in and Regine was at the very first XR Access conference in 2019 and also had some of her grad students presenting some of their latest work at the XR Access 2023 conference that I'll be featuring in a whole podcast episode where I dive into the different poster sessions to get a little bit update for what's happening with a lot of her research in this area. So that's what we're coming on today's episode of the Voices of VR podcast. So this interview with Regine happened on Friday, March 25th, 2022. So with that, let's go ahead and Let's dive right in.
[00:02:15.961] Reginé Gilbert: Hello, I am Reginé Gilbert. I'm currently an industry assistant professor at the New York University in the Tandon School of Engineering, part of the Integrated Design and Media program. I teach user experience design, assistive technology courses, and I've created curriculum for UX for AR and VR. I am researching inclusion and accessibility in the XR space. And in addition to that, I am co-authoring a book with Doug North Cook titled Human Spatial Computing that will be coming out in 2023 through Oxford University Press.
[00:02:53.470] Kent Bye: Oh, amazing. That's really great to hear. Uh, well, maybe you could give a bit more context as to your background and your journey into this XR space.
[00:03:01.413] Reginé Gilbert: Sure. So I am not a veteran of this space for sure. I've gotten into it in the last five years. I used to be a user experience design consultant and I worked out of a WeWork in Harlem and there was a company that sat next to me and they did augmented reality. And so we just started talking and they're like, we need UX help. And I was like, Oh, happy to help. But I'm more web-based, but it was really cool to be able to get to work with them and do usability testing, and at the time I was teaching user experience design, the folks through my WeWork introduced me to the folks who ran Torch. And Torch was a great augmented reality tool that no longer is out there. But it didn't require folks to code. And it was something that I started to integrate into my classes. So I wanted my students to not only think about web, but think about AR. So that's kind of how I started to get into it. And then over the years, I've had a growing interest in accessibility so much so that I wrote another book called Inclusive Design for a Digital World. And I got involved with XR Access a few years ago. So that's really, I mean, that's kind of how I got into it. And NYU asked me to co-create a UX for AR and VR course, which I've done two at this point.
[00:04:22.262] Kent Bye: Okay. Yeah. I know talking to Dylan, he's been working on a number of different accessibility aspects as well, but also a background in user experience. And so there is this three line between thinking about user experience generally and how to make it as inclusive for as many people as possible. Meaning some of the people that aren't fully abled and different degrees, whether it's their sensory experience, cognitive or physical movement ability. So different layers of inclusivity there. So I guess before we start to dive into the XR specific things, I'd love to hear about that moment where you pivoted from doing more product management into going back into digging more deeply into aspects of user experience to the point where now you're teaching it.
[00:05:05.501] Reginé Gilbert: Yeah. So I've actually been teaching UX for quite some time. This is my seventh year teaching UX, almost eight. So I've been teaching regular UX for the web for quite some time and One of the things that I guess made me different as an instructor was that I incorporated accessibility from the very beginning of my courses. And I still do. It's the first thing we talk about in the very first class. Because I think that often times designers don't learn about accessibility. Developers don't learn about accessibility. And I teach at a school of engineering where not everybody learns about it. And so in my class, you will always. Because that's a promise I made to a friend who, you know, when I first met her, I told her, "Oh, I do UX design." And she's blind. And she said, "Do people like you think about people like me?" And I answered honestly, and I said, "No, but whatever I do, people will." And I think that, you know, why aren't we thinking about folks with disabilities when we're working on this technology? It should just be an automatic, in my opinion.
[00:06:14.235] Kent Bye: Yeah, I've noticed that there's certainly in the web sphere, there's been quite a lot of maturation in terms of the types of tools that are available to be able to translate the information that's delivered through the DOM into screen readers or different assistive technologies and input controls. I mean, it seems to be at a lot higher level of maturity, making content that's on the web more accessible. But when I start to think about these new emerging technologies, I guess the story I tell myself at least is that there's been so much time of just trying to get it work for people who are able-bodied, And then once it does finally work for people, I mean, just the many years of latency and all the different things that have had to happen in terms of the technology to get to the point where it doesn't make people motion sick as an example, but there's still yet a lot of work to be done in terms of making these new emerging technologies accessible for folks who are not fully able-bodied. I know that XR Access has been doing a lot of the early work of gathering the community together and start having these different types of conversations. And so I guess from your role in academia, what do you think is the next steps in terms of whether or not you feel like there's more research that needs to be done in order to start to figure out what the baseline is, or if it just needs more funding, or if this is something that companies like Meta need to be participating to more and more degrees. And so I guess when you think about this as an issue, where do you even begin, especially as your role in the larger ecosystem for how to start to close the gap to make these technologies more accessible and more inclusive?
[00:07:46.675] Reginé Gilbert: I think all of the above to everything you just said. I think first and foremost, anything starts with awareness, but then you have to move from awareness to action. I think we've reached a point with web accessibility where a lot of people know about it, but still people are a little resistant, it seems, to incorporating it. I've seen a lot of resistance to incorporating it into the XR world because it's not there, right? It's not in the foundation. One of the things that I did last year was I looked at over 60 different types of tools. I was looking specifically around inclusion and accessibility, seeing how much things cost, the scripting languages people needed to know in order to use it. And if there were any accessibility anywhere mentioned, you know, as part of the tools and out of the over 60, only four had anything accessibility related. And, you know, things cost money and this is barriers to entry. When I think about things, I think about things on a global scale, right, and an intercultural perspective as well. And so there are a lot of barriers to entry in general. And then when you stack on top the fact that most people do not think about people with disabilities when they're creating experiences. That's just an extra layer. So I think in the beginning, people have to be aware that when you are creating something, are you offering options to people that may not be able to hold two controllers, right? That may not be able to hear whatever is playing, that may not be able to see those different colors. Are you providing folks options? And I think at the end of the day, accessibility is about options. And we just have to get into more of a habit of understanding that as designers, as developers, as content creators in VR and AR, it is our responsibility to make sure we're opening this up to as many people as possible.
[00:09:41.302] Kent Bye: Yeah. As you're saying some of those different options, I mean, the thing that comes up to mind for me, at least for a lot of VR games is the locomotion options in terms of there's different comfort levels that people have of locomoting, whether it's teleporting, which is probably the more comfortable options, but some people like the smooth locomotion when they're able to just walk through the virtual space. And, you know, as we've progressed and evolved and matured into the industry that, you know, some of the more higher end games will give options, but it's not something that is just an automatic that they'll have a variety of different locomotion options as an example. So susceptibility to motion sickness seems to be something that people may not consider themselves to be a level of ability or disability, but it certainly is a level of access and an inclusivity as we move forward. So thinking about that in terms of lots of different. Options that are made available. And I know that the XR access has started to make some of those different types of things available, but for your work, where do you start to, I guess, focus in on those other things, whether it's like text and readability or other accessibility categories, when you start to break down this field, when it comes to XR accessibility.
[00:10:51.354] Reginé Gilbert: Yeah. So one of the things that I'm working on currently is heuristics, and looking at specifically what are heuristics for AR, what are heuristics for VR, and what does that mean for accessibility, right? There are heuristics that have existed for a very long time for the web, but you can't just blanket that on, right? We're talking about space. You know, when you talk about AR and VR, it's space, right? So it is when you're talking about AR, you have space that is 90% out of your control. When you're in VR, you're in this environment, but it's all space. And how do you measure? And how do you evaluate that? So my grad assistants and I are looking at that this semester and beyond, and it's my hope that we're able to publish something and release it because there's stuff out there, but there isn't stuff out there, right? There's stuff out there that people can't find. There's a lot of academic stuff. People don't really know. There isn't this, you know, it's like you could go to the W3C for the web. And the W3C has actually published some really great guidelines around XR and accessibility. But if you're a designer, how do you know what to do? How do you know how to evaluate what you're doing? And how do you know if you are making something accessible or not? You don't really. So it's my hope that I'm not going to be alone in working on this, but that we can get that out there.
[00:12:13.918] Kent Bye: Yeah. And in your book, you mentioned a number of different previous accessibility initiatives that were like the web accessibility initiative and then the WCAG from the web areas. And so what kind of lessons are you taking from accessibility that has been done on the web and different initiatives by the W3C and what kind of things do you think translate over pretty well when we start to think about XR?
[00:12:37.107] Reginé Gilbert: I think the things that translate are the groups of folks that you need to think about right, so you need to think about auditory, you need to think about vision, you need to think about people who are neurodivergent, you need to think about mobility, right? So all those things that the W3C has published over the years for the web, you can think about those groups of people and what does that mean, right? Are you creating an experience where you have to hear everything to move? And folks are working on captioning and VR, which is wonderful, but what options can you give to folks, right? One of the things I was doing last semester and last year was looking at different design patterns in VR and seeing where the gaps were, which there's plenty. And at some point I have to put this out there in the world that I haven't yet, but there's lots of gaps for what we see. But I think there's a lot of lessons to be learned from the web in terms of what we can do right for XR.
[00:13:36.076] Kent Bye: Yeah. I noticed that you have a lot of different frameworks and approaches, everything from like the seven principles of universal design or the poor standards. And so what are some frameworks or heuristics that you feel like are also good to start to understand the principles of inclusivity and accessibility when we start to think about XR?
[00:13:57.605] Reginé Gilbert: I think one is that not everybody is experiencing this wave of the metaverse, if you will. That's not my phrase, but it's somebody else's. It's different for everybody, right? And I think that the assumption, especially in the United States, that everybody has this high speed internet and they're just getting on in. No, a lot of people are not, right? A lot of people are not doing things that way. And I actually recently working with a group of storytellers who are going to be using 360 VR in different countries in Africa to tell stories, because I have a storytelling chapter in the upcoming book. And I was like, the way that people are doing VR in different parts of the world is not the way that people are doing it in the West. You know, I have a colleague in Peru who is also doing storytelling with 360 VR. And that's the way a lot of people are getting introduced into it, into this space who are not in the US or Europe. And so looking at that, knowing that there are different ways that people are accessing this technology, makes me say we're so immersed. And I've been very observant over the years of Twitter and VR and what's happening. And it's all beautiful and glossy, but that's not a majority of the world, right? A majority of the world is not experiencing things in this way. And the affordability of VR headsets, like standalone VR headsets are quite expensive. I talked to somebody in India who told me they spent one month's salary on a headset. So that tells me, like, who is this for? Who is this new XR world for? Is it only for the rich? Because it seems to be leaving the poor behind. And that makes it exclusive and not inclusive.
[00:15:44.312] Kent Bye: Yeah. And also as you start to dig into what's already happening in the web world, in your book, you're talking about all those communities and technologies. And as you were writing that, I'm curious if you could maybe expand upon your own process for either going to different meetups you had mentioned in New York city, but also there were communities that you were able to have closer contact with to be able to connect to people who we're actually using the web with these different assistive technologies. And yeah, just as you were starting to not only evaluate the landscape and see that there wasn't a lot of discussion about accessibility in your personal education, as you're going through these classes and that there was this gap there. And so just curious what your process was to start to close that gap and to either seek out those communities of people who were professionally working on that or people who were using those technologies to get access to the web.
[00:16:37.553] Reginé Gilbert: Yeah, so I'm lucky, I think, in that I belong to a lot of different groups. I mean, one of the ways that I first got involved in accessibility was going to a meetup for the first time. And there I met Thomas Logan, and Thomas Logan does amazing work with accessibility and VR and has an accessible VR meetup that happens once a month. And so through Thomas, Thomas and I have bonded and we've been working on the same be our design patterns project together and specifically parsed out questions that were related to accessibility, right, and what those gaps are. So a lot of the work that I've been doing has been looking at those gaps. And in terms of the communities, yeah, I like going to meetups wherever I can. I am very lucky that I get to work with folks with disabilities, and there's nothing better than talking to people with lived experiences. The other day, one of my colleagues that I co-teach a class called Looking Forward With, I said, I want you to try Snapchat. And he's blind. And I said, "I want you to try it, because I think maybe we can make a filter together." I don't know. In my head, I'm like, let's do something that's like, let's make this accessible filter. And so he uses VoiceOver. He has an iPhone. And so he downloaded Snapchat, he got signed in, and then once he got signed in, he's like, okay, now what? Because everything was so visual, right? And so then I was like, should we make a filter? Is it going to be like, would you want to use this thing? And then it had me thinking about what are people getting left out of with the technology that we have? Like, how are we incorporating it so that it can be used, right? And his name is Gus Chalkias. We worked on a concept a couple of summers ago about making an augmented reality audio game. And I had my students concept it, and we didn't actually make it because we didn't have time. But that whole thing came out of a discussion with Gus about him wanting to play Pokemon Go, but he couldn't. And I thought, why, if he's blind, can't he play and why aren't we again, why are we leaving people out. So that's just my ongoing like thing is constantly questioning and trying to figure out how to go about it. And I'm really, really grateful to be part of an organization like XR Access that actually has developers working on things. And I'm more of a researcher. I'm not a developer in this field. And I'm just now getting into design a little bit more and going to become a maker.
[00:19:19.610] Kent Bye: Okay. Yeah. You had mentioned that you had attended the first gathering of XR Access in 2019. Maybe you could talk about that gathering of professionals who were looking at accessibility for XR.
[00:19:29.509] Reginé Gilbert: Yeah, so that was really, really cool. I mean, that was in the before times before the pandemic when we could meet in person. And that very first meeting was so amazing. It was put on by Cornell Tech. Cornell Tech started XR Access. But people from all over the country and all over the world came to this event where people were specifically talking about extended reality and accessibility. So there's a lot of work being done on the academic side in regards to folks with low vision, the deaf and hard of hearing community. So there's a lot of things being done there. Microsoft was there, Facebook was there, and it was good to hear different people working on different things, but we're now in 2022 and the progress has been slow. Right? Because I think there's still this huge lack of awareness when it comes to this field. I mean, I've talked to independent VR developers who tell me, I can't do it. I can't add accessibility. I don't have the time. I don't have the money. I don't have the people to do it. And I understand that. And yet I will still question it. I will always question it and say, "Well, do you want as many people as possible experiencing this thing? Or do you want to limit it? Because what you're doing is limiting your experience and you're limiting yourself, frankly, in my opinion, right?" But I know that's the case. So I hope I answered your question there.
[00:21:01.751] Kent Bye: Yeah, I know that I saw a quote from Steve Feiner, who I've had a chance to meet at different IEEE VR conferences and industry gatherings. And he gave a keynote at IEEE VR a number of years ago, where there's a lot of research that he's been doing just in terms of trying to make text readable or not to include different things within an augmented reality context. But sometimes I find that there still can be a gap between what's happening in the academic research and then what is actually percolating out into industry. where some of the stuff he talked about in his keynote were then, you know, talked about either by a journalist or kind of unaware that some of this prior work has been done. And so I don't know what the solution there is to have a closer collaboration between the academic world and what's happening with the research that's happening with some of the stuff that Steve Feiner is working on, as an example, into the broader industry and how to facilitate more of that dialogue to have the research that's being done, but also taking the time to actually integrate some of these best practices
[00:21:59.842] Reginé Gilbert: Yeah, I actually reached out on Twitter to one of these big companies and I said, "Hey, I'm looking for folks who work on accessibility at, you know, any of these companies doing this stuff." And people actually reached back to me and we had a nice conversation. And I had a guest speaker who came and spoke about audio captioning. And I invited folks from this company to listen in. And they were very happy to listen in. And they also, who knows, might be using these folks in the future for audio captions. Because there's lots of videos that come out of games or experiences And I have no idea what's going on. It's just action, action, action, action, action, action. And I'm supposed to know what's going on. However, if there were an audio description to that video, for me personally, I would like it a lot better. And I think a lot of other people would too. It would also provide, if you had audio caption on a video, even if you're just doing a demo of something, And you were describing what that video is. If somebody wanted to watch it without sound and there were captions, they would totally understand what's happening. And these are the benefits that we can have if we make things more accessible.
[00:23:17.595] Kent Bye: Yeah, I know. Just even when I'm watching Netflix sometimes or different streaming shows, I'll often have the captions on just so I don't miss any little bits of dialogue due to accent or just not hearing everything properly. So I know that that's been helpful to be able to track shows just that I'm watching in an entertainment context. Whereas, you know, before the whole streaming revolution, I would hardly ever watch the captions, but Yeah, I see a lot of that more on like videos that are designed for you to watch it with or without sound. And also the alt text on photos and whatnot, there's ways of adding text. I guess a challenge that maybe is taking the experience of the person that was blind that was trying to use Snapchat, how there is this shift that I see not only in some of the apps that are already not accessible, but then going into these spatial worlds that you're immersed with either virtual world that's either created by Unreal Engine or Unity. And then it gets compounded into almost like this black box that is difficult to know even specifically what you're looking at. WebGL is a good example in the web world where you no longer have access to what would be the equivalent of a DOM to know exactly what people may be looking at. And so there seems to be either a need to have something like the equivalent of a screen reader for 3D or some intermediary technology, or maybe a change in the technology stack that gets closer access to maybe the USD formats or some sort of scene graph so that you could at least have some spatial representation. So I don't know if there's any discussions about this in terms of, you know, how do you even start to create the equivalent of something like a screen reader for VR when we're already down the path of creating these black boxes that are no easy way or intuitive way to be able to start to put something on top of that, like a screen reader could read a DOM and be able to have an experience that is translating that content for people who cannot see, there seems to be some technological barriers for doing something similar for XR.
[00:25:16.024] Reginé Gilbert: Yeah, there are, and there aren't. And I can't remember off the top of my head right now, but there was somebody Oh, it's not coming to me at the moment, but there are people who are working on screen reader and navigation in VR. But for the most part, if you're blind, VR is not the place at the moment, right? I think that there is a lot of opportunity in that space to make things better because you put this thing on and what do you see? You see nothing, right? So it's not as valuable as it is to someone who's sighted unless you can provide options, right, for experience. So there are audio things that exist, but they're not great at the moment. So we have some work to do, I think.
[00:26:04.975] Kent Bye: Yeah, I just had a chance to talk to one of the creators of Notes on Blindness, which as a piece was talking about John Hull, who had gone to the process of losing his sight. And so there it's more of a narrative experience that is trying to build empathy for people who are blind and just trying to give an immersive experience using the conceits of spatial audio. So yeah, I guess that's a piece that's starting to at least close the gap in terms of people who can see and a little bit more of a poetic interpretation of that. But I feel like that, as I read through your book, that the building of the empathy and the different frameworks that you've created to go through the process of speccing out what you need, and then knowing the context and knowing the users, maybe you could just talk a little bit about the role of empathy, but also these other frameworks that you've created in terms of, as you start to go through the process of user experience design, some of these key components.
[00:26:55.346] Reginé Gilbert: I think context is very important to think about in these situations because Empathy is one thing, but beyond empathy, you need compassion, right? And you need to think of others. And I think a part of why I started researching inclusion and accessibility in XR is for my own selfish reasons. I am somebody who wears glasses. I am somebody who I clubbed back in the day and my hearing, you know, I can tell there's a difference, right? You know, I'm getting older and I said, I really like VR. I love doing things in VR, actually, and got my brother into it. I want to be able to use this when I'm older, right? And so that's kind of where my thought came from is like, I thought of it in this way is like, sometimes when people have kids, they'll buy the crib that turns into a bed. And I was like, we need to do this with our technology, right? We need to make sure that it matures with us as we mature. And that means making things accessible, because you may or may not be able to hold the controllers, you may or may not be able to hear, you may or may not be able to see, you may or may not be able to move your head as much, right? And so all of these things need to be thought about. And some people will say, well, it's not good to, you know, think about empathy in the terms of getting older, but I don't know how else to have people think about it other than thinking about themselves. Because if your knees hurt, if your back hurts, if you're wearing glasses, you're wearing headphones all the time, you're staring at a screen all the time, guess what? Things are gonna happen and things are gonna change and you're not gonna have the same experience as your whole life.
[00:28:42.138] Kent Bye: Yeah. Yeah. That's really interesting that as we see the experiences that are there now and just trying to make it for the future, I haven't thought too much about that. So that's interesting. So you, you had the seven principles of universal design of the equitable use, flexibility in use, simple and intuitive use, perceptible information, tolerance for error, low physical effort, and size and space for approach and user. I'd love for you if you could reflect a little bit on these seven universal principles of user experience and how they might start to be applied to XR.
[00:29:15.378] Reginé Gilbert: Yeah, so I mean equitable use is, how can this be equal for people to use right I think one of the beautiful things about XR is that, you know, I can use a head mounted display or I can experience VR on the web, right, a lot of people. When I say, "Have you tried VR?" And they say, "No." And I said, "Well, go to Mozilla Hubs and you can try it and you can try it for the first time." So there is some equitableness in the fact that you can access VR in different ways that so many folks don't realize that they've used Instagram filters and they're using augmented reality. They just don't know because they're like, well, it's just a filter. No, it's actually not just a filter. Or when you watch the game right you watched a football or baseball game and you see it right so we've kind of seen it. Flexibility and use, I would say, is providing options in terms of can I just use. one controller, right? Can I use a switch? Can I use different things? Can I use eye tracking to use something or, you know, gestures has some ways to go, but it's an improvement, right? So that's allowing flexibility and use where I can use the controller, I can use gesture. Simple and intuitive use. means I can just hop into an experience and I don't really need much introduction or that you offer some sort of introduction, like onboarding for me. Perceptible information. Are you allowing me to understand what I can perceive in front of me, either through something visual or audio? Tolerance for error. Can you tell me when I have messed up? I mean, There's so many things where you just get stuck and you're like, "Forget this. I'm just going to go back to the main menu." Right? So if I have done something wrong, can you tell me that I have done something wrong? Low physical effort. Don't make me work hard. I mean, just the other day I was talking to somebody and they were talking about picking something up in VR and they were like, it was so hard. I just wanted to throw my controller. So, you know, is it easy for people to do things or are you making them do a lot? And then size and space for approach and user. I will refer to myself living in a small Brooklyn apartment and trying to do VR and not trying to hurt myself, right? So I think a lot of experiences were made with lots of space and not for, this is something my grad assistant and I both commented on because we were like researching design patterns this past summer and hitting walls, you know? Not great. So it was like, what was this experience made for a small space? Not exactly. So, yeah.
[00:32:00.707] Kent Bye: Yeah. And I noticed in your book, you were doing all sorts of different surveys of different types of user experience methods, everything from usability labs, ethnographic field studies, participatory design, focus groups, interviews, eye tracking, usability, benchmarking. I mean, as you are listing out all of these different existing user experience research methods, do you feel like that as you start to dig into more XR-specific usability, that there's a need for either expanding upon some of these methods or having other degrees that you're able to take into the full context of how people are moving in these worlds, or if there's new opportunities for having even more data that's coming back that's going to be able to discern, because I know there's like Fitts' law and other things that people look at in terms of 3D UI of like the efficiency and ways in which that people are evaluating, is this method better than this method and trying to quantify that. But as you think about these user experience research methods, what do you foresee as you start to look at XR, if there's going to be new opportunities or how those might change?
[00:33:05.010] Reginé Gilbert: I think there's going to be new opportunities. It's the same for designers who I am encouraging anybody listening, learn 3D, learn 3D. So thinking about 3D and thinking about space is completely different than if you're in a 2D environment, right? And the way that you evaluate it is going to be different. And that's part of what I'm looking into now is "What makes it different?" And what does it mean for us to create these things and evaluate them? How do we evaluate them? because a lot of what's happening now is things have become a lot more mainstream and a lot more people are using them. There is that data of people using things and things that work really well and things that don't work at all. So I think there's lots of opportunity for kind of some new ways to evaluate this type of technology.
[00:33:53.745] Kent Bye: And in your next steps of looking at usability research, where are you going to start in terms of user studies or trying to evaluate solutions or what's next for you in that realm?
[00:34:04.168] Reginé Gilbert: Yeah, a couple of things. So I put in a request to work with some folks from the national federation of the blind. And so I will hopefully put that solicitation out there just yesterday. So I'm hoping soon to work with some folks and learn more about screen reader use and web VR. And then also I am going to work with a group of students on doing some heuristic evaluation of augmented reality and seeing what we learn from that. And then working on creating some sort of, based on what we learn, some sort of experience. A small one, not a big experience, but create something quite small.
[00:34:46.527] Kent Bye: Yeah, as I was looking through your book, there's a number of different frameworks and heuristics and things. There was some principles of inclusivity that I wanted to read out and have you comment. So these are some groups from the Paciello group for inclusive design. So thinking about generally principles for inclusivity, providing a comparable experience, consider the situation, be consistent, give control, offer choice, prioritize content and add value. So I'd love to maybe hear some reflections on those principles of inclusivity.
[00:35:15.707] Reginé Gilbert: Yeah, I think, one, I'm a content-first kind of person. This is what I teach in design. What is your content? And how are you making that content accessible? I mean, I have my students do content models first because I think that's important to think about how people are going to navigate through, how people are going to navigate through spaces. And so, yeah, there's so much involved with inclusion that people don't think about except who are you leaving out is the question. I often ask, because when people will propose and present ideas, I say, "Well, what about this or what about that?" And they said, "I never thought about that." And that's why I think one of the things I say to my students is that design is a conversation. And so many folks create really, really cool things, and they're really cool things, but they don't really put them out there in the world. To get that feedback from multiple people, you can't just get it from your friends. You really need to get it from people who are not like you.
[00:36:13.754] Kent Bye: And maybe you can tell me a little bit more about your book that you're working on with Doug North Cook and how that came about and what you're going to be trying to focus on in that book.
[00:36:22.317] Reginé Gilbert: Yeah, I'd love to. Um, so Doug and I met because I applied for a design residency at Fallingwater, which is a Frank Lloyd Wright house. The design residency is for folks who work in immersive technology. So there were 10 of us that were accepted to, I think we were the last program before the pandemic hit. This was November of 2019 and it was five days of being offline but also creating and working on immersive ideas and concepts, but we just use whatever they gave us. So we had cardboard and my group that I was working with, we were given 30 minutes to create a game and they gave us rope and toys. And we ended up creating a game where people were tethered together and stuff. So it was really cool. And being offline actually helped us be more creative. And out of that, Doug and I became friends and collaborators. I worked with Doug on co-creating human-centered design course for Chatham University that I think, let me say I'm getting all these years confused, but it was, I think 2020 was when the course was run. And I was mentioning to him that I wanted to write about accessibility in XR. And Doug has those, and when you showed the universal design principles, I was like, that's Doug's chapter. Doug has a great interest in universal design, just in general, and spatial computing. And I would say philosophy, because he has such an interesting perspective. And so we came together and decided to put together a book proposal, which was accepted by Oxford University Press. And yeah, the title, the working title is Human Spatial Computing. And we'll specifically be looking at human computer interaction, but from a global perspective. Privacy and ethics is the second chapter of the book, and it is overarching theme in some ways of the book, which is why I appreciate you and your work and the things that you've done. We'll be talking about what it means to be human, which is something I've surveyed over the years with my students. What Connects Us All, which is storytelling, which I'm looking forward to telling stories of folks using VR in different parts of the world, not just America and Europe. We'll be looking at universal design, affordances, and immersive tech, the mind and the body, and what the future holds.
[00:38:49.314] Kent Bye: Oh, wow. That sounds like a book that's right up my alley on so many different levels. That sounds amazing. Yeah. And I've had a number of great conversations with Doug North Cook over the years. And so I'm glad that you were able to go to that. I'm jealous that I've never been able to be to that location. It looks like a really, you know.
[00:39:05.667] Reginé Gilbert: You have to go. Yeah. And Doug and I are very lucky. We get to be scholars and residents at Fallingwater this year. So we get to go back.
[00:39:14.337] Kent Bye: That's great. Just the beautiful architecture and that just amazing scene. It looks like a place for those different types of conversations. It's really cool. Uh, well, as we start to wrap up, I'm curious what you think the ultimate potential of virtual reality might be and what it might be able to enable.
[00:39:31.803] Reginé Gilbert: Oh, so I think that there is such great opportunity in the XR space in general, for folks to experience things they never have to go places they never have. One of the things that my students always work with a real life client. And a few semesters ago, they worked with an organization that was doing web VR for older adults who were in a home. And it would be a volunteer and the older adult and they'd get on the computer and they would actually travel to like, if you want to go to Tokyo and they would go together and this web VR experience. Right. And it made folks less lonely. And just the other day, I had a conversation with someone who is working with children in hospice and VR. So I'm like, this is all like, there's some really awful stuff as I'm learning, but there's also some really good and I'm going to focus on the good. I think there's a lot of opportunity for us to create good experiences. and to make bridges in places where they didn't exist before, to bring people together in places where they wouldn't have come together before. So that is my hope that we are able to bridge those gaps and also make sure that we are incorporating folks with disability in the process, in the development, in the design, and making sure we're not leaving anybody out and not leaving out the poor either, because that's something that we don't talk about enough.
[00:40:59.583] Kent Bye: Awesome. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[00:41:04.646] Reginé Gilbert: Oh, well, thank you for listening. If you stayed on this long and please go and make accessible experiences and you want to use this stuff when you get older, right? So that's what I say.
[00:41:17.713] Kent Bye: Awesome, well Reginé, thank you so much for joining me today on the podcast to be able to talk about some of these frontiers of usability and accessibility and user experience design for XR and very much looking forward to this book that you have coming up and yeah, continuing the conversation. Cause I think this is, you know, really talking about the frontiers of this field. We've done a lot of work of just making the technology work. but doesn't quite work for everybody just yet. So still a lot of work to be done. And I think, especially if you are in this field, there's a lot of opportunities for research and also design innovation and finding new techniques of making them more widely accessible for people to start using them. I know that XR Access and the XR Association have created a GitHub for different repositories and there's other resources and. And yeah, the IEEE has a paper that has some more resources that I'll have an interview with some of those authors of that paper as well coming out here soon. So hopefully more resources available for people, but yeah, still lots of work to be done.
[00:42:15.548] Reginé Gilbert: Yeah. Yes. A lot more.
[00:42:17.630] Kent Bye: So yeah. Thank you again for joining me on the podcast. Thank you.
[00:42:20.492] Reginé Gilbert: Thank you so much.
[00:42:22.043] Kent Bye: So that was Reginé Gilbert. At the time of this recording, she was an industry assistant professor at New York University and the Tandem School of Engineering, and has since been named as a James Weldon Johnson professor at NYU. She's also the author of a book called Inclusive Design for Digital World, Designing with Accessibility in Mind. And she's working on a book called Human Spatial Computing that's going to be published by Oxford University Press in collaboration with Doug North Cook. So, a number of takeaways about this interview is that, first of all, Well, there's a lot of different overall heuristics and guidelines and design patterns that I think are worth calling out. There's the Web Content Accessibility Guidelines, the WCAG, which is based upon the POUR, which is the perceivable, which is providing text alternatives, operable, so the different components in navigation, the understandable, so having error identification, and then robust. And so within the WCAG guidelines, there's actually a draft that was just published in May. That's the 2.2 version of WCAG. that is starting to break down each of these specific things of how this gets translated into the 2D web. Mostly text-based, but also how to deal with rich media, with transcripts and alt tags and whatnot. So that's a good baseline to start to see how you can start to apply some of these web content accessibility guidelines into VR, but there's certainly going to be a lot more innovations when it comes to other modalities and other heuristics as well. Some other things that I think are worth elaborating again, there's the seven principles of universal design, making sure that you have equitable use. So for folks who may have different types of disabilities, and including auditory, cognitive, neurological, physical, speech, and visual impairments, that the experiences that folks have of the experience is as equitable as you possibly can. So having Audio cues, text descriptions, captions, all these things that if there are any different impairments, then how do you make it so that the immersive experiences and equitable experience for them? Principle two is flexibility and use. So providing lots of options. If you only have one controller or using like eye tracking or gestures, principle three is simple and intuitive use. So can you just hop into the experience and you don't need any intro. You don't need anybody who's cited to help onboard you and everything. You're just able to be completely autonomous and be able to do what you need to be able to do within the experience. from end to end, which I think there's still a long way for XR to get to that point. Principle four is the perceptible information. So understanding what sensing via sound or other visuals. So having a variety of different sonifications or visuals or captions or haptics to see that there's lots of different ways to perceive information that you're trying to communicate. Principle five is tolerance for error, giving feedback when things go wrong. The sixth principle is low physical effort. So making easy for people to do things. And the seventh principle is size and space for approach and use. So if there's a small space that you have in the context of virtual reality, then just making sure that you have alternatives that you can use a joystick and click turn and other ways that you can have a seated experience as an example. So that's the seven principles of universal design. I just also wanted to give a shout-out to the seven principles of inclusivity from the Pacilio group. Number one, providing a comparable experience. Number two, consider the situation. Number three, be consistent. Number four, give control. Number five, offer choice. Number six, prioritize content. And number seven, add value. Those are some of the different heuristics that I just wanted to call out. And yeah, I just really appreciated Brezhnev's commitment to XR accessibility as a design priority. I had a conversation with Daniel Lufer talking about artificial intelligence. And one of the things that he was saying is that oftentimes there's utilitarian thinking that happens in the context of building these different technology tools where you say, well, it works for most of the people. And sometimes the people that it doesn't work for, or in the case of artificial intelligence, where there may be implicit racial bias within facial recognition, and then the side effect of where it doesn't work may be actually sending people to jail. And so with the AI Act, they're trying to actually take a completely different approach than, say, utilitarian approach. trying to take more of a human rights approach and deontological approach where you're prioritizing certain sets of rules in order to make sure that you're creating a more just and equitable technology rather than this type of utilitarian thinking. And so because of that, then you actually have to look and consider all these different populations that may not be being served by this utilitarian type of thinking. And Myself, I've suffered from this type of thinking in terms of not having the most accessible features of my podcast of not having transcripts from all my backlog. I've been trying to use some of the different AI technology tools like whisper X to get transcripts with diarization. So you have different speaker changes. There's still a lot of cleanup that I need to do, but with this series, I'm launching the whole backlog of transcripts that you can go back and see previous episodes and start to look at some of the different. Conversations that I've had and hopefully add more search features and whatnot So just within the context of my own work that I've been doing trying to make this podcast more accessible Also adding more tags and categories. So trying to make it so that you can search and find information a little bit easier So just making the content that I already have been recording over the last nine plus years to be more accessible and discoverable findable and whatnot and just more categorized and yeah, just trying to make it easier and multiple channels for people to connect to some of the different information. You know, one of the requests that resonate had for me was that I was including a transcript. And so, um, in the process of launching the series, I'm having transcripts for all my backlog. Like I said, there's still a lot of work that needs to be done to clean it up and everything, but at least I think that the rough transcripts are going to be better than having no transcripts at all. But this is a theme that I think comes up again and again, in terms of this type of utilitarian thinking that has been pervasive within Unity or Unreal Engine or just XR industry in general. There are some VRC requirements. These are the virtual reality checks within the context of if you're going to get an app into the MetaQuest store, then you have to meet some of these different minimal requirements for accessibility. But in talking to a lot of these different researchers at the XR Access Symposium, there's still a long way that needs to go. These are kind of rough guidelines, they're not strictly enforced. And also, there's still a long ways to go to be able to make these different immersive experiences fully accessible for folks who are blind or low vision. I'll be having a conversation digging into Cosmonious High and Alchemy Labs to talk about some of the stuff that they've been doing. But yeah, like I said, there's still a long ways to go, even to bring some of these different features, either at the platform level. So either at the MetaQuest operating system, bringing in Android accessibility features, Apple Vision Pro is going to be bringing in lots of different accessibility features, but also with Unity and Unreal Engine, just the way that these engines are producing the content isn't always accessible by default. So there's a lot of work that has to happen on top of that in order to add some of these basic accessibility features. So some of the work that Reginé alluded to in terms of the different heuristics, we will have a conversation during the poster sessions, unpacking that a little bit more. And there's a number of different guidelines that I'll try to point folks to as we move forward. Like I said, the XR access in collaboration with the XR association has a GitHub where they have links to a lot of different guidelines and translations of these different universal design principles. But I think this is an ongoing process in terms of not only trying to come up with some of the different guidelines and back practices, but there's going to be a lot of work that needs to be done for looking at a variety of different contexts of different types of applications and the types of user research to interact with folks with blind or low vision, deaf or hard of hearing, or other cognitive impairments, and just generally making these existing immersive experiences more and more accessible by just doing user testing, which is a lot of what Régéné is advocating for, which is to do the different types of user testing and being in collaboration, being in communication with these different populations and just listening to their feedback and starting to make changes in a minute. So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and I do rely upon donations from people like yourself in order to continue to bring you this coverage. You can become a member and donate today at patreon.com slash voicesofvr. Thanks for listening.
