#97: VR video experiments with eleVR’s Vi Hart, Andrea Hawksley, & Emily Eifler + Diversity in VR

IMG_20140920_094804

eleVR is a three-person research and design group doing a number of different spherical video experiments as well as developing their own WebVR-compatible web video player. eleVR includes Andrea Hawksley as the developer, Vi Hart as the director, & Emily Eifler as the producer.

eleVR is doing some of the most cutting edge and innovative content experiments that I’ve seen, especially the WebVR integrations that I think are going to be huge.

They also discuss some of their concerns about diversity within Virtual Reality with what they see as a gross underrepresentation of women at the invite-only Oculus Connect developer conference. Emily also talks more about the sexual assault that she experienced at Oculus Connect.

Some of measures that the larger tech industry have been implementing to help prevent this are things like a Code of Conduct that outlines as well as diversity statements like the one implemented by O’Reilly Media.

I also mentioned the work of Ashe Dryden who has written a couple of really great blog posts about fostering diversity at tech conferences. In this post on “Increasing Diversity at Your Conference” she says:

The easiest way to get feedback on your efforts is to publicly state what you’ve tried and ask for constructive criticism. Be transparent and truthful. I’ve seen many conferences write blog posts about what they’ve done to address the issue of the lack of diversity and the positive or negative results that they ended up with. This is important for a few reasons: it signals that this is important to you and that you are open to more ideas as well as letting people within marginalized groups know that you are considering their needs and the reality of their situations.

Here’s another great excerpt from a post titled “So you want to put on a diverse, inclusive conference”

How do you advertise that you want to see a diverse community at your conference when you don’t already have one?

  • Admit you have a problem. There is nothing wrong with going to colleagues or to twitter and saying “We want to provide an inclusive, diverse conference experience, but we need help. Can you help us?”
  • Explicitly ask for constructive criticism. Write a blog post on your conference’s site explaing what you have done and ask where you are going wrong or what you might have forgotten. Maybe you didn’t notice that all of the pictures on your conference site are of white people or that the language you use in your CFP is gendered.
  • Be gracious, humble, and kind. It’s hard to hear that you may have misstepped or made a mistake, but it happens to everyone. Before responding to criticism (constructive or not), take some time to examine the truth in it. For best results, ask an unbiased third party to examine the evidence and the criticism and help you understand the problem. Then, humbly apologize and make known the steps you’re taking to correct the situation.

I’d agree that there’s a lot that the VR community can collectively do to help foster more diversity, and I’m really glad to see that Oculus is starting to take steps towards being more deliberate about diversity concerns. They now have a Diversity Lead with Brandi House, and tonight at GDC, Oculus is co-sponsoring a party with Women in Games International. The Eventbrite page says,

VR is still very young, and now is the best time to define what’s possible. To help VR reach its enormous potential, we need a diverse and talented community of developers to make it a fun and engaging experience for everyone. As part of that effort, we want to welcome and support women developers both to the VR community and to the Oculus team.

I’ll be there tonight and look forward to meeting and featuring more women within the VR community.

Theme music: “Fatality” by Tigoolio

Subscribe to the Voices of VR podcast.

Rough Transcript

[00:00:05.412] Kent Bye: The Voices of VR Podcast.

[00:00:12.008] Vi Hart: I'm Vi Hart, and I am the director at LLVR.

[00:00:15.391] Andrea Hawksley: I'm Andrea Hawksley. I'm the developer.

[00:00:18.894] Emily Eifler: And I'm Emily Eifler. I am the producer. LLVR is a three-person research project that is hosted by the Communications Design Group, which is a research group funded by SAP. So our project is funded by SAP. And we do spherical stereo video, and we make our own web players. So we're focused on doing web video VR. In April, we started with a stereo pair of cameras, and we worked up to spherical. And we're now working in spherical stereo, and we're experimenting with different kinds of content. And we've tried stop motion, and we're now doing an episodic talk show. And we've done some art video projects, and we've tried to do everything across the board.

[00:01:00.749] Kent Bye: Awesome. And so maybe you could give me a little bit of context as to where you're coming from with your video backgrounds.

[00:01:07.653] Vi Hart: Yeah, well, Emily and I both come from a YouTube background, so our focus is on releasing content-y content. We don't care too much about production quality, but just being able to express ourselves and get our stuff out there. So when we realized that the flat rectangle is a dead medium, we decided we wanted to figure out how to do this in spherical in VR. And so we did.

[00:01:28.931] Kent Bye: Interesting. So maybe you could just give me a little bit of a background and flavor as to what type of content you were doing on each of your channels.

[00:01:36.440] Vi Hart: on my channel, mostly math videos and microwave videos.

[00:01:41.183] Emily Eifler: And I do a culture and technology show that attempts to both talk about culture and technology, but also help reveal to the viewer the veil inside of which they watch online content.

[00:01:52.930] Kent Bye: I see. And so what have been some of the insights of moving from a 2D to a fully immersive 3D virtual reality environment when you're doing this type of content?

[00:02:03.599] Andrea Hawksley: the tripod and if you have like a big team of like people like standard TV production you're gonna be able to see all of them if you don't do something about it so you need really need people to be hiding or you want to have it be an experience where hey you could turn around and see the producers and maybe that's actually kind of cool maybe you want to do that.

[00:02:23.205] Emily Eifler: I think one of the things that I've been thinking about recently is, in film school, we talked a lot about framing, and how the director's eye is interpreted through this frame, and the close-up, and the two-shot, and the big setting, and all these things, and how all of those different ways of shooting lead an audience through a story. Well, there's no frame anymore. So when you throw away the frame, what is the new technique that leads an audience through? Right now, we've been thinking a lot about VR videos as places. People sometimes come up to us after they've seen one of our talk show videos specifically and are like, I have been in a room with you. I have talked to you. You did not talk back, but I did try and talk to you. So that has been something that I've been thinking about a lot. If you, you know, thinking about the sphere of a place and how the setting of a place can tell a story more than the director getting to choose the frame of a video.

[00:03:18.275] Vi Hart: coming from a more informal YouTube-y background, we don't want to hide the production of what we're doing. We're focused on the content. So we will point out our stitching errors, we'll talk about the technology that's going on right now, and we find that works really well.

[00:03:35.198] Kent Bye: And how are you distributing these videos? Is YouTube a medium in which you can do these spherical videos that people can then put through some process to be able to see?

[00:03:44.035] Andrea Hawksley: So the YouTube right now doesn't let you actually do anything to your video to actually view it in a sphere. So I actually made a player that will let you go on experimental Firefox or Chrome that has WebVR support to watch it on your VR headset. And then right now you can download all of our videos off our website. We're using BitTorrent. And also if you go to our player, there's like a couple web videos that are already there just streaming if you don't want to have to download stuff, although I need to update that.

[00:04:21.904] Kent Bye: I see. So it sounds like you're using some of the new WebGL, sort of cutting edge, you know, browser-based virtual reality. It sounds like it's going a little bit to be able to stream the video, but also give people the ability to have a full screen with head tracking to be able to see these videos. It sounds like where you're going with that.

[00:04:40.355] Andrea Hawksley: So because we came from a sort of YouTube web video background, it was really important to be able to show our videos on the web. And when we started, there were no web players. So actually, one of the big projects when I started was to make something so that we could watch our videos on the web like we were used to. And really, where we think VR is going to be going is that people are going to want to find content. You don't want to sit there and download the video. That's not how people are used to doing it these days. They want to be able to stream video. So we wanted to have a system where you could actually stream videos. possibly on the web. And we've talked to the people who are also working on the experimental web VR browsers at Firefox. And they have a mailing list. And so it's been really interesting getting in on this very beginning of web VR. And it's only going to get better. It's been like maybe two months in production.

[00:05:29.975] Kent Bye: Yeah, I guess one of the challenges of doing a spherical video is that with the DK-1, you could just sort of turn your head left and right. With the DK-2, you're having all sorts of positional tracking. And so how does positional tracking with the DK-2 work when you're using something that's static and doing a video in that way? How do you resolve that?

[00:05:49.091] Emily Eifler: Don't use it. Turns out you don't have to actually use that IR tracker, just use the DK2. So currently, for our just playing a video, you can just use a higher resolution screen and the more comfortable headset, or at least for me, it's more comfortable for me, and play a video with a better view. But we do have some applications. For example, we made a thing called Video Bubbles. It's a Unity environment in which you can use a controller to walk around inside of an environment and walk into and out of large spheres that have our videos textured onto the sphere. So it's something between a game and a menu that you move around inside of. So that's an example where, like, if you move your head side to side, the environment around you would change, but if you're inside of the bubble, like, your relationship to the video wouldn't be changed by the IR tracking.

[00:06:38.882] Vi Hart: Yeah, and this is something where a lot of people are focused on having things in VR be as active as possible, taking advantage of all this stuff, and we like that. For video, we're also exploring more passive experiences where you can get sucked into the story without worrying about, where am I going to put my head? Do I have to move my head around? So this is different ways of consuming content. And now with VR, we're going to find even more ways of being consumers, where it might not be all about having to move your head around. Some people actually find that more nauseating or more uncomfortable, or that it detracts from the immersion that we're looking for in our videos that are very content-focused. When you don't have to worry about all of the head motions, you can just look around you and see what we put there.

[00:07:21.939] Emily Eifler: We find the user interface of a conversation where you naturally look at one person and then naturally turn your head to look at another person and people's necks are really used to doing that and people are really used to that experience so when you put that in a video it immediately helps you feel immersed because people are not used to dogfighting or someone told me yesterday they played Sightlines the Chair which I thought was a really interesting demo and a great way of using attention as a resource inside of a VR game but also they said like the next day my neck was so sore because it required like so much looking around all the time.

[00:07:54.795] Kent Bye: I see. Yeah, and I guess that's another component is the actual audio and the sound. Are you just mixing one track of audio or is it are you finding a way to do omnidirectional binaural recording so that you can do head tracking and actually have the audio change when you're looking around?

[00:08:09.982] Vi Hart: Yeah, first of all, pet peeve, omnidirectional binaural recording is not actually a thing.

[00:08:15.916] Kent Bye: Well, I just talked to a guy who's doing it, so it's so cutting edge and new that... I just wrote a blog post all about this.

[00:08:21.740] Vi Hart: You can record binaural audio in all directions, but that does not make an omnidirectional binaural recording. What you need to do is record a sound field. And that's a mathematical thing that you can't really replace with fancy hardware. You need to get the information out of the sound field, and then do some math and render it back. So, yeah, we're just starting to experiment with that. I guess, like, ambisonics is probably going to be the standard, I think. Yeah, a lot of people are doing different things to play with it and making very interesting experiences, doing really awesome stuff. But in the end, you just have to use some real math. And that's, I guess, why we're having so much fun as a research group is that we do the math.

[00:09:02.961] Kent Bye: Nice. And so in terms of the content, I guess the biggest challenge is directing attention. You know, how are you directing attention in terms of what they should be looking at in these experiences?

[00:09:13.234] Vi Hart: by looking. Like in our talk show, for example, you know, if we both look somewhere, you're going to look there. That's natural. In some ways, we're trying to invent a new language of storytelling. But in other ways, we already have cues. We already live in a space that is like what a virtual environment is trying to replicate. So we already know all sorts of cues of where to look and how to look. And sound is definitely going to be important in that. But so is other people's gaze.

[00:09:39.576] Emily Eifler: I mean, yeah, pointing fingers, looking, indicating is easy. But another thing is just like how you set up a room, right? Like, all of VR has a forward facing bias. So everything is set up so that when you sit down and look forward, that is the alignment that everyone is expecting you to look at most of the time. So If you wanted to have the person change the alignment, you have to consider how much of their torso are they going to have to torque around to look at something for a really long period of time. So if you put something at 12 o'clock in front of them or six behind them, you have to consider how much energy, what the energy cost is of all of those placements. And it's totally fine to put something back there, but you have to maintain it for a shorter period of time than something that's at noon. And then I would also say that there has been a lot of things I've seen where someone walks all the way around the camera, and I'm like, Yay, but why? Just because you have a spherical 3D environment doesn't mean you actually have to use every single pixel. Like right now, for example, we're standing on this platform on the high level of a mall, and there's tons of things behind me, but I don't care. I'm not constantly looking in every direction so that I can see everything. I'll glance at you, and then I'll glance at Vi, and oh, there's a giant elephant. I can naturally use parts of the space for attention and totally ignore the other parts of the space, but also be happy that they're there. They help set the environment. And if I happen to glance behind me, then there's something there to fill out the experience. But it doesn't have to be the whole experience.

[00:11:08.912] Kent Bye: And from a development perspective, you mentioned a little bit about the web VR, but I'm curious if there's any other sort of big thing that you're looking at in terms of the big open problems that you're trying to solve.

[00:11:18.848] Andrea Hawksley: The biggest problem right now with video playback, and I think actually a lot of things, is just that you need to get the frame rate up, you need to get the speed up. For WebVR right now, the idea is that the browser will know about what headset you're using and what distortion should be used and will do that, but that means that you have a two-pass process and that second pass is slow right now. So you end up like, it's more than twice as laggy to do that. From a development perspective, once we start doing audio, that's going to be a big challenge to use that positional audio. Right now, we have our videos encoded as MP4 or WebM. And the audio track, as far as I know, doesn't give you sound field information. So it's going to be interesting to figure out how to do that and play that back. The big challenge with video is really that you have a projection and you're putting it back into a sphere and showing a certain field of view of that sphere. And right now, all of our projections are equirectangular. And one thing that we'd like to experiment with would be other projections. For instance, the cube projection, because graphics cards often are all about cubes, and just cubes like equirectangular, I have to use an arc tangent in order to turn it back into a sphere and get the correct pixels, whereas a cube would be much faster, like you don't have to do this expensive computation. So we're really interested in looking into the use of different projections to store our video. It might give us less distortion and also be faster.

[00:12:51.612] Kent Bye: And what is the SAP connection? How did that come about? It seems a little bit of a leap of, you know, why SAP would be interested in helping fund something so innovative and creative.

[00:13:02.055] Vi Hart: Basically, this is the brainchild of Alan Kay. Alan Kay is a person who likes research groups that do cool things and was part of one for a while.

[00:13:11.561] Emily Eifler: He was a Xerox PARC researcher, which is what most people know him from.

[00:13:15.404] Vi Hart: Yes. So basically, he wanted to put together an awesome research group that had a lot of freedom to be able to explore whatever we want. And he did. And now we have that freedom. And so we decided we wanted to do VR.

[00:13:28.953] Kent Bye: Awesome. What do you guys see as the ultimate potential for what virtual reality could provide?

[00:13:34.720] Emily Eifler: There is no ultimate potential. There is no one thing. That's like saying, what is the ultimate potential of the web? It's like, well, do you like shopping? Do you like, I don't know, conference calls? Like, what are you into? So there will be something for everyone. And there will be someone's favorite thing will be what they consider the most epic thing ever made in VR. But I mean, for me, maybe having I don't even know what the best thing would be. There's just so many options.

[00:14:04.016] Vi Hart: Yeah, like, what is the ultimate potential of reality? Because whatever that is, virtual reality is going to be able to do it better.

[00:14:10.362] Andrea Hawksley: On the note of things that maybe aren't the ultimate potential, but that I think are very cool and something that's being soon rather than later, would be if you're someone like me who has an hour-long train commute every day with a bunch of strangers, being able to take out your headset and just like watch videos and ignore everyone around you. Not that I'm anti-social or anything but sometimes when you're really crowded and you just want to be in a different space that to me is like the thing that's going to be the next literally like months maybe before people start actually doing that seriously. So certainly not ultimate potential but soon and something that I think will make a big difference to quality of life for something.

[00:15:01.465] Vi Hart: On the plane on the way here, I was definitely in there reviewing our videos, doing some last minute stuff on the way to Portland. That was a conference before this one. We've been flying a lot in the past week. But yeah, flying with a VR headset on, it's really great.

[00:15:17.152] Kent Bye: Just to kind of reframe that question, is there anything that you can't wait to experience within virtual reality?

[00:15:22.255] Emily Eifler: The thing I'm most excited about right now is actually something that we're just now being able to do, which is have a video in which there is just positional interaction, so being able to tell the player, like, there's a door right there, if the person looks at the door for long enough, put them in this other place, and have that not be inside of, like, the really, really slow Unity environment, because Unity's so slow in VR, and there's so much judder that having it, being able to do really simple interactions like that, just in the player, is, like, what I'm really excited about, just, like, today.

[00:15:51.701] Vi Hart: Self-expression. We're finally at the point where we can actually come up with an idea for a video and produce it and be fairly certain that we're going to get the result we want. We're a little bit at that point, at least for spherical stereo video. We kind of know we can do it. So being able to use that power to kind of figure out what we want to do ahead of time and spend some actual time creating a video with a real goal in mind that's more than a demo, more than an experiment, but actual self-expression.

[00:16:18.388] Kent Bye: Awesome. So maybe you guys could comment, and in terms of being here at Oculus Connect, you know, there's a distinct lack of diversity, you could say, in terms of the gender balance here. And so I'm just curious, as being a super powerful team of women doing some really cutting-edge virtual reality research, how you feel coming into this environment, but also what needs to happen in order to bring more diversity into virtual reality?

[00:16:42.908] Vi Hart: Yeah, and we happen to be in a room that has a giant elephant in it, I would like to mention. Like, actually, literally. So I'm often surprised to have people be surprised that our team of three just happens to be all female. And I'm surprised when people say, oh wow, it's great you're a woman in VR, because there's not a lot of them. Because there are quite a few women in VR. I know a lot of women in VR. doing amazing things and I think it's unfortunate not to see them at this event, not to see them speaking at this event. I suggested many of them to the organizers that I think should be speaking at this event. Maybe a lot of other people got wind of the already gender ratio apparent in the organization of this event and chose not to come. I don't know what it is. I think part of the problem at events like this I don't actually know the organizational process because it is an invitation-only event. So to see an invitation-only event that has this kind of ratio is extremely disappointing and I know it does not actually reflect the gender ratio in the actual VR community.

[00:17:43.355] Emily Eifler: They took three months to organize a conference, and this is what you get. Also, I would just like to say that, like, it's great. Diversity is really awesome, and I want that to happen. And also, like, it's not just about, like, yay diversity. It's about, like, diversity makes better products, and it also makes better environments where you don't get groped before the conference starts. Because that happened to me yesterday, and I was pissed.

[00:18:04.808] Kent Bye: Oh, God.

[00:18:05.469] Emily Eifler: Wow. Yeah, right?

[00:18:09.651] Vi Hart: God damn it. Yeah, and it's just so crowded here that it's like, who do you punch? You don't even know because there's such a mass of bodies that are for the most part male. So it makes it tough to respond to that kind of thing.

[00:18:25.714] Kent Bye: I think this is an issue that's been sort of widespread within the tech industry and that there's people like Ash Dryden and a whole geek feminism wiki that talk about like certain standards and procedures of having a code of conduct, having an open hall for presentations and more open than sort of an invite only. So there is some concern I think in terms of having it curated in that way and have that big of a blind spot being the worst aspects of the imbalance of the gender ratio within the games industry is just sort of being bubbled up to this is what we get in terms of not having as much diversity as I think there could be or should be.

[00:19:02.417] Vi Hart: And as much as there is in the actual industry, this does not reflect the actual industry and the people in it. So that's the most disappointing part to me is that, you know, before this event I did contact the organizers and I was like, hey, maybe you, you know, I know it's hastily organized. It's tough to organize an event like this in a very short period of time. And it makes it easy to let a lot of things fall through the cracks. So I did make an effort to contact them and, you know, try and give them some options and help them out. And that doesn't seem to have actually done anything. So that's disappointing. It's hard to know what you're supposed to do next.

[00:19:34.188] Kent Bye: Well, who are some of the other women in virtual reality that we should know about?

[00:19:38.130] Vi Hart: Well, in film, like, number one is Noni de la Pena. Okay, yeah, you're nodding, obviously, right? She's huge. She's awesome. So that, for us, is, like, yes! Kate Compton who invented the Seabright augmented reality AR VR helmet. She's super cool and she's local.

[00:19:57.851] Emily Eifler: We also know lots of developers who work at Oculus or on Oculus projects and it's always surprising when you meet a new person and you're like, oh friend! Yes! You get very excited.

[00:20:11.078] Vi Hart: Yeah, there's a lot of people we could name, but I always feel weird about like starting to name people and be like, oh, here's a person who I told, you know, I don't want to like to be, here's a person who I told Oculus they should invite and they didn't because I don't want to single out that person or any of those 10 people and, you know, make them feel bad that now they know that not only were they not invited, but they were specifically not invited. So that makes it a very tough question to answer. And then you throw in some NDAs and that is a sensitive subject.

[00:20:39.522] Andrea Hawksley: I'd like to know like sometimes when people are like well can you name all the women doing this and it's like a little weird like you want me to name all the men doing it like it's a weird thing even like people will use a single woman as an example of all women CEOs or like this person is like if you're not just like this person you'll never make it in whatever and To me, the VR is very interesting because I'm a software developer. I mean, I went all the way through college. I got a computer science degree. I've always been in environments that have been mostly male. And I was working at a software development company before this that didn't have a good gender ratio. But compared to that, this is like so much worse. like a bad gender ratio or like borderline acceptable at like a lot of as a software developer it would be like 10% and this is not 10% this is like 1% maybe women at this conference. And I think that when you have a bunch of guys together, it also affects the tone of the conference in really sort of weird way, like where if it's 10% women, then it's even different tone than if it's 1% women. It's just been so weird to me because out of, I think, our entire group, I'm the one who spent the most time working in environments where is surprising to people that I am a woman or that they assume that I'm the secretary when they come in and they're like, can you help me with this? I'm like, no, because I'm the person you're supposed to be meeting with. And I have a name that is not really gender ambiguous unless you're Italian. And sometimes people still think I'm a guy when they see my name. because they just seem to think it's more likely that as a software developer working in my field, with my name, I would be a guy than a girl. Sorry, I'm ranting now.

[00:22:29.038] Vi Hart: The worst case scenario is that all the people here look around and think, wow, there's just not a lot of women in VR, and that's just not true. And I hate that it looks like that when it's not true. And instead of taking actions to support more diversity at this conference, I feel like it's actually worse than real life.

[00:22:46.597] Andrea Hawksley: Also, the big problem with it being worse than real life is if you want to encourage more women to do it, showing them that, oh yeah, once you go to this conference, you'll be one part of the extremely tiny fraction of women that if you took a random photo of a crowd, there wouldn't be any women in it, and you might also get groped before the conference started. Don't you want to be in this field? It's awesome!

[00:23:05.330] Vi Hart: The field is the only target for groping, so you're going to get every single creep at the conference. Can't dilute the creep when there's no other women.

[00:23:13.888] Andrea Hawksley: Yeah, so I feel like people are like, oh, well, I sort of was trying to get representation in the field, and this is not even that. But even if you were trying to do that, it's just making other people not want to join when you're like, oh, yeah, this is not something that you should want to do if you're a woman. Because I mean, look, clearly there's no women. I didn't invite them. And I know that for me, coming to this conference, I was a little hesitant because I have, at previous VR meetups we've had, had bad experiences. with just how gamer, male, heavy, dominated they are. But I feel like if I don't go, then I'm just making it worse. So it's bad to be like, oh, I didn't really want to go someplace where I would be the only woman, but I didn't want everyone to think there were no women. And that's not even a decision I should have to be making.

[00:24:08.183] Kent Bye: Yeah, I think the code of conduct is sort of like the baseline in terms of starting to at least say these are the ground rules of creating a safe environment and diversity statements. Another thing of saying diversity is important to us. This is what we're doing. Hopefully as a reaction to some of these experiences, we'll start to get some of that in place. But do you guys have any, like if you were to have any desire or wish to moving forward speaking to either the, the larger, you know, Oculus company or the VR community at large, what would your request to be?

[00:24:36.103] Emily Eifler: Set a standard. Say that you won't have a conference unless there's 40% women at your conference. And if you have to invite fewer men for there to be 40% women and you have to throw a smaller conference, then I think you should do it. And I also think that you shouldn't have first-come, first-served tickets. It doesn't work and it always lets all the Redditors in first. And I just think that if you are really committed to diversity, then you should have an application process and you should consider who you invite. I don't think that those things are that hard to achieve.

[00:25:08.002] Vi Hart: Yeah, first come, first serve is the biggest mistake I think a lot of events make. Because all it means is that the already established communities, which are majority male in a lot of the communities, such as the aforementioned Reddit, all that means that those are the people who are going to be first, because they hear about it first. And the people who are not part of those communities are not going to hear about it until after everything's sold out.

[00:25:30.809] Kent Bye: OK, great. Well, thank you so much.

[00:25:33.109] Vi Hart: Yeah, thank you. Thanks. Yeah, thanks for having us.

More from this show