Hallie McConlogue has been waiting for virtual reality to go mainstream for over 20 years now. She used to spend many consecutive hours within a VR design and drawing tool called HoloSketch at Sun Microsystems’ VR lab where she did interface design, modeling, art direction, and animation for 7 years. She worked on the first claymation for real-time playback within a head tracked virtual reality HMD.
I talked with Hallie right after she had tried out the Crescent Bay demo for the first time at SVVRCon. She was marveling at how well the head-tracking from system that she used over 20 years ago has stood the test of time, and even the framerates and resolution were comparable in her mind. Of course, computer graphics have improved exponentially and the cost of the Sun system that she was using was over $53,000.
She talks a bit more about using HoloSketch, which is a program developed in 1994 by Michael Deering. It sounds like the metaphoric grandfather of an interactive painting and design program like Tilt Brush.
One of the things that Hallie is interested in doing is applying the insights that she’s gained from improv acting and comedy over the years. She argues that improv theater actors have been on the vanguard of interactive media for as long as improv has been around. She thinks that improv has a lot of lessons to teach VR designers for how to create a compelling and engaging interactive experience.
Hallie also advocates for the importance of facial tracking, especially being able to track the eyes. Where the eyes are looking can give so much information about how to navigate social situations, and she says that the eyes are a huge feedback mechanism for how improv actors communicate with each other.
Hallie also talks about getting more women involved in Virtual Reality, and provides some feedback about what type of experiences that she’s interested in. She says that romance novels are like porn for women, and that there’s an romantic and emotional component that is worth exploring through the medium of virtual reality. And again, a lot of these types of romantic experiences get back to being able to track facial expressions and to have a strong sense of eye contact between either the audience or characters within a story.
Incidentally, Maria Korolov, the editor of Hypergrid Business, has started a Women in Virtual Reality website aggregating women speakers and professionals who are working with VR.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
Subscribe to the Voices of VR podcast.
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast.
[00:00:12.055] Hallie McConlogue: My name's Hallie McConlog, and I'm working on a little garage startup with a friend. It's a two-generation group of people. One old-timer artist, VR content person from the VR research lab at Sun Microsystems, and the other guy is this fresh Berkeley grad who's hot with the C-Sharp and Unity and stuff. So together, I think we're going to be able to do some really mind-blowing stuff. We're also working on a little bit of tracking stuff, too. And then what I used to do was this thing called Holosketch at Sun Microsystems. And it was an amazing system. It's sitting in my closet right now, dead as a doornail. And I miss it so bad. I used to work 20 hours a day in full immersion, sculptural interface, virtual reality. Head tracking was up above 100 dots per inch. yaw, pitch, and roll. The resolution was 900 lines by 1280 or something like that per eye. And the frame rate was up to 60 frames a second depending on how many polygons I polished at it, but it would get up that high. This is from 92, they had tracking at that 100 DPI to 300 DPI level with no lag. And the head tracking was like that too. And the frame rate on the screen was like, it was no problem back then. Over 20 years ago, I've been waiting for this day for a long, long time to see stuff that's actually coming out. I just saw the Crescent Bay demo. It was almost there. But yeah, the system cost $53,000. So yeah, you know, you still have to pay about 10K to get tracking systems like this today. But I'm like, why is not Vive here? Please, I want to see it. That sounds like it's the only thing that really, really, really gets close. I'm just real excited to finally see VR take off again. It was kind of odd to have spent 20 hours a day for a decade and then have it sort of disappear. But it's really fun to see it come back and see all the new kids and the new ideas and the new energy and these fresh takes and it's a beautiful challenge to get off of my own ego and listen to what other people who are like coming out with fresh eyes are saying and it's fun.
[00:02:27.162] Kent Bye: And so what were you doing in VR back then? Were you actually designing VR within VR or what type of things were you doing for 20 hours a day in VR every day?
[00:02:36.738] Hallie McConlogue: I was doodling in space. Sort of like Tilt Brush, the full VR version, I suppose. I haven't actually used Tilt Brush with a full VR kit yet, but I was able to position stuff. We had sort of a Rube Goldberg type of toy interface that was half Mac Paint for VR, half Rube Goldberg type of stuff. and I would put together our VR demos for SIGGRAPH in that. I was the artist and sculptor on that project. I also was talking to them, of course, about the interface. Of course, I always wish they'd listened to me a lot more than they did, but they did what they could. And, you know, you learn some very interesting things when you put yourself in an alternate reality and working Actually, three-dimensionally working with parallax and hand tracking at really high fidelity and head tracking at high fidelity, it is totally a different kind of very immersive experience. As soon as you get the head tracking, high. And the hand tracking, high. It's a whole other world and please bring my world back to me. Please, all you developers and artists out there, please just bring it back. Thank you.
[00:03:44.236] Kent Bye: Wow, and so what do you want to create in this world? Now, if it was, you know, given to you with the HTC Vive and the whole Lighthouse tracking and you were able to dive in and start creating things, what would you want to make?
[00:03:55.025] Hallie McConlogue: I want to make, here I'm going to go a little girl style on you guys. I actually teach improv, theatrical improv, for fun because I worked in immersive experiences. So, if you're going to have immersive experiences that have interesting, dramatic content, You also want people to kind of be able to act with one another, improvisationally perhaps. I mean, if you want to be the hero of your own movie, do you want to be seen and do you want to have emotional interactions with other people? So what I want is full facial tracking. and I want to be able to actually create these experiences for people. Their souls are there and present and they're actually able to really have, you know, much higher bandwidth of real communication and real connection emotionally and at all levels. And as an improv teacher, I've seen what that does to people to really be able to really connect with one another at a really intense creative level, to be creative together in the same space. is extremely emotionally and psychologically powerful. And I think, you know, that's when you're going to get the women in the industry, when you have facial tracking, when you have eye tracking. And I think it's a little de-emphasized because people who tend to work in this field don't tend to have that much of a draw to that.
[00:05:16.545] Kent Bye: Yeah, well I went to IEEE VR and there was some hints of some of the research that Oculus has been doing and then actually at SIGGRAPH this year, Hal Lee and a number of different people from Oculus Research are going to be presenting their solution for fully facial tracking with an HMD on. So, I do think that facial tracking is a very tough problem and while you're in VR, but I think it's certainly on the horizon. And also there's solutions like FaceShift, which you can do facial tracking, but that's not interactive real-time while being in VR. That's sort of like you have to choose one or the other. So, have you done any experiments with doing facial tracking, or what sort of leans you towards saying that this is going to be sort of a really compelling component for VR?
[00:06:00.002] Hallie McConlogue: I've been doing the closest analogy in real life and that's about all I've been able to do because we work with eye contact a lot in improv and eye timing and all that stuff. So I had to test it out in real reality a little bit more. I invent games in real reality for people to play with each other. I'm not just an improviser, I'm an improv teacher. It's a different skill. And that is the skill that I think will really apply to creating compelling spaces where people really connect with one another in virtual reality. One of the things I do is I go to cafes and teach people how to sing improv music together. I teach people how to do improv acting together. And random, off-the-street strangers, I've hacked into how to get them going. And so that will apply to the virtual space if we get the eyes and the head tracked and the facial expressions. Eyes first. The rest, you know, after that.
[00:06:58.618] Kent Bye: So what are some of the top key foundational components of doing good improv? What do people need to be able to do if this is a key component of having good VR experiences and you've been doing this sort of real-life exploration of improv? What are some things that you found to be kind of the most salient aspects and foundational components of improv?
[00:07:17.252] Hallie McConlogue: I think the most important thing for people to learn to do when they're creating together in a more like body, body, body here, body there kind of way, the most important thing for people to do is for it to be disposable and to add on to each other's ideas. So don't be too precious about it. Just let go. Let yourself do stuff wrong. It's real important to have permission to completely fail flat on your face. do horrible work and then get past that and then you have more control over what kinds of things you're trying to express. So when you really jump in with other people without fear of failing, 90% better stuff just accidentally comes out of everybody no matter what level of training you have.
[00:08:02.483] Kent Bye: And I know from the little improv that I've done is that there's the principle of yes and where you don't contradict or shut things down, but you're always trying to build off of what's already there. So how does that principle apply?
[00:08:15.042] Hallie McConlogue: Absolutely. Yes hand is like what I was saying. I didn't emphasize it quite as much, but yes hand is key. You build on other people's creative ideas. You stay in the moment. You don't plan ahead. And VR's got presence built in, you know, especially if you develop interfaces that involve the breath, you get that presence. And those are all things where there's this synergy that can happen between improv, which I believe virtual reality, is the meta-medium for interactive entertainment. And who's been doing interactive entertainment longer than improv actors? Probably as old as it gets. So, you know, film is the meta-medium for linear entertainment. You want to sit and watch something from the beginning to the end and you don't want to, like, mess it up and change it and change how long it takes and do all this other weird stuff. So, that's sort of the research I've been doing since I graduated in 91 from cadre. at the San Jose State Computers and Art Research and Design and Education. It's a long name. Anyway, so I've been trying to find ways to push this art form for the day 100 years hence. Maybe five, I don't know. I've been thinking it was five years away for about 20 years now.
[00:09:31.948] Kent Bye: You had mentioned with enough breathing in space, maybe you can expand on that, what you meant in terms of how the breath and space can play into that equation.
[00:09:40.608] Hallie McConlogue: Well, breath and being present and being in the moment are connected. And there are some really interesting virtual reality interfaces that just rely on the breath in order to move that happened a long time ago. Beautiful work done by somebody. I can't remember the name of it. Anyway, some beautiful work's been done and shown in galleries early on in the old days. With using your breath, it was just a rubber band around your chest and like some kind of potentiometer or something that was measuring your breathing. And in order to do this diving simulation, You breathed less to go down, or out to go down, and you breathed in to go up, and that just makes you, it was so much more relaxing and in the moment. And I see that there's also some other more meditative VR things tending to happen. And VR right now, with the current limitations, works really well with meditative type experiences. Because, you know, the slower you go, the better it is at the moment. Because then you avoid some of the sickness factors, and you avoid some of the lag factors a little bit.
[00:10:40.842] Kent Bye: So what's been some of the most compelling VR experiences that you've ever had then?
[00:10:45.244] Hallie McConlogue: Well, I just saw the Crescent Bay. It was awesome. Still not quite as good as the Holosketch we had back in 92 in terms of the tracking. Better, of course, in texture mapping and all that silliness, but we had probably that many polygons and probably roughly the same resolution and better head tracking, which is amazing to have really good head tracking and to be working in an environment where you can move around quickly and put your hand right in front of your face and work on something and sculpt on something. To be able to grab something and puppeteer it and then you have the puppeteering thing there and it's a thing you can click on and group with something else and you know nested puppeteerings and it was just the most amazing toy. So I have to say Holosketch back from 92 to 2001 was still Michael Diering's work. Still really really stellar amazing and nobody ever saw it
[00:11:39.085] Kent Bye: And you had also mentioned something about starting a new group about girls in VR. Maybe you could describe that a little bit about what's happening there.
[00:11:45.549] Hallie McConlogue: I noticed that all the girls in one particular workshop were all sitting next to one another. So I snuck over there and I sat next to them and when the talk ended it was the Unity talk, I said, hey, all the girls are all sitting next to each other. We got to start a group. So somebody already had a laptop open, and then she started an email. We all put our emails in. And any girl I meet, I'm going to try and put them on the group, because I'm a second generation computer graphics artist. My mom started in 1959. with plotters. X and Y, machine code, yeah, all that stuff. So I've got stories, they've got stories. We should put our heads together and figure out how to get more women in here. And I love the guys I've worked with. They've all been awesome. And there's a translation problem.
[00:12:34.047] Kent Bye: Yeah, I think so, and I think that there does seem to be kind of a disproportionate number of males right now, but I do think that there are going to be, in order for VR to really succeed, that it has to be adopted equally by both genders, or all genders. And so I'm curious, like, for you, when you see these types of VR experiences that are kind of running around in big wild adventures, I'm curious if there's a type of VR experience that you think may be more well suited for women or things that you're just not interested in that maybe other people are or things that you see little sproutings of things that you think might be really compelling that you want to see a lot more of.
[00:13:09.380] Hallie McConlogue: There's stuff that I can imagine that would be a lot more compelling to my feminine side. That being said, I also love Quake. Quake was where it was at early on, and a lot of the first-person shooters, I adore them. There are things that come from the tendency of a woman's perspective, giving birth to kids and stuff like that, that is going to really be important going forward if you want that half of the market share. I mean, like romance novels versus, dare I say it, porn, right? What do you need for a romance novel to fly in VR? You need eyeballs. You need to feel like that romantic, beautiful guy is looking at you. That's when you're going to get the girls, you know, because that's girl sex. I mean, porn is for guys. It's like, you know, it's kind of something else is the focus. But for women, it's the relationships and it's the eyes and it's the emotion. And yeah, girls also want to have kind of control of that, want to have a buffer sometimes, too. You know, it can get too intense. But learning how to play with that edge is just going to be so powerful someday.
[00:14:22.178] Kent Bye: So what do you think the key components of emotional immersion might be then?
[00:14:27.203] Hallie McConlogue: Eyeballs, eyeballs, eyeballs, eyeballs, eyeballs, eyeballs. And I'm talking, that means the cheek too, because the emotional content of eyeballs is affected by cheek movement. It's going to be eyebrows, it's going to be eyelids, it's going to be the corner of the eye, it's going to be the dilation of the pupil. And pretty dang good tracking on that eyeball rotation, where you don't just put the angle of rotation in the center of the eyeball, people. And yes, the cornea pushes on the eyelid and that changes where you see the person looking and where the person's looking is what matters to people. Is that person a threat? Is that person family? Does that person love me? That is a huge deal. Someday that's going to be really important and I didn't hear anybody even talking about it in the interface design thing. I got there a little late but I didn't hear people talking about eyeball interfaces in the virtual reality interaction thing. And this is why women need to be in this biz a lot more, pushing things forward. Because Snow Crash, the first person... Snow Crash is where I got my first VR job. If I hadn't read that, I wouldn't have gotten my first VR job at Sun. Anyway, the first most important hero, at first, in Snow Crash, was the woman who cracked faces. And I think that's true. He got that right. He talked to a lot of technical people for a long time before he wrote that book, and he got that part right. And there is going to be... People aren't going to care. They aren't going to really, really care in their heart and their soul until you get that eyeball. Really, who are you looking at? Even for, like, aggression stuff and first-person shooters, when they put those eyeballs in there, whoo. I think that's where it's at.
[00:16:18.513] Kent Bye: Is there anything in VR that you want to experience that hasn't been out there yet or things that you're really excited to get into?
[00:16:24.059] Hallie McConlogue: I wish the Vive were here so bad. Man, I want to see that. It sounds like it's the closest thing to what I had before. So yeah, and then once they add eye oils to that, I'll be really happy. Of course, the whole face would be really nice too. And then the body, in that order.
[00:16:40.173] Kent Bye: Yeah, I think that's coming. Just at SIGGRAPH this year, I think there's going to be some big announcements and some initial research there. And finally, what do you see as kind of like the ultimate potential for what virtual reality may be able to enable?
[00:16:53.492] Hallie McConlogue: Oh, my. It will be probably good and bad for human interaction and very therapeutic. The stuff I saw with diplopia and autism so far, the hints I've gotten on that would be amazingly good. Of course, medical applications, therapeutic applications, but being a little bit selfish, I'm really looking forward to the sculptural interfaces that don't drive 90% of the workforce into the disability system. When you make it so you can arbitrarily change your daily motions to something that's healthy, for example, instead of something that turns you into a body by Lennox kind of thing, you know, couch potato body, instead you can arbitrarily change the motion to something that's therapeutic and healthy. That'll be huge. Absolutely huge. And it should be coming soon. I hope. I've said that before, but we'll try. I'll be hopeful again.
[00:17:51.706] Kent Bye: Thank you so much.
[00:17:53.107] Hallie McConlogue: Thank you for listening to my soapboxes.
[00:17:56.329] Kent Bye: And thank you for listening. If you'd like to support the Voices of VR podcast, then please consider becoming a patron at patreon.com slash Voices of VR.
