Linda Jacobson got into VR when she helped organize the 1990 CyberArts International gathering of artists and technologists who were using virtual reality technologies. She edited a compilation of CyberArts essays from that first gathering, and she also documented the Garage Virtual Reality DIY VR maker movement of the early 90s. In 1995, she became a VR evangelist at Silicon Graphics where she helped to sell VR into enterprise VR applications including engineering, architecture, construction, medicine, military training, automotive, aerospace, heavy equipment manufacturers, and oil and gas companies. The enterprise companies and applications of VR during this time period were pretty secretive and proprietary, but Jacobson was on the front lines traveling around the world seeing a huge range of different virtual worlds and use cases for VR.
Jacobson has continued to work in VR since the 90s ranging from entertainment to medicine to AEC, and has a lot of in insights about the evolution of VR in the enterprise space. I had a chance to talk with her at the Virtual Reality Strategy Conference in October about her last 20+ years in enterprise VR, her mentor Morton Heilig and his Sensorama, VR as a counter-cultural approach to computing, the CyberArts gathering of artists, DIY Garage Virtual Reality, and the major figures and companies who bootstrapped the commercial VR industry.
LISTEN TO THE VOICES OF VR PODCAST
This is a listener supported podcast, considering making a donation to the Voices of VR Podcast Patreon
Music: Fatality
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. So when I was at the Virtual Reality Strategy Conference back in October, the big focus was on enterprise VR. And there was a number of people there who had been involved in virtual reality for a long, long time, many decades. The thing about virtual reality is that as soon as the first commercially available VR systems became available in the late 80s, There's a number of big enterprises and military and engineers, construction, these whole industries that were using virtual reality technologies, but they weren't necessarily talking about it. It was a bit of a competitive advantage for whatever they were doing specifically to be able to either better design and architect their 3D objects or any other different training tasks. So I had a chance to catch up with Linda Jacobson who started into virtual reality back in 1991. She helped to organize the cyber arts gathering of artists and technologists who are coming together to share about whatever they were doing creatively within virtual reality. And so Linda went on to then write for Wired magazine. She wrote about DIY garage virtual reality and eventually ended up at Silicon Graphics, which is one of the four companies that were doing these high-end graphics systems. And she was on the front lines of engaging with a lot of the different companies that were using virtual reality technologies. So we'll be taking a journey through virtual reality from the late 80s and throughout the 90s and 2000s in order to get a sense of all the things that were happening and the transition and journey that Linda has taken from what she's doing now in VR. So that's what we'll be covering on today's episode of the Voices of VR podcast. So this interview with Linda happened on Thursday, October 26th, 2017 at the Greenlight Virtual Reality Strategy Conference in San Francisco, California. So with that, let's go ahead and dive right in.
[00:02:04.956] Linda Jacobson: My name is Linda Jacobson and I started my career in VR covering audio actually. I was brought into the whole idea of immersive visualization and real-time graphics when I had the opportunity to go to NASA Ames and learn about the binaural audio recording being used to send information in 3D space to astronauts as another way of communicating with them and that led me to reading the the Media Lab, Inventing the Future at MIT, and I thought, oh, this virtual reality is really the communications medium of the future. So I, at the time, was working for a music industry magazine, and our publisher put on an event that I got to produce called Cyber Arts. It was the first trade show for artists and technologists to meet each other and talk about using VR for expression and for artistic communication. I wrote a book called Cyber Arts and then became really interested in the whole do-it-yourself movement because I had this whole like hippie kind of zeitgeist about power to the people and I was really frustrated that VR was really limited to well experimentation in VR was limited to techno elitist institutions and academia and military so I wrote a book called Garage Virtual Reality. I was One of the co-founding editors of Wired magazine at the time and wrote about garage or handmade maker VR for Wired and at the time was also performing as a 3D computer generated puppet called Rigby with a musical multimedia ensemble called Dekuku. That was a motion capture activity and in the mid 90s I was recruited by Silicon Graphics to be their virtual reality evangelist and identify opportunities to sell high-end real-time graphics and high-end immersive peripherals to different vertical segments. And so I had the chance to kind of put together partnerships and go out and tell the world about VR, mostly for design and engineering, training, surgical simulation eventually. And I've been in it ever since.
[00:04:19.016] Kent Bye: So what year did you first get into VR? I know that Ivan Sutherland and Tom Perness and Morton Heilig back in the early 60s for Morton and then in the late... Oh Morton mentioned... Yeah and so did Myron.
[00:04:33.941] Linda Jacobson: Both these guys. Mort in particular. Yeah I adored him. but I started writing about binaural audio for simulated environments in 1988 and pursued the concepts of synthesized and simulated surround sound in 89 and then in 1990 was part of the team that produced cyber arts which is where I had the opportunity to help VPL research and fake space labs and Mandala and a bunch of other early people in the field. Myron was there too, Mort Heilig was there at that first event and then I published a book based on the event. So I would say my professional entry was 1990 We worked for two years from 92 to 94 on Wired Magazine. Garage Virtual Reality was published in 94. Timothy Childs, Peter Rothman, and I had already co-founded the first VR meetup, which had been meeting at the Exploratorium every month since 92, 91, 92, 93. And I went to work for Silicon Graphics in August of 1995.
[00:05:40.586] Kent Bye: Wow. And so did you ever get a chance to try out Morton Heilig's Sensorama?
[00:05:45.827] Linda Jacobson: Yes, sure, yes. Yeah, he had one left and it was an arcade type of installation and he showed it at the first Cyber Arts Convention in the Biltmore Bowl in Los Angeles in September 1990. So I had a chance to try it then.
[00:06:03.205] Kent Bye: Yeah, just in looking at the history of VR, Morton Heilig seems to be a guy who sort of got the vision for the potential for immersive storytelling and putting together a multi-century experience that maybe you would see in an arcade or something. And then it wasn't until like the mid-60s where both Tom Furness as well as Ivan Sutherland had started work on their own from the Air Force as well as with the Sword of Damocles, but Morton Heilig predates them in a certain way. I don't know if they were aware of each other at the time, but maybe you could just talk about what it was like to actually experience the Censorama.
[00:06:36.698] Linda Jacobson: The Censorama was inspired by Mord's vision for something called Cinerama, right? So he wanted not just a one-off, like a single-person experience, but an entire theatrical experience. And he was frustrated by his inability to raise the financing and put the money together and there's some amazing stories. He had a backer who was Mike Todd, the guy who had been married to Elizabeth Taylor who died in a plane crash when he was like on his way to get some funding for Mort Heilig's vision. But trying it was amazing. It was very basic. I was on this motorcycle seat and it showed an image of a flower market in Brooklyn and you're on the motorcycle and it's vibrating. It was fairly rudimentary, but it was delightful simply because it was a unique interface. So Myron brought this heavy-duty influence of video as art and of communications medium. And then Ivan Sutherland brought the whole flight sim vision into the arena with the emphasis on high-performance, low-latency displays. And Mort came from a storytelling perspective, however that was based on his work with Disney and as a filmmaker. So worlds were colliding and that I think was confusing for a lot of people but also there was a lot of synergies and a lot of mutual respect and it was a small community so it was exciting to meet and work with these people.
[00:08:11.389] Kent Bye: Yeah even here at the Virtual Reality Strategy we have like this weird mix of enterprise and entertainment and Hollywood all kind of intermixed together so I think that sort of weird mix is still kind of like in some ways virtual reality is like this medium that is combining all these different domains
[00:08:27.942] Linda Jacobson: Pipelines are different. So that's what makes it somewhat challenging for people who are crossing over between entertainment and media and digital prototyping AEC because there we've got a CAD based pipeline as opposed to an animation and CG based pipeline so the languages are different the production approaches are different, but ultimately we want the same goal, which is to, you know, remove the interface.
[00:08:52.855] Kent Bye: Yeah, that's interesting. Well, maybe you could paint the scene a little bit more in terms of what it was like in the San Francisco Bay area in the early 90s with all of this virtual reality technologies.
[00:09:05.377] Linda Jacobson: Similar to what we're seeing now, there was a real sense of community among the people who were interested in this technology, although the community was much smaller. There were much fewer people in it, and the companies that were involved in supporting the emergence of virtual reality really took a big role, but they were much smaller then. So the biggest one was like SGI and HP and they were very big, but Autodesk was involved. The people who were to become Adobe, you know, there was the alias Wavefront people. So it was a blend of very revenue-driven type of activity by people who believed in this as a vision and knew there wasn't any money coming soon and we were challenging the status quo and taking a risk with associating ourselves with a technology that at the time was being promoted heavily as a counter-cultural approach to computing by people who had experienced it and then claimed it was, you know, the new wave of going inward. So there was definitely an association with Timothy Leary and the Electronic Frontier Foundation and some more kind of psychedelic and subversive cultural influences.
[00:10:30.004] Kent Bye: Like Terrence McKenna?
[00:10:31.625] Linda Jacobson: Right, like Terrence McKenna, that's a great example, right. So it was very happening. It was every rave you went to, there was a head-mounted display and someone painting with 3D graphics. So it was really a party scene, too, and there was an odd blend of cultures, of people from different cultures blending. Like, you know, CS Snow, was it? I forget. There's a book called The Two Cultures about science and art and so this was a co-mingling of military people, hippies, video media mavens. So it was a lot of cross-pollination and a lot of arguing too. There were companies, small companies, getting their funding from the DoD through SBIR grants who would be, at my VR users groups, People like that would be shouted down by people in the audience for taking money from the military. So a lot of feeling too about who's going to fund the development and are you going to take anyone's money to do it. So there was a lot of concern about the influence in the developers area.
[00:11:37.708] Kent Bye: That's fascinating, because I know that Palmer Luckey, since leaving Facebook, has now started his Anduril company, which essentially is going to try to get these defense contracts to do drone surveillance on the border to be able to make sure that there's not people who are illegally immigrating, and then having citizen patrol officers watch through VR. I don't know what the exact connection there is, but I know that he's also received flak of going down a path of this more defense-oriented. Yeah, that's just something that just has come up and there's also people who are sort of angry about that.
[00:12:13.238] Linda Jacobson: First of all, virtual reality wouldn't exist without the money and the funding and the research work that was invested by the various departments in the federal government that were invested in it. But it's not just for the military or the war fighting, also for medical. I mean, there was so much money put into medical visualization, surgical simulation by the government too. This was also done for NASA and for aerospace efforts as well, so there's a time-honored tradition. Well before Google Earth existed, the concepts behind that technology, geospatial terrain mapping, was attracting government agencies to come to Silicon Graphics to speak with our teams about using satellite imagery and real-time graphics to map and control borders. So I remember an amazing experience I had presenting to the Princess of Thailand. who brought a contingent of people from Thailand. Her dad, the king, was or is a cartographer as a hobby, and she was tasked with researching and understanding the use of geospatial terrain mapping to protect the borders of Thailand. And so it's not unique, and maybe Palmer Luckey's interest in it. Who knows where it comes from? But he's just jumping on a bandwagon that has been fueled for many, many years. And it's nothing new, what he's doing. It's been going on for years, and all over the world.
[00:13:52.048] Kent Bye: Interesting. Yeah, I wasn't aware of that history. I'm curious to hear a little bit more about your first artist event, because it seems like that, you know, I've been covering the VR industry and trying to cover everything from the enterprise to the artists and storytellers. And, you know, I get really excited when I see the artists and what the artists want to do, because it's often not about trying to make money, but it's more about what do the artists want to experience. And they're giving you an experience that you would have never had, had they not either found the resources or the time or energy to put that together. So take me back to your first conference that you had of all these people together, and what type of stuff people were thinking about, what they were doing, and what kind of experiences they were able to put together.
[00:14:32.429] Linda Jacobson: Well, you bring up the art as a key driving force behind the creation of CyberArts. CyberArts was the vision of Dominic Milano who was the editor of Keyboard Magazine and his artwork was 3D graphics and 2D graphics based but he was interested in exploring the digitization of art media to produce new types of work and also to celebrate 10th anniversary of Keyboard Magazine. So that particular event was very grounded in music and other performing arts and so it brought together not just visual artists but there was a dancer who was using motion capture for choreography and there were people who were just experimenting with olfactory simulation and scent and how to simulate and distribute scent but there was an art gallery there and To a large extent, it wasn't that different from a SIGGRAPH event at the time, too, where there were art installations, interactive media experiences, and people talking about how they can make art through computers. So what stands out is David M., who is an artist who used digital media. M spelled E-M, and he made an impassioned call for artists to explore 3D graphics and digital media as a form of expression. And he said he'd been asked over and over, how can you make art with a computer? And his answer was always, I can make art with mud. So what are you talking about? So there was kind of a realization all these people had been in their own little worlds experimenting with early Macs and art making programs on the Commodore Amiga and other types of homemade pixel pushers and it was kind of like a coming out for everybody to realize that there was a community emerging. So it was a great sense of community. There was a big party and there were a lot of musicians there. So Rick Emerson was there from Emerson Lake and Palmer and a lot of people were following him around. Jaron Lanier had an entourage. and it had this great sense of excitement around it and no one wanted to go to sleep. You know, it was, well you could imagine what it was like, but it was serious too.
[00:17:04.832] Kent Bye: Yeah, and it seems like, just met Ben Delaney and it seems like he was kind of chronicling things at the time in a way of doing something equivalent to something like Road to VR or like the podcast that I'm doing here, kind of chronicling the industry and so I'm just curious to kind of hear how people were hearing about this and learning that you were serving as an evangelist and so there's this time of, I'm just trying to get a sense of the parallels between what's been happening with this modern resurgence of VR and what was happening back in the late 80s and early 90s.
[00:17:34.935] Linda Jacobson: So a critical medium of information for all of us was the Well. Have you heard of the Well? Yeah, yeah. Okay, so the Well and before that Usenet had a number of bulletin board threads and dialogues going. One of the groundbreaking ones was Sci.virtualworlds on Usenet and then the Well had a VR conference. People like Ben and myself and others who were reporting on the scene were very active in the well and on Usenet as well. So a lot of information flow came through the modem, through a dial-up modem. And a lot of it came through the SIGCHI, so we had a gather, right? industry conferences, and that was the reason for user groups popping up around the country, like Verge, the virtual reality group here in the Bay Area that I co-founded, and a VR event group out of the Electronic Cafe in Santa Monica. Now I'm not sure if I'm answering your question. I think I took a left turn.
[00:18:38.590] Kent Bye: I was trying to get a sense of the media ecosystem, and I know that Howard Rheingold was also somebody who had written a book.
[00:18:43.812] Linda Jacobson: He was a good friend, yeah. Howard at the time, well he was very active on the Well, also HLR at Well, and he had partnered with Lewis and Jane to start Hotwired at the time. Right when Tim Berners-Lee and that group put forth the World Wide Web, so that was first emerging and that actually took a lot of attention away from VR developers because it was so accessible and so powerful and held so much promise for incredible interactions. Books were really important conferences, but offline, I mean online conferences were critical for the ecosystem to form. So Howard wrote a book called Virtual Reality and so did Myron wrote a book and all these people, we carried them around like Bibles, but these people were also accessible and very interactive and available online as well, which was critical for the creation of awareness across the world.
[00:19:46.474] Kent Bye: You said by 1994 you started to work at Silicon Graphics as an evangelist and maybe could share what was it like to serve as an evangelist for a medium that you were on the front lines of telling a lot of people about VR for the first time or giving them some of their first VR experiences.
[00:20:02.738] Linda Jacobson: It was really fascinating. I joined in August of 1995 and had not ever worked full-time in a big corporate environment. So it took me a while once I got there to understand that I actually was in product marketing inside an advanced graphics engineering division. And my role and my performance was reviewed based on sales of high-end supercomputers and the APIs that shipped with them. but I did have a lot of resources, a lot of support, and I was tasked with identifying which of the industry segments that Silicon Graphics served would be ready for early adoption. I had a number of demo systems at my disposal that I could choose to loan out for periods of time, three to six months, in order to have the recipient develop applications, they weren't called apps then, right, to develop software use cases for selling the machines and then we would use those demo environments to promote the machines to others. So at the time there was a really, really well-established effort and partnership between Silicon Graphics and the military, so I wasn't involved in the visual simulation aspects. Same thing with entertainment industry. So I was looking at what became known as serious games, in what is now known as Enterprise VR. And my role was to look for early adopters as well as set up software developer partnerships with companies addressing, at the time, engineers, architects, construction teams, designers, and eventually, in the late 90s, moved into the medical field. But we're crossed over into military for medical and surgical simulation, anatomical training, and such.
[00:21:57.942] Kent Bye: Yeah, that's something that is one of the frustrations I have with the modern consumer VR is that some people kind of put the origin point of VR with the Oculus Rift and say, oh, I've been doing VR for two or three years, and that's a really long time. And it's like, well.
[00:22:09.775] Linda Jacobson: Skunkworks project with Nintendo creating belt pack VR using the Nintendo... I don't remember what engine they were on back then. What year was it? It was 96, 97 and Nintendo and Silicon Graphics were hard at work on a belt pack driven game based head mounted display system. I talked and met with Sega, they had one too. Mattel, of course, had the Power Glove that had been released and killed a couple of years earlier. So there was ongoing work and there were consumer-level VR headsets that were available. They were not high-performance and they were very expensive, but they were there. So I listened to some of the rhetoric from some of the VR events here where people say, oh, we're inventing it, but we're not. It's just a new influx of adherence to the cause. But just yesterday, someone on stage, an investor, was saying, oh my god, wouldn't it be great if you could have surgical simulation? And doesn't it make a lot more sense to use real patient data to prepare surgeons to send radiation into a cancer But that was something that we were talking about and doing 15 to 18 years ago using patient data from a cadaver that had been donated by that person himself to science. He was an inmate and he was the first person ever to donate his body for volumetric visualization and the virtual human project it was called and that was real patient data that was then used as a prototype for creating surgical simulation platforms to teach surgeons how to go through a procedure using real patient data and to practice it before going in. craniofacial surgeons were doing it. I worked a lot with NASA Ames doctors who were partnering with Stanford and there was a lot of development going on and companies like Intuitive Surgical were new then and getting started but MystVR came out of that and in the UK came out of that time and continues to be a clinically ready system. The University College of London invested a lot of money and was at the forefront of implementing surgical simulation using patient-based data. There you go.
[00:24:34.026] Kent Bye: Yeah, yeah when I've gone to three IEEE VR conferences now over the last three years and I have an opportunity to talk to VR researchers from around the world and a lot of them have been engaged in VR for the last 20 to 30 years and I get the sense that after the 90s a lot of the enterprise VR just they wouldn't talk about it because it'd be a little bit of like a competitive advantage and so it's a little bit more of a secret.
[00:24:59.288] Linda Jacobson: You nailed it and I admire that insight because very much everybody was doing this very quietly and there was also a perception that boards of directors at public companies would maybe be uncomfortable that their company was funding this R&D. But every major automotive, aerospace, heavy equipment manufacturer in the world was actively using VR devices to prototype and engineer and test their products. Now, they didn't necessarily call it virtual reality and they didn't always use head-mounted displays. There was a great reliance on projection-based wraparound systems and cave systems and desktop systems. as well as boom-mounted head-mounted displays, where you pulled the headset over to your face rather than strapped it on your head. So this is active development, the tools, the SDKs, open source, it's all been bubbling. But because of the media attention and the profile of Facebook and Mark Zuckerberg, that is what gave the new VR such a great boost. You should talk to Mark Bolas.
[00:26:08.994] Kent Bye: Have you talked to him? I haven't had a chance. I've talked to him just, I've met him a couple times. He's at Microsoft now, so I'd love to just sort of talk to him about the whole history of everything as well.
[00:26:19.982] Linda Jacobson: Yeah, he's critical in all this. I mean, this field would not exist without Mark Bolas. Palmer Luckey would not have done what he could do without Mark Bolas.
[00:26:29.055] Kent Bye: Yeah, I have a couple of interviews with David Crum as well as with Evan from USCICT talking about sort of the history of USCICT, the Institute for Creative Studies there and their involvement with the military contracts. And so I feel like there's parts of the history that haven't fully been told.
[00:26:46.873] Linda Jacobson: I was directly involved in selling the high-end supercomputer to Jackie and her team at the ICT and was there at the opening. And it was very much a desire by the military to get into Hollywood and to use Hollywood storytelling techniques to do better visual simulation. And Hollywood was hoping to get contracts too. So it started out in that kind of vein. But Jackie's a good one too to talk to about.
[00:27:15.218] Kent Bye: I have a couple interviews with her, one that I've aired and one that I haven't. One of the questions that I had for you that you may have some insight into is that I get the sense that Ivan Sutherland as well as Tom Furness was starting in the 60s working on virtual reality and that Tom was working with Air Force simulation and all the way up to like in the 80s, mid 80s, maybe up to 89 when he left to go start the VR lab that's up in University of Washington. And I imagine that the work within Virtual Reality had continued into some of the flight simulation stuff that he had started and then he wanted to start to bring some of this technology into the public domain in terms of education and health and healing. promote VR as this public resource but I guess my question is is like do you have a sense of what has been happening with virtual reality technologies from like the mid 60s up until the 90s because to me that's a bit of a dark part of the history of VR that I haven't had an opportunity to talk to very many people that had an insight as to the other things that may have been happening within the context of the military up until the resurgence of VR and the 80 onward.
[00:28:25.510] Linda Jacobson: Yeah, I, you know, a lot of that between the 60s and the 90s, that development was really focused on understanding and building engines to deliver 3D graphics. So it was really foundational in terms of tool development and understanding how to implement concepts, you know, thousand-year-old concepts of drawing in 2D space and extending that and creating a language, much less a toolkit for operating in 3D space. So, flight simulation was very much a, I think, a springboard for understanding that, where initially it was film projected outside of the flight simulation gimbal, but there was than a desire to use computers instead of film because film was so expensive. So a lot of the development had less to do with what we know now as UX and UI and had much more to do with just the engine and how is it going to display as much as possible and, you know, the GUI hadn't even appeared yet, right? So there were a lot of enabling technologies that had to be developed in, say, the 70s and 80s in order to enable the VR of the 90s, which is the same VR we know today.
[00:29:52.153] Kent Bye: I see. And so what was the sort of turning point in terms of, like, all this technology coming together to kind of be what we would call today virtual reality?
[00:30:02.125] Linda Jacobson: The internet has been critical. The existence of broad bandwidth in order to accommodate better graphics. The engineering of bigger machines. The fact that we don't need a $100,000 supercomputer to drive beautiful graphics has been critical. you know, the commoditization of graphics engines and the availability of high bandwidth communications, telecommunications, coupled with awareness of user interface, what it means to create and enable a user to interact with the computer. The whole development of the QWERTY keyboard and the mouse, all these devices used for manipulating digital data were developed for different reasons, not for use with 3D graphics, but for use with 2D graphics. had to be that awareness of, wait a minute, if we're going to control a 3D object in a simulated 3D space, do we really want to do that with a keyboard or a mouse or a command line? What are other ways to do that? So Logitech, for instance, 3D Connection out of Germany came up with early six degree of freedom control devices. So there was a lot of exploration and innovation going on also in the world of sixth degree of freedom control and interaction that enables today's.
[00:31:34.815] Kent Bye: What would be sort of the first VR system that you would consider? Was it VPL and Jaron Lanier or what company was really putting that together?
[00:31:42.945] Linda Jacobson: The first commercial VR system that was a complete ecosystem was VPL's. VPL Research and their iPhone and their Dataphone and their their API, their world building kit. That was the first. There were other companies certainly who were putting together other pieces of infrastructural support. You know, Fakespace had displays and was exploring with music simulation and certainly Evans & Sutherland, Hewlett-Packard, Silicon Graphics and Sun were looking at the engines of creating 3D graphics engines and they were funded by the military. But yeah, at that first CyberArts, VPL Research had a 10x10 booth and Fakespace had a booth. There was a guy named Vincent John Vincent, if you follow him. He was doing fantastic artistic work with using video to allow people to interact with 3D graphics, with video capture. He had a commercial system that he was trying to introduce to schools and artists. That was critical as well, and it was commercially available. So it was called the Mandala system. Vincent John Vincent is the guy behind it. So that was a commercially available system, and VPL's was too.
[00:32:57.003] Kent Bye: Yeah and reading through Ben Delaney's book it was interesting to kind of read through some of that history and I got the impression that Jaron Lanier and VPL was getting a lot of media attention and you know Jaron coined the term virtual reality but yet at some point they kind of imploded and then there were other people that continued to carry the torch within virtuality in that community. Maybe you could talk about that moment when you had realized that their business was kind of failing and then like then it was sort of moving on to other people to continue on with the work of VR throughout the 90s.
[00:33:29.242] Linda Jacobson: Virtual reality is a phrase that came out of, I think, an HP technical documentation manual in reference to virtual memory, which was then a thing, starting to be a thing, virtualization of storage. And Jaron seized on to that beautiful phrase, virtual reality, and then he popularized it to explain what it is that he and his company were doing. But VPL, you know, I had the opportunity to hang out with those folks and they really were working hard to sell the entire system, but there wasn't enough of a, I think, a return on investment that could be documented for enterprises to invest in it. And I didn't have like a direct interaction with understanding the business. It was later when I was at SGI when I was asked to help analyze the IP portfolio that Thompson, the French defense contractor, had inherited because they had invested in VPL research and then VPL, when it ended the business, Thompson got the whole patent portfolio. And at that time I was able to work on it with a bunch of patent attorneys and engineers who analyzed it and said it was too broad but we should bid on it. But Sun Microsystems outbid SGI and bought the portfolio. But, you know, nothing ever came of it as we know. So at the time, though, VPL Research was one company in a larger ecosystem, and others were experiencing success with what's now called the tools and pits and shovels or whatever, the basic infrastructure, rather than trying to do an entire ecosystem. So I think what happened was then a pullback and a reliance on the companies shipping the big graphics engines to be the boat or the ship on which the peripherals rode in on.
[00:35:27.604] Kent Bye: Yeah, and I'm curious to hear from your own personal journey from, it sounds like you were at SGI doing the evangelism and enterprise, but at some point you decided to sort of shift over into health and healing in VR, and then that's sort of been your focus ever since, sounds like.
[00:35:43.092] Linda Jacobson: Well, it's been my focus for the past 10 years but at SGI what was happening was outside in the world the commoditization of graphics and the appearance of NVIDIA and the availability of low-end graphics engines was making it really challenging for Silicon Graphics to sell its high-end supercomputers into enterprise and so my work became more and more focused on the very high-end system users in the oil and gas industry and in the military. And I just felt at that time that I really wanted to be involved in creating applications and working as a producer and developer, working with developers and starting to create content rather than sell the hardware. So with SGI's blessings, I left SGI and with SGI's machines and access to briefing centers, I started a start-up. I started a VR production studio called Glasshouse Studio with four partners and we positioned ourselves to develop Unix-based real-time graphics systems like using SmartScene from MultiGen and a number of open source tools, OpenGL and Performer, to create custom virtual environments for companies who were in the education and entertainment space. However, 9-11 happened, and that put a big hold on non-essential spending by entertainment and education companies. So I had to pivot my company to focus on where I thought there was more opportunity, which at the time was architecture, engineering, construction, for the purpose of emergency walkthroughs and situational awareness training and things that were really of high awareness at the time. And so I shifted the company and we spent the next 3-4 years working with Bechtel and Sun Microsystems, an architect named Stanley Sadowitz, and developing custom, immersive, interactive applications. for those companies. At the time everything was projected. We did not do any head-mounted work at all. We were using projection displays, stereoscopic projection displays with electronic shutter glasses and pinch gloves and interactive gloves and space ball and other types of six degree of freedom controllers. But for a variety of reasons, I went back to Xerox PARC in the late 2000s to help run a marketing communications group as part of Xerox's rollout of PARC as an independent R&D company. So I actually left VR for a few years to work in open innovation and working with entrepreneurs and scientists and documenting and promoting some technologies. But all along I thought I need to get back into VR and when I at the time had to help provide care for my mom and my grandmother and I was dismayed by how old school the medical area was in the delivery of services and information to people involved in senior care. And I thought, wow, this is ripe. This is an area where VR could really help on so many levels. So I left PARC to focus on how I could help bring technology to the advantage of people involved in health and wellness, especially for long-term care and people who are impaired.
[00:39:28.180] Kent Bye: Because you've been in VR for so long, and I imagine you've experienced so many different experiences, I'm just curious to hear what experiences were turning points or most memorable in terms of you having an experience in VR that helped you understand what this meeting was all about.
[00:39:44.412] Linda Jacobson: That's a fantastic question. for. I mean they stand out and I probably have been in as many immersive environments as anyone has all over the world because that was my life. I lived and breathed it and traveled internationally and had amazing experiences but you know there was a discussion yesterday morning about realism that realism was critical to the experience And I heard, I was told that Tony Parisi said, no, your mind fills in the gaps. The experience that propelled me into this world was my very first immersion experience at VPL Research, where I was wearing the iPhone's head-mounted display and taken through a number of different very basic and cartoon-like environments that were based on reality. But there was one experience that involved a flying purple and pink striped gremlin that had a high-pitched voice and a buzz and I was simply in kind of a garden scene there was very little objects in the scene and suddenly there was this like laughing gremlin flying around my head and imploring me to grab me, grab me, grab me and I was wearing the data gloves and I kept trying to grab it and it was flying around my head and there was binaural sound in space, spatial audio, that was tracked to this little gremlin and the gremlin was probably, you know, all of ten polygons, who knows, it was ten polygons just simple shading but fantastic audio and a lot there was a real kinesthetic experience because I was twisting and turning my body trying to catch this thing and I would grab it and then he would say throw me throw me and I would throw it and it would whirl off into the distance with the Doppler effect keenly going laughing ha Then the next thing, it was like, boom, back around my head. Catch me, catch me. And it was just delightful. And this was a perfect example that immersion and suspension of disbelief did not necessarily require photorealism and actually could be a really engaging experience with very little. So that was amazing. And I'll never forget it. And I'll always be grateful to Jaron Lanier for that experience. The other experiences that changed my life, one was a simulation of an elevator ride, a skyscraper, coupled with an airplane ride. And this was at Georgia Tech University with Larry Hodges, who was developing environments in order to help people with phobias acclimate to heights and to fear of flying. And these experiences were extremely realistic and scary to me. You know, it evoked my heart beat faster as I rose up in this ridiculous skyscraper in a head-mounted display. And I was stunned by the sense in the airplane synthesizer that I was actually on a plane. I had another amazing experience working with NASA Ames and Stanford in depicting stereoscopic cadaver, human body, floating on the top of the immersive workbench that Fakespace had developed, which is now sold by zSpace as an educational tool. And to see a stereoscopic image of a person on a table that was being examined by real doctors was amazing. There was one other thing that was stunning to me. I was at GM. I was doing a photo shoot, I think, at the General Motors R&D cave and they invited me. It was my first time going into the GM cave in order to see a simulation of a new car interior, which is why they use the cave to do ergonomic design from the driver's seat. And I walked in, and again, this is not a head-mounted display environment, it was stereoscopic projection. And I didn't even have a controller. It was being controlled by someone else. But I sat down on a desk chair and was already wearing the glasses. The simulation had started. I was inside a car of the future and could see outside. And here was the dashboard in front of me, the steering wheel. Everything was as if I were sitting inside a car. I remember I wanted to do something with my left hand, maybe reach out and adjust a mirror, and I took my clipboard, which was a real clipboard I was holding, and I reached over to my right to put it on the credenza in between me and the passenger seat, and the clipboard dropped to the floor. because it was not a real credenza, you know, not a real console. But my suspension of disobedience had been so solid that my brain had said, oh, that's the real thing. So that also was startling. I was embarrassed. I was the only one. I was so embarrassed. And yet I was thrilled that I could have that experience knowing that it was pixels in one half of my brain. I knew it. But my experiential precognitive brain, it was real.
[00:44:53.470] Kent Bye: Yeah, I had that similar experience of the first time of having that dimension of presence was at the Silicon Valley Virtual Reality Conference of May of 2014, where I was doing the Sixth Sense demo of having the six degree of freedom, track controllers, and shooting a gun. And it had really been the first time that I had seen my hands in VR. And then when I was done, I was like, I'm going to just put this gun on the table, and then I'll be done. And it was that similar kind of like, oh, wait, yeah, this is not real. And my belief had been so suspended that I had just believed that it was real.
[00:45:23.693] Linda Jacobson: So, Paul Malenik, who created MakeVR, and Steve Hanstead, who's the biz dev guy at Sixth Sense, were my business partners in Glasshouse Studio. And they previously had been at MultiGen, and they have a long-rooted history in creating realistic simulations. So, yeah, I was blown away by the Sixth Sense lightsaber experience, too. I'm sorry, how did you feel after you did that?
[00:45:52.441] Kent Bye: Well, it was a thing where I guess I felt a little bit of a shame, a similar like I was like, oh, wow, I should have known like, you know, it's kind of like the shame of not knowing that I was so it was sort of like I felt bad that I did that, but also it felt exhilarating to have been so immersed to that degree. And it really kind of showed me the power of VR at that moment of the import of hand-tracked controllers in that way. And that's part of the reason why when Oculus had announced that they were going to release the Oculus Rift without any 6-degree-of-freedom controllers, and it was a while until the touch controllers came out, I was talking to Sebastian Kuntz and just, you know, going through some of the demos that they had at Oculus Connect 1, and my comment was, yeah, but it's not the degree of presence that I know is possible. And it's like, this is disappointing that this is not being released as a holistic experience that was emphasizing sit-down experiences with a gamepad controller was a lot different than what I knew the potential of the full immersion of having your full body immersed. So there was a little bit of like as it unfolded there was like frustrations that the decisions were being made to not sort of give the best VR that was possible but because it didn't come out another year later or so then I can understand why they want to get it out there and kind of iterate and add it later but the fact that the HTC Vive was able to come out with a fully six degree of freedom room scale approach, I think was in a lot of ways driving the overall industry to emphasize that level of embodied presence of like, what's it mean to have your body in VR? And I think that I've been frustrated with Oculus of focusing on, you know, 100 degree track controllers that makes you do a level of abstraction to turn around to put the burden on the user to have a mental model of where the cameras are in real reality while you're immersed. So you can't just completely get immersed. And so it's still these after effects of like not fully getting the power of that full embodied immersion and some of the design trade-offs that were made in order to kind of preference the ease of setup over the immersion that you get from full embodiment.
[00:48:02.535] Linda Jacobson: Well, that's a great observation. It is a trade-off. And they made that decision and decided, well, we're not going to invest in empowering people who want to use their hands. You know, we're primates. Even if you can't grab or touch something, you still want to use your hand because it gives you power and control, much more so than just seeing something. So they have to be coupled together because we use our hands to protect our faces and to get through the world. So we're all one system and they decline to leverage the way our human perception works.
[00:48:38.893] Kent Bye: Yeah, and it sounds like your other experiences also had the hands, also spatial audio and haptics, you know, all the sensory experiences of, like, new levels of immersion. And for me, the first time I did The Void, which was a untethered, redirected walking experience where I was able to kind of walk infinitely in VR for the first time, I got that another dimension of that, of, like, the depth of presence that can come when you are unconstrained to be able to, like, just get completely lost within an experience. But the audio for you, because that was such an important part of your entrance into this space, it seems like that was kind of driving your interest of, like, the new spatial and immersive audio that was kind of evolving at that time.
[00:49:19.541] Linda Jacobson: That's right. It's a great channel of information, and it helps heighten realism and suspend disbelief. So it's critical, and we're missing it.
[00:49:31.275] Kent Bye: So for you, what do you want to experience in VR?
[00:49:36.920] Linda Jacobson: That's a great question. I think that I would really like to experience other people in VR. Virtual reality has been a really lonely place and I am excited about social VR and the ability to bring people together over miles who could enjoy things together in VR. I really want to play ping-pong in virtual reality. I really, really, really want to do that. And I also am just really interested in using VR as a tool to bring engagement and comfort to people whose quality of life is not really high. So, for me personally, that may wind up happening. but at least I'd like to work with the companies now who are looking at providing virtual environments to people who are perfect audiences for it, who can't experience life the way the rest of us do, but could if it were in a virtual reality world. Also, I hope, and I think I mentioned this earlier, I really want to help drive the development of virtual colonoscopy systems. And all kinds of non-evasive
[00:50:50.965] Kent Bye: Image based diagnostic systems that sounds like that's been a driving factor there So and and finally, what do you think is kind of the ultimate potential of virtual reality and what am I people to enable?
[00:51:08.718] Linda Jacobson: Silicon Graphics was involved in using VR and computer vision to drive a robot around the Chernobyl plants. Kent, I don't have an easy answer for that. The possibilities are limitless for having so many experiences available to us that we can't have. I'm really delighted by augmented reality development and mixed reality development. So for me, my interest right now is in helping to push that forward. I really don't think headsets are the future. I think they'll be a blend. They'll be progressive lenses, if you will, in the use of smart glasses that could go from isolating you from the real world to laying graphics over the real world to adding music and delightful sounds to the real world. I am really interested in the use of augmented displays to help people who don't have their memory anymore or to help people who don't have all their perceptual systems. So that's my particular vision and dream. I love the art too and what Google did with Tilt Brush is a gift to the world. What Fantastic Contraptions did is a gift. So the use of VR for whimsy and exploration and discovery, that will never end.
[00:52:41.674] Kent Bye: Is there anything else that's left unsaid that you'd like to say?
[00:52:46.177] Linda Jacobson: You're doing really important work and I am grateful for it. So thanks for the time together.
[00:52:50.400] Kent Bye: Awesome. Thank you so much.
[00:52:52.642] Linda Jacobson: Thank you so much.
[00:52:54.448] Kent Bye: So that was Linda Jacobson. She's the author of Cyber Arts, Exploring Art and Technology, as well as Garage Virtuality, and has been working in virtuality since 1990. So I have a number of different takeaways about this interview is that, first of all, it was just super fascinating to hear from Linda from more of the creative side of somebody who wanted to use virtual reality technologies for creative expression. And, you know, one of the things that she mentions is that her two mentors were Mort Heilig, who created the sensorama, as well as Mayim Kruger, who wrote a book about artificial realities. So both of these two people, both Mort Heilig was certainly ahead of his time back in the 1952 had these visions of being able to have these immersive cinema experiences. And then back in 1961, he built the sensorama to be able to try to get funding to show to these different movie companies like, hey, we can do like immersive 3D film, we can add smells and haptics and 3d audio and all these things that he was working on which was like way ahead of his time. So there's that there's also Myron Kruger who wrote the artificial realities and he was doing stuff with a 2d video projection so you would take like a video of himself and then project it onto a screen and then you'd be looking at yourself and having like these interactive experiences. So it was a little bit of like a virtual world but it was a 2d world and Myron had called it artificial reality And he was working on that back in the late 60s and then did his PhD on that and published a book in 1983 and then re-released it called Artificial Realities II back in the early 90s. So this Cyber Arts Exploring Art and Technology book is actually a really interesting read in terms of just taking a cross-section of all the different artists that were doing different things at that time. It's got essays in there from Jaron Lanier and Mort Heilig and Mayr Kruger. And all the big people that she was mentioning throughout the course of this interview are featured and have essays in the cyber arts, exploring arts and technology. And also this whole idea of garage virtual reality, looking through it, you kind of get this sense of like this hacker and maker mentality, and that there was this whole group of people who just wanted to democratize the access to virtual reality technologies. In a lot of ways, if you look at the history of what Palmer Luckey was doing before the Oculus Rift is that he was doing a lot of these modifications and portabilizing of game technologies that kind of led into this resurgence of the virtual reality with the Oculus Rift Kickstarter in 2012. And that the other thing is that there is this whole history of what was happening throughout the late 80s and throughout the 90s and even after that into the 2000s that that history of virtual reality isn't very well documented just because a lot of that stuff was behind the scenes and proprietary. So it was happening. People just weren't talking about it. And so there's just a tendency for people to get up and say, Hey, this is the first time that we've done this type of surgical simulation. And then, you know, yet. This is one of the very first applications of virtual reality back in the late 80s with the military and doing different medical surgical simulations. But it's just interesting to hear all the different demographics that were involved at the early days. You know, a lot of the military and entertainment were other sections that by the time that Linda had gotten to Silicon Graphics, She was in the Sirius games, also known as the Enterprise VR. Everything from engineers, architects, construction teams, designers, medical and surgical simulations, anatomical training, cranial facial surgery with real patient data, automotive aerospace, heavy equipment manufacturers, and then the high-end oil and gas industry and all sorts of other military and training simulations that are out there. there's just been a lot that's happening in virtual reality and some of it is still like just classified and that we don't quite know the full extent of what has been happening. but that a lot of these virtual reality technologies were not all head-mounted. There was the boom-mounted head-mounted displays like from fake space labs and projection maps, as well as the cave systems and desktop VR systems. So just a lot of other technologies that were out there in this whole ecosystem that had been developed and very, very expensive, certainly not consumer grade. So I think that the major point that I just want to make with all that is that there's a whole ecosystem of bespoke applications that were out there before we got to the point of having consumer VR. And that what I'm seeing is that there's a whole other continuation of those markets that have already been using virtual reality technologies. And there's more opportunities for enterprise VR to go beyond these very early adopters and pioneers of virtual reality technologies and to see all the other different applications for all these other different businesses that are out there. And I had no idea that there was a Skunk Works project with Nintendo that Silicon Graphics had worked on. Of course, they have the Virtual Boy that they had released, but that sounds like this was another project that came after that. And finally, it was super interesting to hear about how virtual reality was being promoted as this countercultural approach to computing. And, you know, I think a lot of people who are doing psychedelics and getting in touch with these different realms of reality through these altered states of consciousness, that there is these different similarities that people were having when they were getting in touch with their sensory experience, getting in touch with the human body, paying attention to the nature of consciousness and phenomenology and how virtual reality was able to kind of hack into their consciousness in a way that was able to take them into these other realms. And so there's a lot of different parallels between the psychedelic culture that was happening as well as virtual reality technologies and that there's people like Timothy Leary as well as Terence McKenna that were adopting virtual reality as this new psychedelic experience. Interestingly enough, in the book by Jaron Lanier, A Dawn of the New Everything, Jaron actually was against those associations of, you know, psychedelic culture with virtual reality. I think in part because these different dreams or psychedelic experiences are coming from the imagery from those are coming from your own sort of experiences and they're very specific to you and that in virtual reality, the imagery is coming from somebody else. And so it's kind of a difference between your own imagination and something that somebody else is giving to you. And so I can definitely see how those differences would cause Jaren to say that those are different. But at the same time, Jaron says in his book that he's never actually tried out any psychedelics or even marijuana. So he doesn't have any direct experiences in terms of to see how much there's a similarity in terms of when it comes to phenomenology or a direct experience or, you know, being centered in our body, how there's enough overlap for people who've had enough psychedelic experiences, then there's enough analogs that you can kind of get a sense of this is a bit of an altered state of consciousness. So that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, there's a couple things you can do. First of all, just spread the word, tell your friends, and consider becoming a donor to the Patreon. This is a listener-supported podcast, and I rely upon your gracious donations to continue to bring you this coverage. And just a few dollars a month makes a huge difference. So you can sign up today at patreon.com slash Voices of VR. Thanks for listening.
