Walter Greenleaf has been involved with the intersection of virtual reality and medicine for 35 years, and I had a chance to catch up with him at Oculus Connect in order to get some of his impressions from the keynote announcements. He was at the same time excited for the new announcements, the momentum of tetherless VR like the Quest due to it’s ease of use and potential medical applications, but he was also concerned about the lack of honest conversations around the deeper ethical and privacy implications of immersive technologies. Facebook talked about the future of the AR cloud and how it would be possible to capture your personal environments with photogrammetry scans, but the lack of any discussions about privacy-first architectures was somewhat disturbing for Greenleaf. There’s also going to be a lot of possibilities for the technologies to be able to make medical diagnoses, and he suggests that some of the biometric data that will be available may need to be regulated by something like HIPAA if Facebook doesn’t try to proactively architect to protect the capture and use of biometric data. So we cover some of the ethical and privacy implications of VR, as well as a brief update as to what he’s seeing in terms of the medical applications of VR.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye and welcome to the Voices of VR podcast. So Oculus Connect 6 is the annual conference that's put on by Facebook to bring together the virtual reality community. It's the sixth conference that they've had, and I've been to all six of them now and went the day before and day after, ended up doing about 27 different interviews with about 17 and a half hours of conversations. There's a number of themes that had come up, whether it was education or training, that was something that over and over again seemed to be a theme for me as I was doing my coverage there. There was a number of different announcements. This is what I'm going to be doing in this series of six different interviews that were covering a variety of different topics but also covering some of the big announcements that were being made, what was being said, what wasn't being said. I think privacy and ethics was a topic that didn't have a lot of conversation and so it ended up being relegated to people at the conference talking about some of the stuff, getting their gut check about What they were excited about, you know, the big technical announcements were like the hand tracking, being able to eventually be able to track your hands, had a chance to do that demo. They announced a Facebook horizon, which is kind of like their vision of the metaverse, had a chance to do that as a demo as well. I'll be talking about that with John Oakes, just for more of Facebook's overall strategy and social. And there is an Oculus Link to be able to connect your PC VR into the Oculus Quest to be able to actually play virtual reality games on your Quest that were being driven by your PC. And there's a number of different other games and other announcements as well that I'll be diving into as I talk to these different people throughout this series. But I'm going to start this series with Walter Greenleaf. He is somebody who has been in virtual reality for over 35 years now, and he's really focusing on medical applications. He's somebody who is both advising a lot of these different companies that are the forefront of these different medical applications, So he's super excited about a lot of the potential of what's happening with virtual reality. But at the same time, he also had some concerns about what he, I guess, didn't hear from the Oculus Connect keynote. And his concerns, I think, match a lot of my own concerns in terms of the deeper ethical questions around what's the business model behind Facebook as they move forward? Are they going to continue with their surveillance capitalistic model of trying to harvest as much data as they can? And as we move into this realm, where a lot of this biometric data is going to be probably classified more along the lines of medical information, then what are the different ethical implications of that? And I think Walter was feeling a little bit of unease of not hearing a clear vision of what Facebook's vision for this was. And I mirror a lot of that as well in terms of that, not just being a part of the conversation when it actually should be, especially when they're announcing things like the capabilities of being able to use the cameras on your VR headset to scan your environment and to do this whole photogrammetry scan of all your private spaces, which on the one hand could be amazing for being able to invite people over into your living room. But, you know, that's something that has a lot of privacy implications that weren't necessarily talked about within the larger context of Facebook and where this data goes and who owns this data, what kind of control you have over that data. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Walter happened right before John Carmack's keynote on the second day of Oculus Connect on Thursday, September 26th, 2019 at the Oculus Connect conference in San Jose, California. So. With that, let's go ahead and dive right in.
[00:03:36.116] Walter Greenleaf: Well, hi, Kent. Good to see you again. I'm Walter Greenleaf. I'm a specialist in the medical applications of VR. And I guess one notable item is that I've been involved in this arena for almost 35 years now. I got involved a while ago. And I'm a neuroscientist by background and a medical product developer. So excited to be here at Oculus Connect 6.
[00:04:02.107] Kent Bye: Yeah, no, I've seen you at a number of these different conferences. We've talked a number of times over the years. And we've also partaked in different VR Privacy Summit and the Canadian Institute for Advanced Research workshop. And so we were just talking a little bit about our reactions to the keynote yesterday. And so what's sort of your gut visceral reaction from that?
[00:04:18.529] Walter Greenleaf: Well, I'll be very honest and very direct. I was both exhilarated and somewhat concerned and maybe even mildly super concerned. What was exhilarating was seeing the industry move forward and some of the fantastic potentials of VR technology and of course to my passion for health and wellness, the opportunities there. What I found concerning was The part of the talks that talked about building both an AR and a VR platform that's going to be the way we work, the way we play, is going to be the ultimate interface between us and other people in the world. I think there's some fantastic opportunities. but also some severe privacy issues and those are what concerns me. The fact that we can digitize the world and understand what objects are in the world and how people interact with the world means we can diagnose, for example, different health conditions. And on the positive side of that, we can help improve people's health. But on the privacy side of it, I'm very concerned that we don't have the ethics and privacy systems in place to keep up with the rapidly evolving pace of the technology. I'm concerned that your behavior, my behavior, the way we interact with the digital world, is going to reveal our mental health status, our mood, our interests, our gender biases, so much personal information that I'm worried about the business model of that. I'm worried that it's going to be protected and I'm worried on the other side that people are going to exploit us to either discriminate against people or to provide barriers to access or to not leverage it the way it should be. So in general, I left feeling excited about technology and its implications and what's going on here at OC6. I wish there'd been more discussion about the issues in terms of the business model behind all this information and about the privacy issues.
[00:06:13.279] Kent Bye: Yeah, I mean, I think I left with the same reaction, which was that there was a lot of things that weren't said. In fact, somebody asked me beforehand, they said, what's your wild prediction about what's going to happen? I was like, well, the wildest prediction is that Facebook will come out and say we're doing a privacy-first architecture for VR, and that we've come up with a new business model that we're getting away from surveillance capitalism, and we're going to commit to OpenXR and WebVR and an open metaverse. That would be super wild to hear that. And that would be shocking and surprising. There's this closed-wall garden approach, but also what I think of as a bit of this colonial mindset, which is like, we own anything we capture through our technology, so we're going to own your data, we're going to own your environment, we're going to be able to seize it and capture it and use it for our benefit. And perhaps people are going to have some benefit for their own, but I guess there's the risk of asymmetry of the power between what they're able to do with this information and how they're able to subtly change our environments to change our behaviors. So if you can predict what we're doing, you can change what we're doing. And I feel like that is the deeper ethical context that was just absolutely no discussion about that at all. And I just found that profoundly disturbing.
[00:07:20.922] Walter Greenleaf: I totally agree. And I think on the positive side, the affordances enabled by this next generation of AR and VR technology are going to be so positive. But I do think we have to bake in right from the beginning the concern for making sure there's an alignment between the business model and the fundamental architecture of the systems and what provides privacy and protection and support for individuals' needs. The way businesses are structured in the U.S. right now, there's a fiduciary responsibility to maximize shareholder return. And that, in a way, gives almost an unobstructed pathway to pursue profit without necessarily building in social benefits, social good. And I think the only way to push back about that is for regulatory pathways, and I hate to say regulation, But I think there needs to be either an industry-generated effort to address these concerns before it gets too far ahead of us. Actually, it's already very far ahead of us. Or at least elaborate what are the best practices in this arena.
[00:08:23.227] Kent Bye: Well, I think one of the challenges that I found after going through the VR Privacy Summit at Stanford with you and 50 other companies and individuals from the VR industry, as well as talking to people from like Mozilla, Magic Leap, and 6D AI, and then Agency, we had a whole panel discussion at SIGGRAPH, talking about the different architectures for privacy, and also talking to Dr. Anita Allen, who's one of the founders of the philosophy of privacy. And she said at the American Philosophical Association this past January, she said, there is no comprehensive framework for privacy. that philosophically, we don't have a model for what should be public and what should be private. So without that theoretical framework, it's left up to each of these individual companies, whether it's Google or Facebook or Apple or Microsoft. As they're interfacing with their customers, they have to define what that line is. And the same thing when talking to the panel at SIGGRAPH was trying to define a universal framework for privacy because it's so context-dependent. How would you even start to bound it? And a lot of people were like, well, we have an operational definition, and we're going to use this in order to operate. But there seems to be a deeper values-driven approach to that collapsing of the unboundedness of all the human experience to try to define what should be public and what should be private. And I feel like now is the time to set forth what that framework is and to have that drive the design decisions. And in the absence of that, you have things like Facebook, without really thinking about it, saying, we're going to capture everything in your home, everything in your environment, all of your biometric data. We're going to capture and record all of that. And that's great. We're all happy, but it's sort of like, At the same time, they're sort of doing this again, move fast and break things without thinking about the ethical or moral unintended consequences of all of this. So it just makes me feel like screaming of like, hey, let's pause or stop or can we just at least have an honest conversation about it before we sort of rush off into this optimistic future without actually having a deep, honest reflection about all these various issues.
[00:10:18.038] Walter Greenleaf: I share your concerns, but I'm sure that there are many people listening to this podcast that will also share it. And, you know, let's take action. Let's work together to solve it. And I don't think that the companies that are out there and our players are, like you said, it's move fast and break things, but we can maybe help make sure that there's some bubble wrap around the things and help them evolve the right policies. But it's going to take some effort. We need to work on it.
[00:10:45.280] Kent Bye: What do you think would be a good approach next step? Because we have HIPAA, we have a medical context, we're moving into a realm where a lot of this stuff that used to be medical information is now going to be consumer available to companies and anybody who makes an application. So what would you suggest? Because the unboundedness of the privacy policies are so context dependent, we have context and laws and frameworks from the medical field. Do you feel like that's a good starting point?
[00:11:09.577] Walter Greenleaf: You're absolutely right. I think starting with the privacy rules that already have been established to protect medical data, for example, and further elaborating that to this next generation of technology, I think a working group needs to come up with our recommended best practices, and then we can work to promulgate them. I gave a talk at a university in Costa Rica last week, and someone who was in the audience who's working with the government to come up with their privacy policies came up to me and said, How can we work together to do this? Now, if one country were to come up and implement their policies, now Costa Rica's a small country, it may not have the leverage on the world economy, but it could set an example. And I think if we just come up with what we think is the best way to proceed, some key points, and get that out there as a pathway for people to work on, that's a great starting point. And I do think that we can look to what's been evolved in the medical arena and start from there.
[00:12:10.699] Kent Bye: I know that you've been traveling around giving keynote lectures about what is happening in the medical field and that there's so many different industry verticals that are happening with innovation with VR and medicine and so what have you been seeing that's new or exciting that you've been adding to your presentation that you've been giving?
[00:12:27.667] Walter Greenleaf: Oh, gee. Ken is asking me what my favorite novel is. There is so much going on. I think what's really exciting to me is that some of the more difficult problems that we've had in healthcare, we finally have some breakthrough approaches. And that's in the field of behavioral and mental health. Anxiety, depression, mood disorders, addictions, cognitive aging, all these things have been intractable problems that we haven't had the right tools to do assessments or we haven't had the best tools to do it. But finally now we have a way to do what we call precision medicine where we can, instead of asking for a subjective report of how you're feeling, we can challenge you with a virtual environment and see how you behave. We can come up with culturally sensitive and age-appropriate assessment environments, and this will allow us to tailor interventions to the individual. And because of the power of VR, we can also use it to shift people's moods, people's attitudes, people's behaviors, and help people who are struggling with addictions, for example, or people who have social anxiety disorder and need to practice social skills. There's so much we can do. So, other fields like stroke rehabilitation and surgical training and Every medical arena is being impacted by VR and AR technology. But the ones I'm personally very excited about, because I think it gives us some new affordances and new capabilities, is the field of behavioral mental health.
[00:13:49.236] Kent Bye: So for you, what are some of the either biggest open questions you're trying to answer or open problems you're trying to solve?
[00:13:57.581] Walter Greenleaf: Well, I guess some of the biggest problems that I see that need to be addressed, I'm really worried about the fact that there are so many different platforms out there. It's very hard for a hospital system or a healthcare network to say we're going to focus on this one way of going. It's the usual conundrum of, will our technology be obsolete next year? And the answer is yes. The technology is evolving really, really fast. So again, I think in addition to coming up with privacy protection, which is one of my biggest concerns as we talked about, I think it would be nice if we can come up with a way of having data portability, and medical environment portability across platforms. And there's certainly ways to do that right now, but they're not easy. That's my concern is that there's so many people who are leveraging the power of this technology in the health care arena, but I'm worried that it will be fractionalized. And so that's something that is on the top of my mind right now.
[00:14:50.270] Kent Bye: And finally, what do you think the ultimate potential of immersive technologies might be and what they might be able to enable?
[00:15:00.783] Walter Greenleaf: Well, it's going to be part of everything we do. So what I hope it will do, what I believe it will eventually do, is it will allow us to have healthier, happier, more enriched lives. And again, I think the fact that we can now address some very difficult problems such as addictions, anxiety, depression, I think it will help us have more comfortable lives too, not just be more productive at work. but connect us with each other in ways that break through barriers and make us feel, I don't know, just enjoy our lives better.
[00:15:33.496] Kent Bye: Is there anything else that's left unsaid that you'd like to say to the immersive community?
[00:15:37.289] Walter Greenleaf: Well, I guess, Kent, I just want to thank you for your efforts as an advocate, for connecting all of us, for providing information, and for your efforts, I know, to help address some of these issues, such as ethics and privacy, too. So thank you. Awesome.
[00:15:51.604] Kent Bye: Thank you very much. And yeah, enjoy the rest of your Oculus Connect there. So thanks.
[00:15:56.690] Walter Greenleaf: All right. Let's go hear that talk that's about to start.
[00:15:59.911] Kent Bye: So that was Walter Greenleaf. He's been working with the applications of virtual reality into medicine for over 35 years now, really tracking the progress and evolution of that. He teaches at Stanford as well as advises a number of different companies looking at the applications of VR into medicine. So I have a number of different takeaways about this interview is that, first of all, well, this was an interview where I just wanted to capture the zeitgeist of some of this gut feeling that Walter had shared with me briefly before the start of the Oculus Connect 6 keynote with John Carmack. And that is this, on one hand, a lot of excitement about the potential of hand tracking and tetherless Quest headsets, something just people are very excited about in terms of adoption with all these different applications that were never even possible before. From Walter's perspective, he is totally stoked in terms of all the different ways in which VR could be used for good, for health and healing. And I think on the other side is a lot of things that weren't being said in terms of the deeper ethical frameworks that Facebook is using. They did not come out saying we're doing a privacy first architecture for everything that we do. We're really concerned about that. And that's juxtaposed to what Mark Zuckerberg had said at F8, which is the future is private. There was no similar sentiment of that the future is private at Aukus Connect. And I think that. when they come out and say in the keynote that they're going to suddenly have the capability to be able to map all of both the public areas and all your private areas without any discussion about the deeper consent around that as to whether or not that is a good thing or legal implications of that, especially when it comes to the third party doctrine. Anytime you give over any data to a third party, then you have no reasonable expectation of that being private. So what's it mean for you to start to scan all of your private area in your home? The government can start to go to Facebook and say, we want to get this information and they don't need a warrant at all. So there are certain aspects of the third party doctrine that need to either change or there needs to be some deeper discussions about the ethics of. where this data is going, what are some of the unintended consequences and to have that a part of the conversation. You know, I think Facebook has an opportunity being the market leader to be able to be ahead of this conversation and start to at least acknowledge that there's a lot of unanswered questions and to just kind of push forward and say, yeah, this is possible. And it was in the context within the keynote saying that, yes, this is technically possible. They weren't announcing a specific date or anything. They were just trying to show like the potential of VR for you to connect to people. And to be able to match up your personal living room and to be able to hang out with family members and to make it feel like you're actually hanging out with them, that indeed is going to increase your sense of place presence and allow you to create these additional memories with your loved ones. But there are also lots of ethical implications of that technology alone. And so in my discussions at Oculus Connect, there was a bit of an emotional labor within the community that had to talk through it of like, yes, this is awesome, but also just people feeling a little bit unsettled. Moving forward, Facebook is really trying to push forward the platform, but make it a closed platform. They weren't advocating for the open web, you know, a number of years ago, Nate Mitchell got up on stage and really talked about the merits of web VR and the potential to be able to bring in all these different things. Well, we haven't heard anything about web XR web VR from the keynotes in a big fashion from Facebook for the last two years now. And. It's really like they're pushing forward this vision of a closed walled garden metaverse. And it's going to have its advantages in terms of having a consistent user experience. But when it comes to like the future computing platform, there is nobody that I talked to that is excited to have Facebook own everything that they want to try to like create this app store ecosystem, have everything very tightly controlled. And so there's a sense of being unsettled that we're not really getting a lot of. assurance from Facebook saying that, you know, we believe in the open metaverse, we believe in open XR, we believe in open platforms, we believe in privacy first architectures, you know, there's absolutely none of that in the official capacity. And I think it just was unsettling for a lot of people. And as I was having a lot of different conversations with people at office connect, that was a theme that I think came up over and over again, in terms of what wasn't being said. So I wanted to start this series just by talking to Walter and to have him just talk about the things that he was concerned about, which I think is these deeper ethical considerations. Diving into the ethics and the privacy of immersive technologies, I think that's a series that I'm going to do after this one. And then I'll be asking my Patreon members to kind of get some feedback in terms of what they want to hear next. education and training was a hot topic at Oculus Connect 6 just for me as I was doing a lot of the different coverage there seems to be a lot of traction when it comes to the applications of VR as a training and so just talking to see what people are doing what they're finding and the different applications and yeah just to kind of hear from other people. I covered that a little bit with Stryver and what they're doing but there's a lot of other companies beyond Stryver that are doing a lot of really great work when it comes to education and training and diversity training. And so I wanted to start to cover some of those other companies as well. Also, VR was featured pretty heavily within the keynote in terms of surgical training. And they had the statistic that people who did VR training, 83% of them without any further additional one-on-one advisors were able to then go in and basically do the surgical training on an actual cadaver. And At South by Southwest, I talked to another surgical training company that was talking about the power of surgical training and be able to do these fast iterations. And it's a type of spatial memory that the more iterations that you have, the more that you can learn. And it's very difficult to do that if you don't have x-ray vision. And so there's just a lot of things within VR that you can start to train your procedural spatial knowledge in ways that just translate super well into things like surgical training. So that was something that was highlighted in the keynote and something that was a topic that came up in previous Oculus Connects when I talked to Osso VR, as well as I've talked to other surgical training as well. So we'll be diving into all that later. And just to kind of like tie it back to the medical applications, Walter Grunew has an amazing slide deck that he has up on SlideShare. And I think in his latest iteration, he says that there's over 20 different clinical sectors of medical applications with about 157 emerging medical VR AR companies that he's been keeping track of. And that's applications for everything from phobias, PTSD, stress management, relaxation, surgical training and planning, physical rehabilitation, pain and difficult procedure mitigation, depression treatment, cognitive rehabilitation, optical rehabilitation, addiction treatment, neuropsychological assessments, cognitive training and wellness, sports training, disability solutions, speech therapy, autism spectrum disorder, stroke and traumatic brain injury, patient education, preventative health, ADHD and senior care. So those are about 20 of the different clinical sectors that Walter has identified. And so all these different applications within the medical field, and I expect that over time, it's going to continue to spread out into more and more different medical applications. And so having a tetherless VR system deployed out in the field, I think it's going to have a lot of really amazing applications. And I think Walter's extremely excited to see where this potential is going. And especially for him and thinking about the. Medical applications privacy is a huge part of that He was talking about just like the different ways in which having access to all this biometric information Even if it's how you move or if you start to integrate what your environment you're gonna be able to actually diagnose people and this is potentially gonna be able to have Medical information that's diagnosable in the hands of these commercial companies. And so how do you deal with that? do you start to look at different HIPAA regulations and start to try to and see if there's a deeper regulatory framework because there's a bit of a conflict of interest between the fiduciary responsibility for Facebook to serve their shareholders for profit and this conflict of interest of what's in the best interest of consumers to be able to have their rights to privacy. And that's a different fiduciary relationship. So something that needs to be settled out either through regulation, which is what Walter says, you know, he hates to get to that point, but It doesn't necessarily see that the companies are just going to do that on their own free will. So it's in this wild west phase of the development of virtual reality, needing people to think about these deeper considerations. And, you know, if there's a big outrage of people that are demanding it and saying, we are not going to buy this virtual reality headset until you start to give us the same affordances that you're providing these enterprise companies, then, you know, that'd be one thing, but I don't foresee that happening right now. I think everybody in industry is just trying to get VR bootstrapped and up and running. And I talked to Andre Elijah about this a little bit because we've had a number of different back and forths on Twitter, and we had a chance to kind of bury the hatchet and suss out each of those different perspectives. And as an independent developer, Andre Elijah is like, hey, look, we're just trying to get VR working. Let's not prematurely slow down that by being too cautious and trying to come up with too much of these regulatory frameworks before things have even launched off the ground yet. So there is a bit of that tension that things do need to progress, but I think we also need to have some checks and balances of some of these deeper ethical issues for companies like Facebook as we move forward. So that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of Patreon. This is a listener-supported podcast, and so I really do rely upon the support of listeners. Even if you listen to the Voices of VR podcast periodically, or if you sort of tune in and out This is a podcast that goes around the world and just helps people get bootstrapped into the immersive industry to provide a larger context. And also I've recorded over 1300 oral history recordings of the evolution of virtual reality. Um, that's something I need a lot more support to help just maintain that and to continue to give this out as a resource to the community, uh, to be able to download all these different episodes, uh, to cover my bandwidth costs, but also just my cost of living to be able to travel and to survive. as an independent artist, creator, oral historian, to be able to continue to document the evolution of this as it unfolds, but also to talk about these different issues that not a lot of other people are really digging into, especially around the privacy and ethics of virtual reality. So, if you want to see more of this coverage and for me to continue to do what I'm doing here and provide this as a service to the community, then please do consider becoming a member of the Patreon. Just $5 a month is a great amount and allows me to continue to bring you this coverage. So, you can donate today at patreon.com slash voicesofvr. Thanks for listening.
