UCSF professor Adam Gazzaley is a pioneer in the realm of experiential medicine as he’s getting a video game approved by the FDA for ADHD treatment. He’s a co-founder of Akili Interactive and the chief scientist at JAZZ Venture partners, where he’s on the bleeding edge of the latest immersive and interactive technologies, integrating biometric feedback, and helping to create a whole new class of digital drugs through video games. I had a chance to catch up with Gazzaley briefly at the XTech Conference in 2017 where we talk about his book The Distracted Mind: Ancient Brains in a High-Tech World, the Glass Brain project to be able to see inside of your brain in real-time through an immersive EEG, the foundational principles of neuroplasticity, and how far we might be able to push the potential of neuroplasticity with immersive technologies.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
Here’s some photos of the SynSync Sensory Immersion Vessel that Gazzaley announced at the Awakened Futures Summit on May 18, 2019.
.@AdamGazz just announced SenSync and the Sensory Immersion Vessel that he's been working on as a form of experiential medicine at the Consciousness Hacking Awakened Futures Summit pic.twitter.com/0foGZU94pQ
— Kent Bye (Voices of VR) (@kentbye) May 18, 2019
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.412] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR podcast. So continuing on in my series of looking at the future of neuroscience in VR, there's a neuroscientist named Adam Ghazali at UCSF who has been working in VR for a number of years and has a lot of really interesting projects that are looking at this intersection between games and health and healing, as well as immersive technologies and neuroscience, really on the bleeding edge of what's possible when you start to blend together virtual reality and neuroscience. He's also got Jazz Ventures, which is investing in different technologies, but he's also got Akili Interactive, which is getting like FDA approved games. This interview I did with Adam way back at the Experiential Technology Conference in 2017. This was right after he had just published a book called Distracted Mind, Ancient Minds Living in a Modern Time. And so he gave a keynote at the Experiential Technology Conference there. I've since run into a number of different times since then. He was at the VR Privacy Summit at Stanford University, and he had a lot of great ideas. I did a brief interview with him, potentially having an institutional review board of privacy. What's it mean to look at these different privacy policies and have an IRB start to overlook that? That's a big open question as to whether or not these big companies are going to have something like an IRB. Is something that is held within the scientific research community? And what would it mean to start to bring some of those concepts and apply that to privacy? And then I also ran into him at the Wiccan Futures Summit. I didn't have a chance to do an interview with him, but he actually was making some really interesting announcements there at the Wiccan Futures Summit, which was put on by Consciousness Hacking, looking at the intersection between psychedelics and immersive technologies like VR, as well as different meditative contemplative practices. And so Adam announced some stuff there with this whole sensory immersion vessel that he's going to sort of do all this synthesis of your biometric data and then be fed into real time into the experience as it's unfolding. And so I'm super curious to see where that ends up going. But he announced that back on May 18th, 2019. And Noah Falstein mentioned that he's collaborating with Adam Ghazali in different ways. He worked on the Neuroscape game, which was on the cover of Nature Magazine. And at the Awakened Futures Summit, he talked about how they're going through the process of getting FDA approval to have a video game be prescribed as a treatment to people with ADHD. So doing this kind of real-time biometric feedback into the game, and then that's gonna be like an FDA-approved drug. And so at the Awakened Futures Summit, he was talking a lot about the future of experiential medicine. But this was a couple of years before that at the Experiential Technology Conference back in 2017, where he's just kind of talking about some of the things that he's looking at at the intersection of neuroscience and virtual reality. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Adam happened on Wednesday, March 15th, 2017 at the Experiential Technology Conference in San Francisco, California. So with that, let's go ahead and dive right in.
[00:03:13.856] Adam Gazzaley: Adam Ghazali, I'm a professor at UCSF. I'm a co-founder of Akili Interactive and also a co-founder of Jazz Venture Partners. So I'm involved in VR from lots of perspectives, from the academic research perspective of creating VR experiences and then validating them through experimentation to see if they can improve brain function. And as an investor in companies that are doing interesting things in the space to help human performance.
[00:03:40.428] Kent Bye: Right, so you're giving a keynote this morning and talking a lot about this distraction culture that we have, and then some research and writing about this. Maybe you could talk a bit about the state of where you see the cognitive load that we're receiving from all of this today, and then how we deal with these ancient brains in a modern time.
[00:03:59.402] Adam Gazzaley: Yeah, so my research focus, I'm a neuroscientist and a neurologist, and my research focus for many years has been to understand how interference impacts our brain function and our performance across many different cognitive domains. We study that with functional brain imaging. We study it in healthy young people, but really have asked the question how it changes as we get older. Recently, I wrote a book, a pop science book on this topic called The Distracted Mind, Ancient Brains in a High-Tech World, where we really take an evolutionary perspective on how our brains have evolved in a strong way and where the limitations live, and then how technology has challenged us. And for me, it was important to tell that story because my work has focused on technology as a positive tool and create all these amazing, wonderful influences in our lives. But I like framing it also from the perspective of the challenges that technology has delivered upon us through distraction. I feel that if we're going to move forward in an informed way, we'll do better than just blinding ourselves and not thinking about the human side of technology.
[00:05:07.888] Kent Bye: Yeah, the way that I see it is that with virtual reality as an experience, it's almost like you have to cultivate this contemplative consciousness in a way of, like, you don't just dip in and out of, like, reading a book. You actually kind of have to dedicate yourself the time and set aside that time to really go into a VR experience. And so I see in some ways the virtual reality is trying to get us to completely focus. And so have you started to look at that in terms of what the virtual reality does and to the context of this distracted mind?
[00:05:36.711] Adam Gazzaley: Yeah, it's a great question and I noticed that too, obviously the complete immersion and even the sense of isolation that you experience right now when you're in most head-mounted displays does actually make you focus more than you do in reality, non-virtual reality. can't really check a text in virtual reality right now and so I hear a lot about that aspect of it and that will probably change. I think the social evolution of VR becoming a multi-person experience will change that and that is going to happen undeniably. Our main use of VR in our center, Neuroscape, at UCSF is to develop content that is directed not at entertainment, which as we know a lot of VR content is being developed for, but rather tools to actually help improve our attention abilities and our memory. So we have one program that we've created called Virtual Attention that helps you better distribute your attention in space and time so that you have a broader attention. And that was a seemingly to us a good use of VR because VR has wide depth of field and depth. which no other platform really does. So that's something that we're doing on the attention domain. On the memory domain, we've built a game called Labyrinth, which challenges you to navigate different virtual 3D mazes to help build memory performance. And then we take all of these tools and put them through randomized double-blind placebo-controlled trials to actually show that they have benefit, and also record brain activity before and after to see what the impact is on structure and function.
[00:07:10.210] Kent Bye: And talking to a lot of these different researchers, I just hear that there's a fundamental challenge of proving that doing these exercises in technology, whether it be a 2D platform or VR, has a transfer function such that you have these assessments and so maybe you have a near transfer that you do better on the assessments. Can you speak to that as to what your intuitive sense is as to whether or not you could start to cultivate this deeper sense of attention in this contrived context of VR, but yet see real measurable results or just even qualitative results in your life?
[00:07:40.442] Adam Gazzaley: Yeah, I mean we hope to, and we have already had some success with video game technology that we custom created in-house to build experiences that target neural processes that have benefits across many different interactions that you might have. So, of course we see improvement in the game environment itself, we'd expect that whether it's VR or tablet, but we have shown and we hope to continue to show that if we target very fundamental neural processes and through adaptive closed-loop algorithms apply continuous pressure to them that they will change because of plasticity over time and we can optimize them. So that's the hope and we think that VR might allow us to do it at a higher level than we have with our mobile games because of just the deeper immersion and real-world experience that you get in VR.
[00:08:30.642] Kent Bye: Yeah, I often say that the killer app of VR is neuroplasticity. And I point to James Baja's vivid vision of being able to actually rewire his brain to be able to see in 3D vision, where he wasn't able to do that before. From your neuroscience perspective, what can you tell me about neuroplasticity as a principle, and how far we can extrapolate that into potentially untapped potentials of the human?
[00:08:53.808] Adam Gazzaley: Yeah, I mean neuroplasticity is a fundamental principle that is completely and consistently supported by all neuroscientists, anyone I know of. So, you know, the field has its own complexities and even controversies of how you harness plasticity and what the benefits are outside of what you're training on and sustainability. But neuroplasticity itself is a solid foundation. It's something that we didn't always appreciate. We thought that after critical development periods, the brain was very static and only really exhibited degradation, not continued development. But, you know, plasticity occurs at every level of the nervous system, its function, its structure, its chemistry, all largely in response to experience, although you can have a plastic response as a recovery from injury. And where the limitations of that are, you know, we don't really know yet. And so we look at the new space that we've created of experiential technology, including things like video games, as a very interactive way of creating an experience to harness plasticity. And then all the other tools like VR and AR and motion capture, which lead to even better, you know, richer interactivity. as potential paths of showing that we haven't even come close to tapping the limits of plasticity.
[00:10:13.711] Kent Bye: Yeah, that's my sense. And that's what I think is so exciting. And in your presentation today, you showed a live model of your brain, in that you said that you've been in VR with an EEG, being able to look at your brain activity live in real time. And maybe you could tell me, as someone who's a neuroscientist, what were you able to see?
[00:10:32.881] Adam Gazzaley: Yeah, I mean, it's an incredible experience to step into your own anatomy in VR, no matter what it is. And I think the brain is obviously the most personal and the most dramatic, if presented well. And we have a technology called the Glass Brain developed at UCSF, but we're collaborators from UCSD and NVIDIA. And it's a really very beautiful, dynamic representation of both the function of the brain as captured through EEG, but overlaid and integrated with an MRI image. So the structural data is all personalized to the individual. And seeing it on a screen is amazing. Seeing it pre-recorded is amazing. But seeing it in VR, real time, is the most surreal.
[00:11:15.244] Kent Bye: Awesome. So for you, what do you want to experience in VR?
[00:11:20.978] Adam Gazzaley: Well, I like it all. I'm just pure connoisseur of everything that is pushing the envelope. I like art, music, all sorts of games. From a research perspective, we're interested in VR for impact, but as a person that just loves technology and loves pushing the boundaries, I'm delighted by all of it.
[00:11:44.387] Kent Bye: Awesome. And finally, what you see is kind of the ultimate potential of virtual reality and what it might be able to enable.
[00:11:52.166] Adam Gazzaley: I think it will open up an aspect of human consciousness that we just haven't tapped yet. Our ability to communicate with each other and experience the world around us with a lot of the restraints that reality places on us is just going to take us to places that we haven't dreamed of yet. So that, to me, is the most exciting. It's just the open frontier feeling of it.
[00:12:17.517] Kent Bye: Awesome. Well, thank you so much. My pleasure, man. Thank you. So that was Adam Ghazali. He's a professor at UCSF in San Francisco. He's a co-founder of Akili Interactive, as well as a co-founder of Jasmincer Partners. So I've a number of different takeaways about this interview is that first of all, well, Adam's at this intersection of neuroscience and VR really at the heart and doing, I think, some of the really most interesting research in terms of looking at biometric data and how that could be fed into in real-time interactions with virtual reality environments. So in this conversation that I had with him back in 2017, he had just given a keynote at the X-Tech conference, the Experiential Technology Conference. A lot of the stuff that Adam was working on with Akili Interactive is like the Neuroscape game, which then the follow-on projects that are getting these FDA approval to be able to use gaming and experiential technologies as like this experiential medicine to be able to actually be prescribed these different treatments that have like video game components with real-time biometric feedback. At the Awaken Future Summit on May 18th, 2019, he announced the Sensync and the Sensory Immersion Vehicle that he's been working on as a form of experiential medicine. It's like this whole pod that has all sorts of different haptic feedback. measuring different aspects of your biometric feedback. And I'm assuming there's going to be like some real time biometric feedback fed into that. He announced it at the Wake and Future Summit. I didn't have a chance to talk to him more, but I'm sure we'll be hearing more about that at some point. I think the idea is to try to include as many of the different sensory modalities as you possibly can. And this sensory immersion vessel seems to have haptics and visual and audio and who knows what's happening with the taste or touch, but trying to stimulate all these different aspects of your body. in real time. It looks super cool. I'm looking forward to really checking it out at some point. Adam wrote this book about the distracted mind, ancient minds in a modern time, and just all the different challenges that we have in this day and age of how technology is trying to hijack our attention and get us into these states of distraction. But VR offers this opportunity to have complete immersion and isolation and to actually have you more deeply focused on one thing. They're developing content that's not based just on entertainment, but as tools to improve memory, to better distribute your attention in space and time, a labyrinth to be able to navigate these 3D mazes and to be able to improve your memory. So all these things that are trying to use these experiential technologies to actually improve yourself in different ways. And so there have been targeting neural processes that have benefits across many different interactions through adaptive closed-loop processes. So again, this is a closed-loop concept that I talked to with Tej in the mind maze. Being able to actually have this biofeedback that is measuring what's happening in your body, being fed back into the experience, and then the experience adapting and changing, and then constantly listening to how your body's changing. So having these closed-loop systems, I think, is huge potential for what the future of this intersection between neuroscience and VR and all this biometric data that's coming off our body and finding ways to Understand it make sense of it and to feed back into these dynamic experiences Just briefly talking about plasticity and the importance of neuroplasticity as being this like fundamental concept within neuroscience Which is just to say that your brains are very plastic there used to be thought that there's a critical period of learning but actually it turns out that our brains are constantly learning and adapting and so If you have new stimulus that you're putting into your brain, then it can change and adapt. And so what's it mean to be able to have these virtual environments where we're now able to create these experiences that we've never had before? What does that mean to be able to kind of unlock these latent human potentials that we've never knew that we actually had? And we're able to potentially adapt and change based upon the experiences that we're receiving within these virtual environments. So the concept of plasticity is a pretty well established within neuroscience and just thinking about what's possible now with all these immersive technologies that we're able to create these closed loop systems that are able to modulate our experiences. And what Adam said is that he doesn't think we're even close to really tapping out the potential of what's possible with this concept of neuroplasticity. So they also have this concept of the glass brain being able to actually be in an immersive environment and have an EEG on at the same time. So you can actually like watch your brain firing in real time, which I think is going to be super trippy. Once that gets up to a certain fidelity, especially with these different headsets, like my maze or normal, which I'll have an interview with them as well, kind of exploring what they're doing, but what's it going to be like to be able to actually be in an immersive environment and to be able to actually watch your brain firing in real time. And that's what they were doing with the glass brain. Adam says it's very surreal to be able to see that happen in real time. And, you know, finally, just saying that, you know, all of this fusion of these technologies have the potential to kind of open up different elements of the human consciousness and to be able to take us to places that we haven't even been able to dream of yet, which I think is one of the most exciting potentials with all of this, especially with these different concepts of neuroplasticity. At that same XTAC conference, I had a chance to talk to David Eagleman about this concept of sensory addition and sensory replacement. So you can start to turn your torso into an ear, so finding other ways of getting the information from the environment If you have some sort of deficit in your hearing, then can you get those same signals into your mind through different channels? So yeah, just thinking about how can our body adapt to be able to either add new senses or be able to modulate or change the senses that we already have, or to be able to add completely new capabilities of our sensory experience that we've never even had before. So that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a listener-supported podcast, and so I do rely upon listeners like yourself in order to continue to bring you this coverage. So if you'd like to become a supporting member and to help keep this podcast free for you and everybody else in the community, then please consider becoming a member. Just $5 a month is a great amount to give and just helps me continue to sustain the work that I'm doing. And if more people donate, then I'll be able to expand and grow and potentially add more features to the website. I'd love to be able to start to have transcripts and to. Start to experiment with what's it mean to translate all this content that I have into this whole memory palace And yeah, I've a lot of ideas about what I'd like to do But I just need more support to continue to sustain what I'm already doing but to be able to actually grow and expand out as well So you can find out how to become a supporting member at patreon.com slash voices of VR. Thanks for listening
