Oculus Connect has been rebranded Facebook Connect, and what would have been Oculus Connect 7 is now Facebook Connect 1. Facebook Connect happened on September 16, 2020, and I talked with Doug North Cook on the following day. North Cook is the founding faculty for the Immersive Media Program at Chatam University in Pittsburg, PA, as well as the founding instructor for the immersive design residency for the Falling Water Institute. At Oculus Connect 6 last year, I did a OC6 keynote debriefing with North Cook about privacy implications and accessibility concerns, and so I wanted to check with him again to get his impressions of the Facebook Connect 1 keynote this year.
North Cook says that the name change from Oculus Connect to Facebook Connect is good indicator of the overall tonal shift of having Facebook take more and more control over the future of VR and AR. Facebook presented a lot of their speculative designs of their vision of the future of augmented reality, but we both shared a lot of questions around the merits of an always-on, contextually-aware, AI personal assistant that requires extensive “egocentric data capture” for training with glasses that track everything in your world that correlates what you’re looking at via eye tracking data. We both didn’t find the use cases shown during the keynote to be particularly compelling, and certainly not enough to merit this level of encroaching on our own privacy, but also the privacy of those around us. North Cook wondered if this emphasis on this type of vision for the future of AR was actually more motivated by a long-term data play and desire to capture as much data as they can about us as we move about the world.
We also talked about the recent anti-trust hearings featuring Amazon’s Jeff Bezos, Facebook’s Mark Zuckerberg, Apple’s Tim Cook, and Google / Alphabet’s Sundar Pichai. These tech companies are becoming bigger and more powerful than any individual government, but they have little to no democratic transparency for how their tech platforms are becoming the de facto online public spaces. North Cook says that any consultation with any of the big tech companies would be under strict NDA, and so these companies are reaching out to experts for consultation on issues around privacy, but these discussions are often happening behind closed doors.
We cover some of the other news and announcements from Facebook Connect 1, but I’m also left with the feeling of finding it more and more difficult to have an interactive dialogue with Facebook on topics within the public sphere. Facebook’s previous social VR efforts have been shuttered, and the Facebook Venues and Facebook Horizon are still in early beta and closed beta phases, and so there was little effort by Facebook to try to have meaningful social VR gatherings that brought that wider XR industry together. Their virtual conference sessions were all pre-recorded, and there was no opportunity for any type of live Q&A. A theme coming out of Facebook Connect 1 was that there was little to no effort to facilitate any meaningful two-way communication between the wider XR community and people working at Facebook.
There are so many ethical questions about the future of AR and VR, but yet there has been very little public engagement on any these issues up to this point. Facebook CEO Mark Zuckerberg did tell The Verge’s Casey Newton that “One of the things that I’ve learned over the last several years is that you don’t want to wait until you have [XR ethical] issues to be discussing how you want to address them. And not just internally — having a social conversation publicly about how society thinks these things should be addressed.”
But as far as I can tell, all of these “public” conversations that Facebook may have been having on these issues are still happening behind closed doors. Facebook put out a white paper looking at the challenges around notice and consent and said that in the coming months they want to have “conversations with regulators, civil society, academics, and other companies around the world to dive into the questions raised in this paper.” But when I asked for details for how to participate both publicly and privately, then I get the distinct impression that journalists will not be allowed to attend or cover any of these gatherings or conversations.
During Facebook Connect, Andrew Bosworth said that they got feedback from privacy advocates that it’s not about what they say, but it’s about what they actually do. At this point, I’m still seeing a lot of rhetoric about the desire to have public conversations on these XR ethics and XR privacy issues, but again I’m having a hard time getting clear answers about when and where these conversations will be happening in a context that isn’t under NDA or behind closed doors where journalists like myself are invited to participate and report on those public conversations.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
Here’s more of my real-time coverage of the Facebook Connect announcements as they happened:
1/ THREAD
Facebook Connect 7 is under way, & I'm covering the news here in this thread.Oculus Quest 2 already leaked, Zuckerberg is talking a bit about their augmented reality glasses plans.
I have many open privacy questions.
Livestream here:https://t.co/nPEFRzqxze pic.twitter.com/DciEORKjrZ
— Kent Bye (Voices of VR) (@kentbye) September 16, 2020
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to The Voices of VR Podcast. So on Wednesday, September 16th, 2020, was the Facebook Connect gathering. It used to be called Oculus Connect, where the entire virtual reality community would come together and hear the latest announcements and It would usually be over the course of a couple of days, and it's also an opportunity for the community to come together and to connect to each other. There wasn't a lot of that that happened this year in terms of the community coming together. Facebook still had Facebook Connect, where they announced lots of different announcements, but there wasn't the same amount of community gathering that has happened in years past. So usually at Facebook Connect is an opportunity for me to do lots of interviews with the community to check in. And rather than do that, what ended up happening was a whole private discord server had come up and was having lots of different discussions late into the night after Facebook Connect. And then the following day, I had this conversation with Doug Northcook, who I talked to last year after the Oculus Connect keynote, just to start to unpack and make sense of what we just heard and where this is all going. This year there was actually quite a lot of augmented reality speckle of design that was being shown. There's a lot of other issues with Facebook in terms of consolidating down into the unified Facebook accounts and then the Oculus Quest 2 and announcement of that was a big part of Facebook Connect. But I think probably the biggest part was the vision of the future that Facebook was putting forth in terms of where they see this all going and potentially even their deeper intentions for seeing VR as a stepping stone into this bigger vision of augmented reality and this always on AR device that you have on your head that has a contextually aware AI. And in order to get there, they need to capture a lot of egocentric data capture. And what are the implications of that? So I got a chance to talk to Doug and to unpack all these different announcements and to reflect upon it as well. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Doug happened on Thursday, September 17th, 2020. So with that, let's go ahead and dive right in.
[00:02:11.298] Doug North Cook: Yeah. Hi, I am Doug Northcook. I am the founding faculty for the Immersive Media Program at Chatham University in Pittsburgh, and I'm also the founding instructor for the Immersive Design Residency at the Fallingwater Institute.
[00:02:26.368] Kent Bye: Great. So yesterday was Facebook Connect One, formerly known as Oculus Connect Seven. So I know last year after the keynote, we actually had a chance to like digest and unpack. And right now it's the day after Facebook Connect. And I know that I had a number of different conversations last night. Sounds like you had some as well. So maybe let's start at the top. What are your takeaways and impressions from Facebook Connect One?
[00:02:52.092] Doug North Cook: Yeah, I mean, well, I think it was very much exactly that. It's a very clear pivot towards a more integrated approach to spatial technologies and a very clear departure away from just VR, right? And that's very clearly Facebook's intention of bringing everything under the Facebook Reality Labs umbrella, trying to cast this bigger picture for VR and AR, because traditionally at the Oculus Connect conference and at other Oculus events, you wouldn't really generally hear that much talked about about AR, though we got a little glimpse of that in the keynote and from Abrash last year at Connect. But this was, yeah, a really big shift in tone. Both the actual tone of how people were talking about the technologies, but also what the focus was, which was a clear shift towards focusing on augmented reality and not necessarily on virtual reality as the long-term goal.
[00:03:56.842] Kent Bye: Yeah. So in that you can see a lot of the groundwork that's been laid down in terms of VR almost being a means to an end of this larger vision that they have for AR because they're continuing to grow out the VR ecosystem. I mean, there's lots of different privacy concerns that I have. I wanted to just start with my experience of the conference just as a baseline grounding. Absolutely. Oculus Connect has traditionally been the one time that brings the big cross-section of the VR industry together. When I've gone to the previous six Oculus Connects, it wasn't so much to hear the latest news or see the demos. That's great. And I have always enjoyed seeing what the latest innovations in technology have been over the last six years. But the real value has been to bring the virtual reality community together. And I think the biggest thing I'm taking away is that it was a huge lost opportunity to not make any viable effort to try to bring the community together in any meaningful fashion. You have Facebook has the largest virtual reality community. We have VR headsets. We are hungry to be able to see what the next level is to try to connect with each other in VR. But yet both of their two social VR apps are still in beta. the venues, which is basically like a way to go in and watch the live stream with other people with lots of weird global noises. And it just was insufferable. When I was in there, it was just like, every time you would do a menu interaction within there for anybody, it was globally broadcast those sound effects. So anytime anybody was hitting a button or doing anything, it was just like, Oh, that's some leaps and blops. And so it was just like, who would make that decision? to do that unless they weren't actually using it. Or there seemed to be like a fail of iterative design of something that seems to me so basic as to not globally broadcast your sound effects to everybody in the room when they're trying to listen to a talk or a lecture. And then there was Facebook Horizon, which was still under private beta. So I didn't even have access to that. But after Carmack's talk, which got cut off early, he's like, I'm going to go into a private beta that Not everyone has access to, and oh, by the way, only 20 people can get in to be able to, to have that kind of hallway conversation. When I dropped into the venues, it wasn't much that was going on. So I don't know. I just felt like the impression that I got was that Facebook had things that they want to tell us, but they weren't really interested in creating any viable way for us to either communicate with each other or to be able to directly communicate back to them. So it was like a one way communication where there wasn't a lot of listening that was happening.
[00:06:28.038] Doug North Cook: Yeah, this was a conversation that came up with some friends of mine and I yesterday we hopped on a video call and screen shared the keynote. In part, I think, for us trying to replicate some of what we do, which is that if we're at a conference we spend most of our time hanging out in the hallway or not even in the building. And, you know, we'll go to a couple of talks, but the real focus, like you said, is on making connections with other people in the community. So there's like a sub question here, which is, what does a meaningful conference look like if it's happening virtually, right? And I've attended and spoken at a few different conferences, which I think have done that better than others over the last year. But with this, I mean, for me also, that is the value of Connect. The value is getting to see other developers, getting to talk about ideas, getting to see early prototypes of things that people are working on that they have in a backpack in the hallway. And so without that, it's just a keynote and some talks. which is not, at least for me, and I think for a lot of other people, the core value of being at a conference. And I do think, yeah, there is this bigger question, which still nobody has really proved out effectively, but they're also not necessarily trying to even do this themselves yet, which is how do you actually use these technologies to create some of those kinds of spaces? And yeah, it will be interesting, I think, to see how people will start to do that more. Like I gave a talk in VR in Mozilla Hubs earlier this year, and that was interesting, you know, but it still has all these other issues and problems and you're beholden to how the platform deals with audio and shared space and all of these things, right? So, Yeah, I am not surprised that they didn't have like a substantive social VR component to the conference because I haven't seen that work yet in a way that actually gets at what we're talking about, so.
[00:08:25.748] Kent Bye: Yeah, I think most of the virtual conferences that I've seen have thought of their conference as the talks, like that's the content of the conference. And so they can just do a live stream and then that's it. There's no other content that needs to be delivered. But for me, a lot of the experience of the conference is the relationships and the connections that I have in between spaces. And I guess that was my frustration was that there wasn't any deliberate, like, Hey, there's an opening night party. Cause even the venues, the instancing was anywhere from eight to 16 people. So even if you tried to go in, there was only like eight or 10 people that are there. If it was at a busy time, if it wasn't a busy time, you wouldn't run into anyone. There was literally no way to connect with the friends, the times that I tried it, it didn't work. Or, you know, if you wanted to actually go in. with a group of friends and just have like a private instance. There was no ability to do that. You could go into your own solo instance. So there's just so many things that the social VR has been working out and iterating on for a long, long time in VR chat or old space, you know, where that experience of that is just a lot more. developed and if people already have those social graphs within themselves, then they can start to have those different experiences. But I've definitely seen some innovations when it comes to like the first iterations of virtual conferencing and then the next iterations where it's more about trying to facilitate those emergent hallway types of conversations. And it's really like setting a context of like, we're going to meet here and we're going to talk about this. And the birds of the feather meetings of all the different virtual conferences I've been to seem to do that the most. Because it's like, here's a community of interest to have a problem or some ideas that they want to talk about. And they need to get together and talk with each other in a group where they can use their embodiment within a virtual space to navigate around. So to not have any of that, I think was, I think a lot of people that at least I saw were going to different back channels and, you know, discords and calls. And there's a process that I noticed of like sense-making of like, okay, what did we just see? What do we make sense of it? Where is this all going? And there was a lot of that. Okay. This is a lot of new stuff. What exactly is going on? So I'm just curious to hear like what your takeaways from this sense-making process that you've had so far.
[00:10:32.817] Doug North Cook: Yeah, um, I have been thinking a lot about The parts of the keynote in the presentation and also I think an overall tone in the kind of top layer and like the c-suite in the industry in a lot of places with this obsession with ar And getting to some like mythical magical You're always wearing this pair of glasses and I A lot of the conversations I have been having with people over the last year is show us several really good use cases for that technology that are not just data capture and data manipulation. Because I think, you know, you can point to a handful of, like, easy, low-hanging fruit for how AR can work, right? That, you know, I can look at an IKEA sofa at scale in my living room and see if it works, right? But I think The use cases that were presented during the keynote did not really hit for me, similar to some of the things that they were showcasing about VR. And I think there's this desire to kind of push past VR and get to this AR future. And I think I see part of that as an attempt by a lot of these companies to demonstrate value to their boards, their shareholders, their investors, whoever, to say, we are working towards a product that people will wear all of the time. that will give us a lot more access and a lot more data. And VR is not that product. VR is not an always on product. It's not a product that you're gonna use eight to 10 hours a day. It's just not, right? But it has these very clear, very powerful demonstrated use cases that have been proved out already in entertainment, in enterprise. And we can see those and look at those. And it's interesting seeing this shift kind of move to something We don't know, A, we don't know if we'll ever get there. No one is able to confirm that we will arrive at the ability to do additive displays at a field of view and with enough opacity that will actually achieve what people think is possible. So we don't even know if we'll get there, but we also don't have these demonstrated use cases. And I think I was hoping for a little bit of a deeper dive into the future of VR and how we can engage with that of the still very much unexplored possibilities of that space. And I worry a little bit that the shift is already kind of swinging to these more speculative and what I consider still to be like a very unsure set of technologies.
[00:13:13.396] Kent Bye: Yeah, well, there was clearly the Oculus Quest 2 was the big news that came out of the event in terms of tangible thing that you can go pre-order and it comes out in October 13th. Pre-orders opened yesterday at September 16th. So $299, they're certainly subsidizing it heavily in order to sell it at a reduced cost to try to really accelerate the VR market. So I think that in terms of the VR community, there's the sun setting of the Oculus Rift PC VR altogether. Everything is going to be moving into this mobile future. which at this point, the Oculus Link cable doesn't match the same frame rate or fidelity when it comes to the type of experience you can get from the link. But over time, they're going to be doing software fixes and other things to try to get to that parity point. So we're really having this convergence of the PC VR with mobile VR. which is something that they're really doubling down on. And, you know, I've had a chance to try it out. And the biggest issue for me is that my IPD is at 60 millimeters and the three slots are at 58, 63 and 68. So unless you're right on those millimeters for your IPD, then when I first put it on, I was cross-eyed and I had to like, soften my gaze. And I noticed after the first time using it, I had a little bit of a headache. I don't know if it felt like I'll adapt to that. And it's just like, if you don't have the exact IPD, the Rift S doesn't have any IPD adjustment. So folks have already had to like deal with that to some extent. So yeah, there's this downgrading of certain aspects of the VR, but still kind of accelerating it to have it go to this scale. And I think overall, there's folks that are independent developers that are happy with that, but also a whole other swath of developers that have been shut out, or if they're in direct competition with what Facebook's vision is, then they're getting shut out. So those are some of the things when it came to like the VR takeaways. And for me, I was really interested in seeing what was going to be announced with their AR. I had did an interview with Alex Heath from the information, and he has been reporting that both Apple, as well as Facebook, have been working on these AR glasses. And I think that Apple had an opportunity if they were going to take the first shot across the bow to say, okay, here's what we're doing. But it's very conservative. There was just an update of the Apple Watch and the new versions of the Apple iPad. but nothing about here's what we're doing with our vision for AR. So it was really up to Facebook to start to present their vision. Some of these speculative designs, they had a number of different videos showing what I think are mostly like wayfinding. So making your way around, talking to holograms, having recommendations. If you have vision impairment, then trying to match what's in the environment and amplify that in different ways. And just generally having this device on you all the time they were also saying this project area is going to be their opportunity to do a lot of privacy research, where they're like, technically, they're saying, hey, we don't know how this works. And so we need to use it in the environment. But we also don't know how people are going to react socially. So we're going to do a big giant social experiment by having Facebook employees wear these cameras on their faces and put themselves into social situations where they have to deal with the awkwardness of having a Facebook employee wear these cameras on their faces and not knowing exactly what the implications of their privacy are. So that seemed to me to be like, whoa, okay, we're going to dive straight into throwing this out in the public because I do think that's going to be their biggest hurdle is a lot of the perception of Facebook's brand, as well as what's it mean to be walking around with a camera on your face and having all your eye tracking be tracked. Everything that's in your environment and what you're looking at be this big data dump that they're going to be giving to these data scientists to figure out how to make things private or not private, but I don't know, maybe that's why I'm more attracted to VR because I don't know if I want to be walking around with the camera on the face. And I don't know if any of the things that they were showing me was enough of having to deal with a lot of that social dynamics that are completely new that I would want to necessarily be on the front lines of any of that.
[00:17:11.212] Doug North Cook: Yeah, it's interesting. You know, I've been thinking about this week because there's a huge amount of backlash in the developer community against Facebook and just kind of broadly, right? The Facebook brand, I think, has come under a lot of scrutiny in the last few years. And I've been thinking about the gradations of what all of these different things look like, right? That yes, people are uncomfortable if you just point a camera directly at them. But everybody already walks around with a camera in their pocket that they have out most of the time. Anyway, right. So we already have a device that has cameras and microphones like my iPhone has what four cameras on it. Right. But we use those devices less frequently. Some people use those devices way more than others, right? It's not a question of if we have those things, because we already do. It's a question of how embedded are they? How always on are they? And who has that data? And it's not clear who will come out on top in this. I think everybody kind of hopes, oh, well, Apple will just kind of swoop in and they'll become the main thing. But I don't think that's the answer. I think that Apple will probably put out what is really like the luxury version of this type of device that will be less subsidized. And I think other companies will put out devices that will be heavily subsidized. And in exchange for that, you give up more of your data or more access to some of your information or it's linked up to other services. But it's not immediately clear to me that people will have a negative reaction to these products. If we look back to Google Glass and the social reaction to that, I think that was at a different time. I think people were much more uncomfortable with that level of intrusion than they are now. I think a lot of people now are actually much more comfortable or just ignorant to how their data is used and collected. And I also don't imagine that this device will be marketed to the same group of people that the Google Glass was or will have a similar form factor. I think inevitably it will look much more like a pair of glasses and much less like some sort of cyberpunk device.
[00:19:25.135] Kent Bye: It was something like the snap glasses. You know, you had such an immediate connection between already a platform where people were sharing short videos of what was happening in their lives, and you could augment them in different ways. And it was a communications platform that they had really cultivated and developed. And my sense of just, you know, talking to other people is that in some ways the Z generation doesn't seem to be all that interested in what's happening with Facebook. And that there's TikTok, there's Snapchat, there's all these other social platforms that they're using and that they're on. And that in some ways it's not cool to be on Facebook because that's the social network where your parents are on. You know, what kid wants to be on the same social network of their parents? So there's a bit of a rebellion aspect, but there's also like, I think Facebook seeing that they really want to try to capture that youth audience, because if they don't, then it's just a matter of time before, if they're not growing from that age demographic, that eventually they're just not going to be relevant as a social media platform, because even people that are older and more along our ages are more and more starting to see some of the unintended consequences of social media and opting out and finding that, you know, they have more deeper interactions with people when they get off of Facebook. Not even to speak about all the other issues with, you know, the social dilemma. I think it's worth pointing out that the documentary, The Social Dilemma, just came out. It was premiered at Sundance, which is where I saw it, but within the last couple of weeks. So there's been an additional element of all the different transgressions of what social media is doing to our lives and this kind of manipulative aspect that happens when being the intermediary of our communications platforms. So there's so many issues there that I think we're just starting to reckon with it. But there seems to be a play from my perspective, at least, that both the AR as well as the VR could be that whatever the enthusiast VR community is now, that they really have their eyes set on a whole other community that they're going to be going after, which is, I presume, to be a lot younger generation.
[00:21:23.517] Doug North Cook: Yeah, well, and the pricing of the Quest 2 is clearly geared towards an aggressive holiday purchase, especially going up against the new Xbox and the PlayStation 5, that they are aggressively coming in under those price points to make it the go-to or at least possible go-to Christmas gift, right? And that that is clearly, yeah, pushed towards a younger and active gamer demographic still, right? That's kind of where they're pushing And I don't think that they're wrong to do that. I think making a more accessible device, and this is a conversation I was having over the weekend as part of the developer bootcamp before Connect, is that a VR device is very different from a mobile phone. A mobile phone is a personal device. A VR headset inevitably is a household device in most places, and that whoever buys the headset might be the gamer in the house. because that's who it got marketed to. But inevitably, the whole family is going to touch that headset at some point. And the more accessible, the more easy it is to use, not having it be tethered, not having to troubleshoot it on PC makes it so that multiple people in that household can interact with that device. And that's what I see time and time again, is that different people in a household that have a Quest end up interacting with it and have a favorite app that they jump in and out of, and that could be mom and brother and even a grandparent who likes to watch 360 videos and travel and see things, right? So I do think that movement of trying to make VR specifically much more accessible to a wider audience is a really positive one. I think there's a lot of energy in the VR enthusiast community that really likes to gatekeep people and say that, you know, Everybody should have an index and a rtx 2080 or 3080 in their pc and that's really what vr is and it's like That's great. Like those are all great things but I think that perspective doesn't serve the majority of users who just either want to have a magical experience or Need to be able to do work right that if we're talking about vr for training and things like that that it's It's much more about ergonomics and form factor and cost than it is about fidelity necessarily.
[00:23:43.364] Kent Bye: Yeah. For me, as I'm tracking different privacy issues, it seemed like they were trying to get ahead of the privacy issue. Andrew Bosworth, also known as Bos, he said that they've been consulting with privacy experts and the privacy experts tell them that it's not what you say, it's what you do. okay, what are you doing? And what are the different things that I see? I think there was actually a lot of new stuff that was released at Facebook Connect. They released four principles. And one of the principles was considering everybody. And what they mean by that is not only like diversity and inclusion and people that are marginalized communities and how do they relate to the technology, but it's really this binary switch between the users and the non-users. So people that are walking around with AR headsets, they're the users in their community. And then there's the people who are the non-users. So what are the privacy risks and other implications that they have on people that aren't in there as these Facebook users that are walking around with these in real life surveillance machine glasses that are not only tracking their eye movements, but the world around them? What are the implications of other people's privacy as you start to have these glasses as you're walking around, especially as they're consolidating everything together? So there is some things where they're like, okay, what's the research there? Let's get some requests for proposals and these principles that they're laying forth. But from my perspective, you know, I was asking to talk to somebody about some of the privacy architectures that they talked about. Let's try to interrogate that a little bit. And the biggest thing is that with the Oculus Quest 2, that's going to be a whole new privacy policy that is going to be under not only the Facebook data policy, But there's an entirely new Oculus supplemental data policy that they're not going to even show us until October 11th. So we have no idea what the privacy policy of the Oculus Quest 2 is going to be until next month. So if they were really going to try to get on the right side of privacy, I would think that they would want to at least release that and show us what is in their privacy policy. But that's one of the actions and behaviors that I haven't seen yet.
[00:25:42.325] Doug North Cook: Yeah, and I think it's increasingly important for outside people like you and myself to be actively engaged in discovery around what exists in these policies, right? How is user data being treated and then finding ways to make sure that that's making it into the broader public discourse, right? Because nobody reads the terms of service except me and other people who are more engaged, interested and concerned. But I think, you know, especially because they are leading the charge with these technologies currently, right, that nobody else has a serious standalone VR product. Nobody actually has a real pair of AR glasses yet. And so they really get to dictate a lot of what this looks like and set the tone and set what these policies mean for users. I mean, necessarily another company, probably Apple, will come out with a much more aggressive privacy focus. But it's also not clear that they will ever do that and that they will ever release a product because that's not, there are no givens with any of this, right? Because everybody is still in a very speculative place with all of this. So, but yeah, it'll be interesting to see kind of where they go with it. I mean, I think everybody is very concerned about Facebook's privacy settings and a lot of people are like deactivating their Facebook accounts and subsequently their Oculus accounts with them. And there's definitely a lot of reaction against that. But I would say for the most part, that's in the enthusiast and developer community. I don't think that's necessarily reflected in the general user population.
[00:27:19.450] Kent Bye: Yeah. One of the other things that Facebook had announced back on July 15th, 2020, was this making data and privacy easier to understand through people-centered design. This is a white paper that was talking about notice and consent. You know, how do you navigate giving consent about what you're doing with the data? One of their number one principles that they announced yesterday was don't surprise people. So they want to be able to disclose what data they are collecting and why they're collecting it. And so this is a communication issue as you go into VR, and you're not even potentially even reading any sort of terms of service, you're just thrown into these experiences. And how do you make sure that people know all the data that are being collected? So they had this white paper that said that they're going to be hosting conversations with regulators, civil society, academics, and other companies around the world to dive into some of the ethical questions around privacy and XR, including XR as well as other products and services that Facebook are doing, that they're going to be, within the next couple of months, starting to do that. I reached out to them and said, hey, I'd really love to be a part of this. Of course, I don't think that I'm going to be invited. there's not going to be, as far as I can tell, any journalists that are going to be taking part of any of these conversations. So it seems to me that they are going to start to have these different conversations, but they're all behind closed doors. And maybe under Chatham House rules, maybe they're just signed NDAs. I don't know what it's going to be. They haven't given me a lot of information about it. But that's another area where They're doing these efforts and initiatives where it's like, Hey, I'd love to really be a part of this conversation, but it doesn't seem like I'm necessarily invited to participate in any of those discussions. And so from your perspective as an academic, I don't know if you're having more direct conversations about some of these things or how you play into some of these larger discussions around XR and privacy.
[00:29:02.699] Doug North Cook: Yeah, I would say like, you know, anything internal that I do with any company around any of those issues is always under very strict NDA. And that's just the way that it is. But outside of that, I mean, I take my role as an academic, which I feel like puts me in a pretty neutral position to be able to speak to these issues that I find a lot of these issues very concerning and disconcerting. And I want to be able to be in those rooms as much as I can. So this is an invitation for anyone who wants me in one of those rooms that you can find me easily. But there are a lot of people at these companies that are genuinely trying to reach out to experts and engage with these issues in a meaningful way. Does that mean that those discussions will always make it into a product? No. That's just the reality. But I do think that they're trying to move the needle on some of this, but it's a really hard lift. It's both a very difficult task to help users understand these issues. In a way that they actually care about, which again, I think most users don't actually care. Most people, even if the terms of service are easier to read, they're still not going to read them. And that still at the core of a lot of the largest tech companies, the core underlying profit center is advertising. And the way you do good advertising is by having good data. And it goes back to this question that I've been asking a lot of people, which is how much do you actually value your privacy? And where are you willing to make a cut in the trade-off of giving up privacy in order to have better products, right? Are you willing to have a smart speaker in your house that learns the way that you talk so that you can have more natural conversations and get your Spotify playlists to queue up easier, right? Do you want an easier interaction with technology, which necessarily means providing some sort of training data to any system that you're interacting with? Or do you just want to use technology that's calibrated for a theoretical version of you and not actually for you. And I think that, you know, it gets to this deeper question of how deeply connected do we want to be with computers and necessarily that is being facilitated by a company, right? Because individuals are not going to build these machines on their own, at least not generally speaking. So we will always have some sort of mediated experience that's happening through a corporate layer and that has to be paid for somehow. And I think that's going to be an interesting shift. I think, especially in the last couple of years, as we've seen the digital ad economy start to continue its decline in profitability and how companies are able to make that work and how are companies going to build more meaningful products that don't have to just rely on ads, I think is a part of the bigger question of what does that look like when we're talking about these spatial technologies and That's something I'm really curious to see. But I would love to see more of these conversations around ethical issues and privacy happening in public, but not even just in public, but in collaboration between some of these companies, and not just internally and not just behind closed doors. But that's a big effort. I mean, these are really intimate and difficult and legal conversations. And I think most companies are unwilling or incapable of having those in public based on how their legal structure is set up. So.
[00:32:37.198] Kent Bye: Hmm. Well, so as I understand it, you've signed some NDAs with some of these major companies and that my expectation was that these conversations are happening, but they're under the auspices of either the Chatham house rules or these NDAs. And so what you're saying is that, yes, that's pretty much how it happens.
[00:32:54.629] Doug North Cook: Yeah. I mean, generally, you know, it's pretty rare that if you're an outside expert going to consult with a company that they'll ever let you talk about anything that you saw or heard there. Um, that's just, that's just the way the wind blows. I mean, what I try to do in a lot of my work is invite people out of those companies to come and talk about some of these issues. So I'll be running a seminar in the spring at the university that we'll be recording where Part of my hope is to bring people from a couple of the major companies to come and talk specifically to some of these privacy and ethical concerns that are coming up. And I think that's where in an academic setting, there's a little bit more of an invitation to disclose more than I think you might Feel. I mean, they'll never stray outside of the legal bounds of what they're capable of doing. But I think there's possibility for a little bit more of an open discussion. And an opportunity for challenge right that isn't about the press or journalism, but it's engaging with students, faculty and as part of an academic discussion, which I think at least provides a little bit more of a neutral ground. So we'll see how that goes, but
[00:34:02.345] Kent Bye: Well, one of the things that I've been really trying to reckon with is as we move into this future where the corporate structure of some of these big companies is such that like Mark Zuckerberg, for example, as Vanity Fair has reported has like 58% voting share for his entire company. So he's essentially like this. totalitarian dictatorship over the company where whatever he says or wants to do pretty much goes. And there's not much recourse that the shareholders have to do about that. And of course, they've been trying to fight that and change it and bring up for vote. But every time they bring up the vote, Mark just puts it down to not give over any of his power. And there's other companies that have done that with Google and Amazon and Snap. It's not like Facebook's the only one. But we have this situation where these tech companies are growing in size and power and influence where Facebook with over 3 billion users now, they're bigger than any individual government and their influence goes around the world. And you have what is essentially a corporate structure where there's a few people at the top that are really controlling the destiny of not only the company, but billions of people, like literally billions of people are impacted by their decisions, but yet there's no transparency or accountability or any type of democratic process. And as we even start to have these basic conversations about the future of justice and the future of these virtual worlds, it's almost like we're moving into this world that's going to be completely controlled by these private corporations, and they're not having any public conversation about this.
[00:35:28.799] Doug North Cook: Yeah, I mean, you know, if you watched any, I mean, I make an effort to try to watch as many public hearings that happen in the Senate with CEOs from tech companies, because that's I feel like it's an important thing to engage with as someone who works in the industry. But if you watch the level of competency of a lot of senators and legislators and politicians and their ability to engage with these topics in a meaningful way and therefore create legislation and challenge companies in a way that's actually thoughtful and understands the implications of the technology, you just don't see that. That's not happening in these spaces. You know, so it's not just a lack of education among the users, it's a lack of education and understanding among legislators of what both the potential downfall and difficulties of these technologies, but also the opportunity, right? I mean, you know, you spoke to something that I think a lot of people have been seeing in the last several years, which is that there are corporations that wield much more significant influence than a lot of countries do, right? And, who decides how those companies get regulated. And there really hasn't been too much of a serious effort to do deep regulation of tech companies in the U.S. That certainly has been the case in other places. You know, if you look at the way that the Chinese government interacts with companies, it's very different, right? But I do think that there will be a shift at some point where the government in the U.S. will decide to be more engaged. But I don't think that that will necessarily be a positive move because, again, I don't think there's enough of a core understanding of how these companies work and what these technologies actually do and how they operate. And I think that's part of why a lot of these discussions don't happen in public, right, is that they don't want an intimate discussion about privacy policy to get taken out of context and end up in the news or end up in as part of a hearing. And there is, to me, there is a lot of value in having those spaces where People from companies are able to interface with outside experts and they're able to fail in that space together. They're able to ask questions that they would not feel comfortable asking in a public forum. They're able to dig into issues that are incredibly difficult and uncomfortable that I think are hard to do in public, you know. especially when we're talking about these spatial technologies and we're talking about underage users and how do you manage underage users on a spatialized platform that is open to everyone, those are not simple or easy things to regulate or to design systems for. But I do think it is important to create public spaces to have those conversations because the more that those conversations are happening in public, happening in academia, at conferences, on podcasts, by journalists, those conversations make their way back into the room, right? They become things that get referenced and brought up and inform that conversation, so.
[00:38:37.630] Kent Bye: Yeah. And what I noticed that whenever there's Facebook Connect each year, the developer conference, they make a bunch of announcements and they're talking to the press. It's like the one time where you really get a lot of new information. So I spent a lot of time yesterday going through a lot of the different interviews. Janko Rodgers of Protocol did an interview with Bosworth, the CTO, and then the privacy architect, Jenny Hall, talking about the four principles that they were announcing yesterday. But again, he wasn't really grilling them in detail in any of the specifics of some of the privacy issues that have been raised over a long time. Mark Zuckerberg actually did an interview with Casey Newton on The Verge where Casey Newton asked about, okay, we had Google Glass that came out. There was a lot of issues with putting cameras on people's faces. How are you thinking about wading into the public debate? And Mark Zuckerberg said that, I think the first thing we should do is just talk about the issues up front. One of the issues that I've learned over the last several years is you don't just want to wait until you have issues to be discussing how you want to address them, and not just internally having a social conversation publicly about how society thinks these things should be addressed. And, you know, he's talking about a timeframe of like 2024 where some of these things are coming out and so If some of these issues come up, it's going to be a long time before you can fully address some of those things. And at the end, he says, I just think starting to have those conversations now so that by the time the technology is ready, we can at least have some early consensus about how to approach this. I think that's just incredibly important. Now, he's saying that he wants to have a public conversation, but I'm just not seeing it anywhere other than talking to journalists who are not fully ramped up with all these variety of different issues. And you're doing them Project ARIA, which is like a research project where they're going to be doing some research. They're doing these regulatory sandbox private conversations where they're having a chance to be able to look at the notice of consent and say, okay, we don't want to necessarily replicate all the bad user experience of GAPR of every time you go into a website, you have to click the notice of consent. If you just applied that mindset to experiential design, then you can imagine every time you want to get into a VR experience, you have to click through like 20 dialogue boxes to give all the consent. And so there are certain things that need to be figured out where you're giving informed consent, but at the same time, you don't want to like completely degrade the experience that just makes it insufferable for everybody. So I could see all these things that are happening. And I guess my frustration is I'm seeing more rhetoric about wanting to have that public conversation, but I'm just not seeing the right context for where that's going to actually happen, especially as I'm wanting to have that, I've been asking to have that conversation, but just haven't got any commitment to make anybody available to actually have that conversation.
[00:41:06.808] Doug North Cook: Well, I hope you get somebody to do that with you. You know, I think, yeah, I'll be curious to see how they actually engage with that. I mean, you know, just knowing how some of that design process works, right, they're going to do a lot of interviews, they're going to test with a lot of users, they're going to focus group it, right, they're going to get input from a wide range of people. But I would be curious to see, you know, what does that engagement look like? And how is that feedback coming back? But again, that's where all of this is really different. You know, if this was the government doing this, we could do a freedom of information request and I could look at the process documentation as long as it's not classified. But that's where it becomes interesting to know how to interface with corporations that have so much power and influence but don't have the same openness requirements that government and other organizations do. And I think increasingly that's becoming a question worth asking is what does that look like, right? Which is difficult because the more openness and the more regulation you have, the slower that you have to move necessarily, you know? And I think some people would really like to see the process slowed down and some people would really like to see the process sped up and regulation even pulled back even further, you know? So even like, let's get input from everybody. It's like, okay, well, there are a lot of voices like, in America that could sway that conversation in one way or another depending on who you decide to talk to, right?
[00:42:37.277] Kent Bye: Although they'll say they've consulted privacy experts, but you know, it's unsure as to how much of the suggestions and the recommendations that they gave were even actually listened to and integrated. I wanted to point out a couple of buzzwords that I heard that seemed new to me, at least. This is something to pay attention to, and that is egocentric data capture and contextually aware AI personal assistants. So in order to have an AI assistant that is aware of your surroundings so that it knows what is happening in the context of your life, you're going to have to do what they're calling egocentric data capture, which is essentially capturing everything that you do or say in what environments that you're in from a first person perspective, and have all sorts of AI be able to sift through that and be aware for whatever that AI needs to know based upon the actions that you need to have that take. So they seem to just put this out like this is just a thing that everybody is going to want to have, which is this contextually aware AI that is able to be aware of all of our surroundings at all time, always on, and that we're going to be able to do all these amazing things with it. But at what cost of doing all this sort of egocentric data capture? I understand there's going to be some amazing things down the road, 25 to 50 years from now, that is going to require that egocentric data capture. I guess I'm just concerned around the guardrails when it comes to how that data are used and the privacy protections, the data sovereignty, what kind of ways in which that type of egocentric data capture is going to perhaps subtly undermine my own agency, especially when you start to integrate biometrics of my eye tracking data of what I'm looking at in the real world correlated with what is in the real world to bridge the gap between what I'm seeing in the world and to make a one-to-one scale copy of everything in the world. And for the sake of saving battery life, to be able to have that virtual copy of everything in the world so that you can just beam down into these live maps. So I don't know, there's this vision of the future that they're painting, but I guess it just makes me more hesitant to not know the unintended consequences and the risks that are associated with all that. And to not see like a use case that was like, Oh yeah, I definitely want to do egocentric data capture, have live maps and a contextually aware AI personal assistant with me 24 seven.
[00:44:52.897] Doug North Cook: Yeah. I mean, it's when it gets to that level, I think it really becomes a conversation about who do we want to be? Like what do we want humans to be? And those conversations become like even more low level when you start to look at what Elon Musk is doing at Neuralink, trying to build the future of brain machine interfaces. And when we're going to this level of, I need to capture every object, right? I need to give the headset, you know, a scan of this notebook. that I'm holding in my hand right now so that it can know that if I want to take that notebook with me and it's still on my desk, that it knows what that notebook looks like so that it can ping me to tell me that I forgot my notebook, right? They showed off an example of, oh, you forgot your keys. that was one of the examples of AR that they showed. And it's like, okay, in order for that to happen, right, it has to have positional and contextual awareness of what your keys are, that you're going somewhere, that you would need your keys. And it has enough responsiveness to be able to ping you in time in a way that's valuable and useful. But it gets to the question of, are those things that we want, right? Do we want that level of, presence from our technology in every moment of our life. There's a design framework. I can't remember the name of the designer. There's a couple of different ones in the last year, one called Calm Technology. And then I think I just adapted that in my own head and started calling it Cozy Technology, which is what are technologies that I have no idea that I'm even interacting with them, that they are so in the background and so unobtrusive And there's a version of augmented reality that is the opposite of that, that is just anxious technology, a version of technology that we become overly reliant on in a way that just takes our anxiety and pulls it through the roof, that we always need to know what we're doing, where we are, how we're operating, and what we're forgetting.
[00:46:54.937] Kent Bye: And I don't know if that was Amber case comp technology, but yeah, I know she's done a little bit of that, but yeah, I think you're right in the sense of does some of these use cases they're showing, does it, would it require like an always on glasses? I know that Google glass when it first came out, they had a lot of these kind of speculative sci-fi futures that they were trying to sell the story of what this was going to be able to enable you to do. And when I looked through and actually pulled out some of the different screenshots of different stuff, there's. Like you're at a market where there's a foreign language and you want to have a translation happen in real time. So you're like traveling or you're in an area where there's a foreign language and you want to have that. Well, when you're in that environment, you could also use your phone because your phone also does that type of thing. also as you're walking around the city and you want to maybe have like recommendations or directions that are there so you're wayfinding but also seeing what the directions are and where to go as you're walking around and then when you're in a music store they have like records that were on the wall and they're like here's a rare find that you may want here's a recommended one that you may want and you know, how much of that is ads and how much is that genuinely just, you know, tracking everything that you do to be able to make music recommendations as you walk into an in real life music store and, you know, where are the ads and are we creating like the Keiichi Matsuda's hyper-reality where you just are walking around and you're just completely bombarded with visual stimulus all the time. So I don't know, like I agree that these examples for me at least are not compelling enough to be like, oh yeah, I can't wait to put these glasses on my face.
[00:48:25.690] Doug North Cook: The idea of using egocentric data capture and an always-on device to recommend me records is completely hysterical to me because the whole point, at least for me as someone who collects records, is to not engage with anything like that at all. It's to have something that is like a purely connected pursuit, right? That for me that's like a form of my personal cozy technology is having a record player, having something that is intentional and slow and it's not connected to my data. I don't get recommended a record from my record collection. I go and I make that decision myself, right? And so the idea of using that device to kind of erase some of those things is interesting to me. You know, yeah, there was so much location awareness. And then, yeah, you talked about this, about having, you know, transparent avatars and other people being able to kind of show up in the space with you. And one of the jokes we were saying yesterday was this is a device for people who are always lost and want to see ghosts.
[00:49:34.118] Kent Bye: Well, what are some of the other big takeaways as you walk away from Facebook Connect One? What's the big next steps for you as you do your work there at Chatham University?
[00:49:45.060] Doug North Cook: That's a really good question. I think, I mean, I'm excited to see the progression in standalone hardware. I'm also excited that they're building it on this new Qualcomm spec that has me hopeful that other companies will be able to build on a similar spec and take what Qualcomm demoed with, you know, they had a demo headset on this hardware like a year and a half ago. So I think I'm hopeful that I think we're at a place now where Facebook has proved out that this is a device that consumers want and will use and that there's opportunity for other people to dip in and compete in this space. So that's something that I'm that I'm excited about. I mean, for us at the university, I mean, for me, you know, there's a big push to find new ways of making sure that the work that me and my colleagues at other universities are doing, that we're doing that in a way that is very public, that is very shared and shareable, so that we can help to act as a neutral ground, as a place that can facilitate some of these difficult conversations that I think a lot of us hope will happen over the next several years. So that's something that we're actively working on now is what are the best mechanisms for facilitating those conversations, for bringing multiple companies together to be able to have an informed conversation around issues of ethics and privacy and user engagement. So that's something that I'm still personally very much engaged and feeling newly encouraged and energized to work on is making sure that we're Engaging as academics in the right way because it's really easy in academia to kind of just hide away in your tower. Do your personal research, make sure that you get tenure and and just let it be. But I think I view my role very differently in that as an academic, I have a unique opportunity to speak into the industry as someone who doesn't necessarily have a vested interest in the success of any company, my own included, because I don't work at a company. So that's what I'm trying to do and work on, and I know that there are a lot of other people that feel very similarly, and I think this is an incredibly exciting moment for VR. I have seen applications in the last couple of years that have just completely blown me away. I don't know if you got to play Tender Claws' The Under Presents Tempest live show, but that to me, as someone who loves immersive theater and has missed being able to go see immersive theater during the COVID times, That was a very genuinely surprising experience to me that I think gave me a glimpse into the magical parts of VR that I think, for me, I can get very bogged down in a lot of these complex ethical issues and privacy concerns and hardware problems and development issues. But having these experiences pop up that invite us back into what, for me, is the magic of VR is being able to create experiences that are not possible in the real world. And I think that Tender Claws does that. They're very good at capturing that. But I think, you know, doing these live actor facilitated showings of the Tempest in VR, where you're having these really beautiful interactions with strangers with no voice chat was really powerful. And I think I'm really excited to see what developers are going to do in the next year, especially as the user base expands. And hopefully we have some more companies jumping into the standalone fight.
[00:53:14.017] Kent Bye: Yeah, one of the things that's happened in the film festival circuit in the VR industry is that you have festivals like Tribeca collaborating with Cannes to be able to do a virtual display of a lot of their experiences. And then the Venice Film Festival just had like 11 days where they had most of their experiences available for free on Viveport with like three different performances. So they had like a live dance performance with motion capture. They had a theater performance in VR chat with Keira Benzing and Double Eye Pro that actually won like the best VR user experience award So one of the three main awards. And then Metamovie was in Nios VR. So again, you actually became like one person out of the people that participated got to be the main protagonist. So I was like, I had to like role play with these other characters. And it was like this live action role play mashed up with this adventure that I'm going on in VR. And it was really quite profound. And I'm really excited to see where that goes. And yeah, checking out both The Ender Presents as a piece that has immersive theater actors, as well as The Tempest, which has, you know, more of an explicit show. So yeah, I'm excited to see experientially where that all goes. And, you know, the big takeaway for me, at least, is that each individual has to negotiate their relationship with Facebook. To what degree are they going to participate in the hardware? Or, you know, for me, my takeaway was like, I can't unconditionally recommend the Quest because there are conditions. So as long as you're happy and satisfied with those conditions of what those trade-offs are to your privacy and to the overall ecosystem, where on the one hand, it is growing the ecosystem, but on the other hand, there's things that are, I guess, just more concerning to see more and more behaviors from Facebook where they want to take these unilateral actions without much dialogue or feedback from the community. And I think that's in an odd way, it actually comes out in the experiences they create, like the venues. Like if they would have had somebody actually go into some of these experiences and try to watch a lecture and then have all these sound effects, it's like, that to me should have been like brought up to them, like in a very early beta test that shouldn't be there at their main event of the year. And maybe that just the release cycle is that they haven't had to do much testing before that, but I think the fact that they have been so slow to actually launch their big products and then every social VR experiment they've launched has been shuttered up to this point with the venues as well as the horizon being their next iterations that haven't even fully launched. Even John Carmack said, it's kind of embarrassing that Facebook as the social media company hasn't actually created a viable social VR platform. And I think that there's maybe some deeper cultural issues there in terms of how they do experiential design and how they don't have this more iterative engage with the community and this deep listening process that happens, almost like they want to go off and have it fully baked and deliver something that's ready to be launched. But even when they try that, then it's still not ready. And so, you know, how do they get that feedback process when you go into venues or Facebook Horizons, there's no clear way to even provide any beta testing feedback anywhere for any channels for how to even receive some of that feedback. So there's some deep issues there in terms of the listening parts of Facebook that I think not only come through in their behavior of how I have my own personal experiences of my interactions, but just how they treat the wider VR community in general, but also how they do their experiential design.
[00:56:27.182] Doug North Cook: Yeah, I mean, I think there are conditions to every product, right? And part of this conversation is, I've seen a lot of conversations in the last year, I mean, really for a long time now, about dozens and dozens of companies and whether or not you should be engaged or invested or involved with what they're doing, right? And the difficulty of being a person that is alive right now who is ethically aware of the consequences of your life and your lifestyle is that everything that you consume for the most part, is ethically compromised. Whether it's the meat that you buy, or the jeans that you wear, or the neighborhood that you live in, or just being an American. That there is no ethically neutral or net full positive position. That you are, by being alive, you are compromised. And so it's always interesting to me where people choose to take their stands. And I think it is really important for people to be engaged in critical discourse with Facebook. But I also think it is worth in that bigger conversation acknowledging that they are not the only company and not the only thing that exists in the world that we engage with. And I think currently it is very like culturally cool to hate Facebook. and not as culturally cool to feel similarly about other companies that have very similar practices and engage in a lot of similar behaviors. And it's interesting to watch. It doesn't change any of the things about what Facebook is doing, but it is interesting to me to see so much of the focus shift and land on them. So I don't think that's unfair, but I do think it's worth just noticing that that's happening.
[00:58:18.121] Kent Bye: Yeah, there's some other folks that brought up that exact same point, especially around Google, because Google has a lot of similar surveillance capitalism behaviors. I think there's two things that I noticed, at least with Google, is that for one, they've kind of gone AWOL from the VR industry where their daydream has been pretty much shuttered at this point. And whatever they're doing in the immersive space is around ARCore and their mobile platforms. And, you know, they brought North and maybe they'll have some sort of their own glasses or smart glasses and relaunch of enterprise Google Glass. But aside from some specific experiences like Tilt Brush or Alchemy Labs or the Google Earth or YouTube VR, they seem to be otherwise sort of not really engaged in the wider discussions, especially when it comes to hardware and people having a direct relationship with a major tech corp. But there's also, I think, not as many articles from the news media that have connected the dots to some of the transgressions that Google may have been doing, and then seeing that there's a clear harm that had been done. Even if that's happening, it's more harder to cover or harder to tell the story of. With Facebook, there seems to be a lot clearer paper trail in terms of the different scandals that they're involved with, because they're at the heart of different aspects of our communications platforms. So there's a lot more issues, not only here in the United States, but really around the world that they're engaged with, that they just have like a magnifying glass based upon all their movements, because they've become this huge communications platform. And they're saying that they're not a publisher, they're just enabling people to communicate, but yet they've created this network of 3 billion people that has created a target for many state actors to be able to use that to be able to do geopolitical warfare and information warfare and experiential warfare. There's all these things that are happening on that platform in particular. So I can understand why those other companies that may be making similar transgressions, it's just maybe we don't know what those full stories of the full extent of that is. And people's experiences of Google is like, hey, I need to know a piece of information and Google just told it to me, but may not know that that algorithmically derived piece of information that you just got was helping to accelerate a deeper polarization and filter bubbles that are also playing into the larger division that we have in our culture.
[01:00:23.653] Doug North Cook: Absolutely. And I think, yeah, I'm not interested in like giving Facebook a pass, but I am interested in making sure that we're engaging in the broader conversation about what this looks like across companies and across countries. Especially as inevitably other companies will become more invested in this space. That's just something that's going to happen. It might not be a lot of companies, you know, we may end up in a Apple and Android situation where, you know, we have two or maybe three companies that are heavily engaged, but yeah.
[01:00:57.289] Kent Bye: Yeah, just our theories of justice, how we deal with punishing people, how we deal with free speech, how by default everything's a public room within Facebook. I mean, there are options to have private solo rooms, but what's it mean to have like an invisible moderator that's watching over you at any moment in all these different virtual spaces? There's a lot of things that, you know, we're creating a world that we are having in real life existence now, and we're moving into this virtual future that has all these, I guess, more cyberpunk dystopic potentials of what is essentially the Ready Player One IOI being embodied within the actions of having one company wanting to have complete control and dominance over the metaverse, and turning that virtual world into the same model of Apple's iOS store, essentially, or Google's Play Store, where they have complete control over everything that happens on their platform. I think there's certainly people like John Carmack who are certainly in favor of making it more of an open neutral platform, but there's these other business decisions that seem to be moving this whole platform into a new trajectory. Maybe we need to have that to be able to actually get a critical mass to be able to have the open alternative, but I don't know, it just, for me, the most concerning thing is that I don't see like the structures of a democratic future that has like openness and transparency and accountability and open dialogue and open discussions about what we already know are going to be some huge issues, but to not engage in it is creating a virtual world that is going to be like a totalitarian virtual society is what it feels like, at least to me. And that's my, as I sit here at the end of all of this, that's sort of my big takeaway is that there's just a lot of red flags that are going off in terms of what would make me feel a lot more comfortable as we move forward.
[01:02:39.642] Doug North Cook: Yeah, and I think you're hitting on something really important here, which is that we don't have a good model for how democratic decision making can exist in relationship between users and corporations, especially around digital products, right? That social networks are not democratic platforms. right? Like Twitter is not, Facebook is not, TikTok is not, right? The users are just exactly that, they are users, they are not participants, right? And you can make arguments about that about government too, but you know, at least with government there are significant mechanisms for being heard as a constituent, whether that's by voting or talking to your city council people or showing up at an open meeting and those things are maybe occasionally questionable in their efficacy as well, but But I do think it's worth trying to imagine that. But also, I really hope that someone can showcase a meaningful model at scale for how that can work. Because I think most companies don't think that it is in their vested interest to even think about trying to do that. But it would be interesting to see if somebody could. But yeah, we just haven't seen it. I would love to see it. I would love to also just see if anybody has even good theoretical models for how that could work. I would love to see those. So send them to me.
[01:03:59.432] Kent Bye: Well, it seems like what's happened in Second Life, there's at least some levels of governance. It may not be like a full democracy, but... And what's happening in the cryptocurrency realms with like Decentraland or CryptoVoxels or these other cryptocurrency driven platform co-ops where they're really driven by the owners of the land. Somnium Space is another place that's doing some experiments there in terms of people that are becoming landowners. So there are some experiments that are happening. It's just at a more lower down on the stack at the technology layer. We do have like open standards, like OpenXR, which is great. It's amazing that we have the ability for competitors to start to have a common set of APIs that they can start to interface open alternatives in terms of technology and even within the software to be able to have experiences with on these platforms. We haven't seen that at the mega tech corp level to see what those different types of governance processes will look like. Is there going to be a truth and reconciliation process? Because if you are 13 years old and you get banned from Facebook, what happens when you're 26? Do you get banned from this virtual world forever for the rest of your life because of something you did as a teenager? So these are the type of questions of like, you know, is there a jail that you go to? Is there a way to make amends and to own the harm done? What is the truth and reconciliation models mixed with some sort of more punitive justice that is punishing people in that way? I mean, they've implemented what is essentially putting people into exile and there's no recourse, there's no ability to appeal. We don't even know if like there's evidence that is presented against you and the Facebook Horizons that has these 30 second rolling cameras, what if that can be gained? Do you have eyewitnesses? Are you going to be relying upon the moderators who are overlooking it? Are they going to have enough cultural context to even understand what's happening in the world? Because as we've seen in Facebook with going around the world, They had cleaners who were responsible for doing content moderation, but not being culturally aware to all the nuances of these cultures that they had to moderate content for around the world. And so what's that mean when you have these virtual reality platforms where you're trying to be attuned with in real time communications and the nuances of how language works in different languages. So, I mean, there's just so many issues there that we can see where it's going in the future, but they have to create something where they have the harm done and then you can talk about it. It's not like. architecting this whole robust future, because they have to iterate it. And they're like, five to 20 years away from any of the things that I just said there.
[01:06:25.305] Doug North Cook: Yeah. And I think even that idea, right, of being banned from a corporate platform, affecting your ability to operate and exist in the world, is a reality, honestly, that we already live in, right? That if you don't have access to a Facebook account, currently, then you have less access to the full listings of apartments that are for rent in your area, right? Because Facebook has expanded to be such a utility for how we shop, buy things, assess real estate, like all of these different pieces, right? So it starts to become, when you have companies that have so much ubiquitous utility, it becomes a bigger question of what does that mean for how individuals get to exist in society based on prior behavior and who's writing those parameters and also yeah how can they be gamed how can people be targeted which all of this is becoming so much more difficult to do and so much more complex especially as we start to pull people's bodies into digital space and vice versa and bring digital space to our physical environments that The issues around privacy, around harassment, around, you know, what do you do with banned users? All of these things become increasingly difficult to deal with, difficult to manage. They become much more labor intensive to moderate. Yeah, it's going to be interesting to see how these get implemented and rolled out in a serious way. I had joked in a consultation about just having VR hell and that you like employ someone as the devil. And if you get banned, you go to VR hell and you have to like full like seven layers. You have to work your way out of hell by doing tasks. And it's like an arduous process. You can work your way back, but you basically have to like go through therapy and what a job, what a job it would be to be the Lord of VR hell. So. That might be a real job. It might already be a real job.
[01:08:23.262] Kent Bye: Great. And, uh, and finally, what do you think the ultimate potential of virtual reality and augmented reality might be and what it might be able to enable?
[01:08:33.900] Doug North Cook: I mean, for me, I think I am very much still drawn to some of the things that I was talking about earlier, which is that to me, virtual reality has this really special power, which allows us to transcend what it currently means to be human. to engage in impossible experiences, to share impossible experiences with other people, to let someone else visit a dream that I had. We have these potentials with VR specifically to do things that to me make being alive and being human more interesting. Augmented reality, I am very curious what that's going to look like. I don't see the path for me personally to wanting to wear an always-on pair of glasses. I'm open to being convinced. I'm a firm believer that I'm wrong about pretty much everything. But I am interested in other forms of AR, you know, a more advanced set of headphones or earbuds that can augment spatialized audio in a more meaningful way is something that would be very interesting to me, and I'm sure will become increasingly interesting to me the older that I get. And as my hearing changes, it would be incredible to be able to augment that specifically. But when we talk about augmenting more and more of our senses, I think, again, it becomes a core question of who do we want to become as humans and how much do we want to change our core functioning and how we interface with our vision and our hearing and our sense of taste and smell but Yeah, I'm still double down on magic and experiencing magic with other people. That to me is still where VR shines and is the most compelling. I've also been working with the National Safety Council as an advisor for VR and AR for safety and safety training. And that work has been really, really interesting and really positive. And I'm excited to see these technologies being used in a way with a group of users that would very rarely self-select to engage with VR, but then are finding an incredible amount of value that it is genuinely saving lives, making people safer, keeping people from injury, helping people acquire skills faster. And those parts of the technology, which are much more functional than magic are also very exciting to me right now.
[01:10:57.240] Kent Bye: Hmm. Great. Is there anything else that's left unsaid that you'd like to say to the broader immersive community?
[01:11:04.853] Doug North Cook: I think we've said a lot. I mean, my constant encouragement to everybody that's doing this work right now, which is that we should all be actively seeking ways to invite as many different kinds of people to engage with these technologies as we can, whether that's empowering other people to develop and create experiences or creating experiences that are for a wider audience and for a much broader group of people. And that that looks like inviting different kinds of people to be engaged with your design process and giving you feedback. Right. We've talked a lot about feedback in this conversation and that there is still an incredible opportunity to grow this space, but also to help decide who gets to write the future of what this looks like. And my hope is that everybody who is invested and engaged in doing work with these technologies is actively looking for ways to engage people who aren't like them and maybe haven't had the same opportunities that they've had to be exposed to these technologies and helping to find ways to give them space and give them resources to engage in this work with us.
[01:12:17.447] Kent Bye: Great. Well, Doug, it was great to be able to catch up with you again after the latest Facebook conference this year, Facebook Connect last year, Oculus Connect 6, but to do our own collaborative sensemaking of trying to resolve what's happening, where this is all going and how we relate to it. So I really value your perspectives and where you're at in the industry and being on the inside and helping to bring about these different types of changes behind closed doors. And I'm upset as a, as a journalist who wants to have these up in the public and conversation, but I can understand why the sensitivity of them might want to be public. It's just, as we look into the future, we have, you know, this future where I don't want to live in a future where all of these conversations are happening behind these closed doors and you have to be in the inside group to even have a discussion about some of these things. And How do we start to bring in some of the principles of a liberal democracy into these virtual worlds as we move forward and not just have things be run by a company that's in control by a single man for billions of people? So anyway, thanks again for joining me today and for doing this whole unpacking and debriefing from Facebook Connect.
[01:13:20.231] Doug North Cook: So thank you. Absolutely. Always a pleasure, Ken. We'll have to do it again for Facebook Connect too.
[01:13:26.883] Kent Bye: So that was Doug Northcook. He's the founding faculty for the Immersive Media Program at Chatham University in Pittsburgh, Pennsylvania, as well as the founding instructor for the Immersive Design Residency for the Fallingwater Institute. So I have a number of different takeaways about this interview is that first of all, Well, this general skepticism of augmented reality and these AR glasses and jumping into this vision where we wear these devices on our face all day long and they're recording everything around us and we have an artificial intelligent assistant that's there to help us and to be aware of all of our context and help us provide all this additional information about our lives. I guess I share that level of skepticism, especially after seeing all these different speculative designs that they were showing here at Facebook Connect. There's nothing that I see that's like, OK, definitely I want to put this device in my face that has cameras and it's tracking my eyes. You know, I can imagine a future down the road where maybe this has become something that everybody uses, especially if we start to have like AR devices that replace all of our screens. So if we're able to have what's the equivalent of working in an office and have all that information like right there in our glasses, and that would just increase the amount of screen real estate we had so that you had productivity boost and all this stuff. But Generally, augmented reality glasses are more limited field of view. It's more about being out in the world. And we also have like a global pandemic. So a lot of these use cases for a lot of people as a shelter at home and not really going out in public that much, they think there's probably just a lot less applications that are immediately obvious. Now, a lot of these are clearly speculative designs, and I think there's a huge gap between the potential of what AR could be and what you can get from it today, or even next year with smart glasses. You know, one of the things that Mark Zuckerberg says is that you already have like the Apple Watch, so you already have this ambient way of getting notifications without getting the full aspect of what you're getting from your phone. So there's already that offloading some of those important notifications that you might want on your watch, but having it in your field of vision available at any moment, any time, I think for me, at least, I've already have issues with looking at my screen too much. And so, you know, just to have glasses that has it there all the time, then what kind of world is that creating to just have this overwhelm? Keiichi Matsuda is an architect and he put out this video called Hyperreality, which really explores the potential of what that world could look like if you just get it into the wrong hands of trying to get all this information clutter in front of you and things that are just trying to grab your attention all the time. So it doesn't necessarily feel like this calm technology that Doug Northcook is talking about here in this conversation. It seems like anxious technology that's trying to grab your attention all the time rather than calm technology. Now, this is not to say that this is not going to be useful at some point, but I think that if you look at technology diffusion, it usually comes up with the academic idea. And then you have like a custom bespoke application that's enterprise use case. And then at some point, once you prove the value and enterprise use cases, then eventually you cross the chasm and you start to develop it for these mainstream consumer products. And we could see how these early efforts from like Magic Leap and Meta and these other companies that have tried to like jump straight into the consumer market with some of these AR devices, then trying to really prove the utility and the value of something that's going to be changing so many different social dynamics. Project ARIA for Facebook is this experimental research project where they're going to be putting these devices that are recording both the eye tracking and the video of these Facebook employees as they send them out into public doing this social experiment to see how the public reacts to having these Facebook surveillance cameras on their faces. Now, they're saying that they're using AI to scrub the faces and everything else, but there's also personally identifiable information just by what they're wearing, where they're at, how they're walking, lots of different information that can still be identifiable even if it's not your face. So there's still potentially lots of PII that's in there, but you have the situation where you're walking around with these cameras on your faces in order to see what is the reaction going to be. And I suspect that the reaction is going to be a little bit of resistance. Like, do we really want to just jump into this future with egocentric data capture and contextually aware AI? Which is, I think at this point, still pretty speculative in terms of how much is AI going to be able to really understand about our context, our environment, and what do we really need this AI there for all the time that we can't already do with our phones? And I'm sure there's going to be lots of different, very interesting, compelling augmented reality applications at some point that are able to bring people together who are strangers and be able to interact in some way. This is something that Pokemon Go actually did quite a bit in terms of bringing people who are in this shared context of this game world in, and they're able to start to make these connections to each other. So I actually think that, you know, both connecting to other people, but connecting to the world around you in different and deeper ways, that there's certainly some compelling applications there. And I guess it's this trade-off of whether or not you want to wear this device that's both tracking your eye tracking as well as the world around you. So a big thing that I took away from this conversation, Doug Northcook was saying that he watched the antitrust hearings. And when I did this conversation and I hadn't watched it yet, I went back and watched the five and a half hours of testimony from Mark Zuckerberg from Facebook, Jeff Bezos from Amazon, Sundar Pichai from Alphabet slash Google, as well as Tim Cook from Apple. And I think that this larger conversation as to what's happening in the tech mega corporations and the role that they're playing in society, I think that the antitrust legislation has been really outpaced in terms of even making sense of how to ensure that there's still competition that's happening and that there's not anti-competitive behaviors, which has been to any time there's been any type of viable competition to one of these big players, it's usually gobbled up and neutralized in that way. So there's certainly a lot of things that have been happening even within the VR industry in terms of potential anti-competitive behavior, just with the interviews and conversations I've had on the record with Darshan from Big Screen as well as Guy Gooden from Virtual Desktop. So there's these different instances where you're starting to see a little bit of this anti-competitive behavior. But during the antitrust hearings, there's a lot more information and context that was brought up in talking about Instagram with these threats of like either you get bought out by us or we're going to just clone your features one by one. So there's actually some email conversations that had to be submitted to the Judiciary Committee to look at some of these different claims for antitrust. And so they actually had some evidence that wasn't just the CEOs testifying. And so the thing that I noticed, and one of the things that Doug had said, is that, you know, Facebook is getting a lot of the grief around these issues. What Amazon has been doing, what Apple has been doing, as well as what Google has been doing is also there's things that all these big major companies have been doing as well. I think from the VR perspective and the VR industry, at least, you know, Facebook is by and far the biggest player of all these other entities. And so that's natural that people within the XR industry are going to be focusing on Facebook above and beyond what's happening with these other companies. But we'll say after looking into it a little bit more that the story of what type of anti-competitive behavior that Google is doing just in terms of the ad markets they have and how they're buying and selling at the same time, it starts to get into these really wonky aspects that unless you're really up to speed on how healthy markets should work versus markets that are really dominated by selling and buying on the same end, having insider trading that's happening, not to speak of all the different complications of the PageRank algorithm As you search for something, then you have potential different answers to those searches that are happening based upon different people. And is that reinforcing different aspects of filter bubbles? So there's all these other issues that I think it's not only more difficult to understand for people who are already in technology and to explain clearly, but there's some things that are happening, like, say, in Facebook's realm where the harms are a lot more clear when it comes to, like, hate speech or ways in which that they have to do content moderation and fake news and misinformation. see a trace of that, but a lot of these other companies, a lot of those traces are just harder to see. It's not that they're not happening and that there isn't similar anti-competitive behavior that's happening across all these big major tech corporations. It's just harder to not only like capture and tell the story of, but for people to intuitively understand like what's actually happening. So I did a deep dive into that antitrust because I do think that we have a number of different levers that we can press when it comes to bringing about change. Lawrence Lessig's Pathetic Dot Theory, he talks about the law and ways that laws can start to change these different dynamics and antitrust has certainly played into the current dynamics that we have right now. The market is the certain market dynamics like without a viable market competitor, then you start to have more unchecked behaviors where Facebook can kind of get away with more stuff because there isn't any market competitors that are really having this inherent check and balance from a market perspective. And so the only check and balance is is something that the government's gonna step in. And it's an open question at this point as to whether or not some of these antitrust hearings are gonna have any teeth and if there's going to be any of the breaking up of Facebook. And even if that did happen, it's questionable as to whether or not VR would even survive in that breakup because a lot of how VR is even existing right now is because of Facebook as an entity has this war chest of all this extra money to invest in something that at this point isn't bringing any immediate returns on their investment. they're doing this loss leader to build up their market position. And the other things that people can do is the culture to be able to talk about it and, you know, have discussions like we're having here and to just educate people and to bring about different cultural aspects of like, how do we relate to this whole situation? And the final thing is just the technological architecture and code. So are there decentralized alternatives? What's happening with OpenXR? What's happening with WebXR? Things like SideQuest is a great example of a technological solution to the over curation that's happening in the app store. And that's actually like brought about some change within Facebook where they're starting to adopt some of those different concepts of having like unlisted apps where people can have an app that's listed, it's not listed in the store, but they can still update it and they can still have access to the Oculus APIs. And there's certain restrictions in terms of how many you can actually download. But that was new information that came out of Facebook Connect, where they're going to be making these unlisted apps that are available so that people can start to deliver some of these different applications. So that's just an example for the technological architecture and code, how that can kind of bring about change. But we're really left with people voting with their dollars. Facebook is pricing it at $299 and at $399, they're certainly per unit basis losing money. And so they're betting in the future that they're going to be able to recoup that money based upon creating this ecosystem where they're able to sell other experiences, but also potentially to start to bring in all sorts of other data into this big data capture. So, you know, that was the thing that Doug Northcook was saying is that, you know, is this just a big data play? to get into the future where the promise of augmented reality of having these always on glasses that you have on your face with cameras looking out into the world combined with what you're actually looking at in that world is like a goldmine of data to be able to start to use that to be able to potentially even if you're doing real-time inferences in terms of what you're looking at and what you're paying attention to, and if that raw data is not stored, but at least you're getting information and trying to do that real-time processing to do computer vision and to identify objects and to look at your sentiment analysis and if you're interested or not. That is quite a lot of information to have on your body all the time. If you listen to what Facebook is saying, they're saying, well, these are different ways that we can make these products more accessible. It's like all this surveillance on you is making the immersive technology is just more accessible for more people because we can bring it at a price point that's affordable for folks. And the final thing I'd say is just this discussion around getting private feedback from people like Doug Northcook, who has been under NDA and doing different consulting with a variety of different companies, presumably also including Facebook, and that he's able to provide feedback behind closed doors, but hopefully, from his perspective, try to bring some of those people out into public and have these conversations within an academic context. there's lots of different ethical and privacy issues with where this is all going and also just justice issues of like we're creating a virtual world that is going to be a replication of our real world in so many ways in which that we're building a society but that society is privately owned by a single corporation that has real world impact in our lives. And one of the things that Doug said is that, you know, you already have like apartments that are being listed on Facebook. And if you're like banned from Facebook, or you decide not to use it, then you're actually cutting yourself off from some of these things that used to be done within a newspaper or something that didn't require like a terms of service or a corporate account that, you know, you have to sacrifice all this different aspects of your privacy in order to really engage with. So they're becoming like this public utility, but yet they're privately owned, which means, you know, there's all these implications for free speech. And what's that mean for should Facebook be the arbiter of free speech for 3 billion people in the world? Is that something that is really that they should be their job? Should it be more broken up and fragmented and left into the hands of the people within communities? Because when you have these communication networks, you create these new problems. OK, like how do you actually deal with this issue? Essentially, they're becoming a government. as they move forward technologically, it's like this technological virtual world and they become the dictatorship of that world. How do you have different levels of transparency? Is it a worker co-op? How do people take ownership or have a vote or a voice? Is there any level of democratic feedback? Like this relationship between corporations and users in a more democratic fashion just hasn't been fleshed out yet for what that even looks like. And as we get deeper, deeper into VR, We're going to be asking these questions even more. And so we're starting from a point where there's no democracy, no feedback, no transparency, no accountability. They're just going to do what they're going to do all behind closed doors and then kind of just rushing forward without even being willing to engage in this conversation. And I think that I think is what I see is the most worrisome, is that where is the entry point to be able to enter in and just have a public conversation? And after I did this conversation and interview, I did hear back from some folks from Facebook saying that they were going to make someone available to talk about some of the different privacy issues, potentially, I mean, they haven't committed, actually, they said that they, they want to potentially have me talk to someone, but I haven't had that confirmed yet. So that's certainly a move in a positive direction. But just these larger issues that I'm talking about, you There's a certain way in which Facebook will hide behind some of these corporate lobbyists, trade associations, in ways that they will let them speak, or they'll do these private consultations and maybe release a white paper here and there, or they'll have some of their researchers give a talk at a conference. But in terms of really engaging in a robust public conversation and dialogue about this stuff, I just don't see it happening, despite what Mark Zuckerberg says his intention is, that facilitate a public conversation. I haven't seen it yet going and doing a few interviews with journalists who are not really ramped up on all the context and issues in the space. I mean, a lot of these times, Mark Zuckerberg will go and do an interview for like 10 or 15 minutes, maybe 20 minutes at tops. And that's it. They'll get like three or four questions, but it's not like a long extended Deep dive conversation to really dig into some of these issues and these questions and how do we actually like facilitate this complicated nuanced? Conversation that goes beyond the headlines and that sound bites and just to really like have a real embodied live conversation but also to facilitate that And, you know, one of the things that Andrew Bosworth, also known as Bos, said is that, you know, they consulted with privacy experts on different stuff. And part of the feedback was, don't just listen to what we say, listen to what we do. And I think that is spot on. There's a lot of things that are said during Facebook Connect. And it's like the one time of the year where they release all this new information. So it's, Taking quite a bit to just to start to sit through all the different things that they announced and you know I did a whole Twitter thread trying to unpack it and I'm not gonna go into all the nuances and details here of everything that came out They did a huge data dump did lots of interviews. It's their time to really say, okay This is where we're at and this is where we're going and so, you know, that's one way of having a public conversation but again, it's totally one-sided in terms of they just are talking at us and there's no receiving of that and there's no creating a context to even really have that dialogue happen. And what do we want in terms of like this more engaged dialogue? There was no like opportunity even to question and answers of any of the sessions of Facebook Connect. I mean that to me is such a perfect metaphor for how Facebook is not listening at all. They don't even have an opportunity to ask any questions. It's just like we're just gonna talk at you and we're not even have time for Q and A. no way to speak back or to participate in no shared spaces with anybody from Facebook. So I don't know, I think that's quite telling a metaphor for what I get, at least from Facebook, is that they're kind of set on what they want to do, they're going to do what they're going to do, and they're going to move forward without much feedback or input from folks. And, you know, that gives me pause in terms of, These larger questions, not only of AR and all these different ethical issues and whatnot, but also just VR in general and biometric data and like where this is all going and some of the roles that Facebook have in terms of how do you actually engage in ways to actually have some level of transparency. Like Doug said, there's no Freedom of Information Act to see any of this additional information or context. And Mark Zuckerberg having a 58% voting share, he's basically like in complete control of this company and He can do what he want without much recourse, not even from the shareholders. So what's that mean when you start to move forward and you have a company structure like that, and how do you have these different conversations and interactions with the public? So yeah, lots of unanswered questions that I've been trying to look at, but for me, that's some of the big takeaways and some of the experiences that I had from Facebook Connect. And yeah, I'm just grateful for Doug to be able to have this conversation and to be there in the moment to have all the stuff that was going on as well. So. Anyway, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast, and if you enjoyed the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a list of supported podcasts, and so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash voices of vr. Thanks for listening.
