At the VR Privacy Summit in November 2018, I met a number of privacy engineers who were actively implementing privacy-first architectures. I wanted to bring together some a group of these forward-thinking privacy architects together for a more technical panel discussion at SIGGRAPH in order to share some of the insights and open questions and problems yet to be fully resolved. Here are the panelists that I brought together for this discussion:
- Matt Miesnieks: Co-founder & CEO 6D.AI
- Diane Hosfelt: Privacy and Security Lead on Mozilla’s Mixed Reality Team
- Taylor Beck: Privacy Operations Lead at Magic Leap
- Samantha Matthews Chase: Founder at Venn Agency focusing on self-sovereign identity, safety & security
This is the first time that Magic Leap talked about some of their privacy-first architecture philosophy, and I’m really impressed with how seriously they’re taking the ethical implications of mixed reality. Beck cited the eight Fair Information Practice Principles as a key document that helps them operationalize their privacy practices at Magic Leap. Miesnieks wrote an article for Tech Crunch titled “AR will mean dystopia if we don’t act today,” which lays out what’s at stake when it comes to the future of the AR Cloud. Mozilla’s Diane Hosfelt wrote a paper in May titled “Making ethical decisions for the immersive web,” which gives a lot of great context for privacy engineering for the immersive web. And Samantha Matthews Chase has been working with helping form the W3C standards for Decentralized Identifiers and self-sovereign identity, and previously participated on panel looking at how blockchain technologies could be used with immersive tech to help people take more control over their data.
A big take away from this panel is that Privacy in XR is Hard, but it’s definitely worth trying to come up with ethical frameworks that can help provide some best practices.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye, and welcome to the Voices of VR Podcast. So continuing on in my series on XR ethics, today is a panel discussion that I did at SIGGRAPH of 2019. And just a little bit more context as to this panel and how it came about. So at the VR Privacy Summit back in the fall of 2018, that was at Stanford, put on by Jeremy Bailenson, High Fidelity, Jessica Outlaw, and myself. And we brought together like 50 different people from around the VR industry. And we had this like eight-hour day, this workshop really talking about a lot of aspects of privacy. And I think there was some intention that we'd be able to get together, talk about it, and at the end of the day, be able to present to the wider community, OK, here's how you address privacy. And the striking thing about it was that at the end of the day, there was no way to kind of distill down all the complexity of this issue into a document that we could give back. So it's still kind of like an open question for like, how do we handle privacy with these numerous of technologies? So since that meeting, I've been doing a lot of different talks at different locations, like I did a panel discussion at Southwest Southwest, and then this talk at AWE. And so at this point, I want to bring together some of the people that I had met at that VR Privacy Summit that I thought had very specific engineering and architectural insights about what the different trade-offs they have to weigh in their mind whenever they actually make a decision. So privacy engineering is just this concept that like you're looking at all these variety of different ethical issues of technology in general, but also like how do you do privacy protecting architecture? And I tried to get together people that I had met at the VR Privacy Summit and beyond to be able to have them come together and have this technical discussion about some of these different trade-offs. So in this panel discussion, I brought together Matt Meissinks. He's the co-founder and CEO of 6DAI. So he's doing like the AR cloud as a startup, a small startup, his perspective of the different ways that he's trying to think about some of these different privacy implications. And he actually also wrote an article in Tongue Crunch, kind of going through some of his thoughts on privacy within these immersive spaces. And then Diane Hossfeldt, she is at Mozilla. She's in charge of privacy and security of mixed reality. And she's been writing a lot of academic articles. She actually helped a lot helping piece together, you know, in order to do a panel discussion, you have to submit a technical paper for SIGGRAPH. And so we had to pull together all these different citations and stuff. And so she helped out quite a bit of putting that paper together. Then Taylor Beck, he is the privacy operations lead at Magic Leap. So I had had discussions with Taylor at the VR Privacy Summit, but at that point, Magic Leap wasn't talking about anything about their privacy architectures and actually thinking quite deeply and doing a privacy first design architecture. And he's at the center of a lot of these different discussions and trying to negotiate how this mixed reality system of Magic Leap is going to be on the right side of privacy. And I'm actually really impressed with a lot of their privacy architectures, and I actually really pushed and advocated for them to talk about it here for the very first time, to come out and start to discuss some of these ways in which they're doing these privacy-first architectures. So I was super happy that he was able to get clearance to be able to come and talk on this panel about what Magic Leap is doing when it comes to privacy engineering, And then finally, there's Samantha Matthews-Chase, who I met a couple of years ago, talking about self-sovereign identity. She's somebody who is really working at the cutting edge of the blockchain and decentralized identifiers and self-sovereign identity, and really creating these bleeding edge architectures and implementing them in specific contexts. And it's a nice dialectic, I think, to be able to think about, you know, if you were to really try to focus on data sovereignty and privacy from you owning your own data and you doing what you want with your data, then she's really advocating for a lot of those positions. So this was an interesting conversation to bring together and talk about these different trade-offs in different ways and just how difficult it is to do privacy engineering. So that's what we'll be covering on today's episode of the Voices of VR podcast. So this conversation with Matt, Diane, Taylor, and Samantha happened on Monday, August 29th, 2019 at SIGGRAPH in Los Angeles, California. So with that, let's go ahead and dive right in. So welcome everybody to this panel discussion. We're going to be talking about the ethical and privacy implications of mixed reality. So my name is Kent Bye and I do the Voices of VR podcast. And so for the past five years, I've done over 1,100 interviews in the virtual reality and augmented reality space. And one of the topics that has emerged from the community is all of these different ethical, implications and privacy implications of mixed reality. And so just this past fall, I helped organize a VR privacy summit, which brought together lots of different people from the VR industry to have this day-long summit talking about some of the different architectural options and trying to get a better sense of what this landscape means and the different design decisions that companies could come up with. And I think the outcome of that VR Privacy Summit was that there's still a lot of open questions. And so three out of the four panelists today were at that Privacy Summit. And so I wanted to gather together some of the people that I thought had a lot of really distinct insights into the specific architectural decisions that they're making within their context and their companies to be able to have their own design philosophies around privacy, but also try to figure out how to navigate this new realm. So I'm going to invite my panelists to just introduce yourselves and give a bit of a context as to who you are and what your relationship to this topic is.
[00:05:47.991] Matt Miesnieks: Okay, I'll start. I'm Matt Meissnecks, founder and CEO of a startup called 6D.AI, and we are a computer vision company that's building software that lets regular phones capture and crowdsource a 3D model of the world in real time. So if you've heard of concepts like the AR cloud or the mirror world or even Magicverse type of thing where there's this digital replica of reality, My company's building the software that's going to aggregate and assemble that model. So obviously there's a bunch of privacy nightmares that we could stumble on.
[00:06:24.430] Diane Hosfelt: I'm Diane Hossfeldt. I work at Mozilla on the mixed reality team where I'm the security and privacy lead. And I just kind of accidentally started working on the ethics of mixed reality because somebody said something. I was like, oh, this could go horribly wrong. And then it just kind of spiraled from there and here I am.
[00:06:51.113] Taylor Beck: Hi, my name's Taylor Beck, and I'm the Privacy Operations Lead at Magic Leap. I've been in privacy for about nine years, and I've been at Magic Leap now for just over a year and a half. As a privacy professional, having the opportunity to work on a device that has two cameras pointed at your eyes was something very intriguing to me, so that's why I made the leap, so to speak.
[00:07:12.487] Diane Hosfelt: Oh, you did not just. You just did that.
[00:07:17.360] Samantha Matthews Chase: My name is Samantha Matthews Chase. I am the founder and CEO of Venn Agency. And we've done a whole bunch of stuff, but our main claim to fame has been hosting virtual environments, specifically scanned environments on tiny servers and also through IPFS hosting distributed worlds. And I kind of fell into identity and privacy because I've been building worlds on the virtual web for since 2015 and got really freaked out by actually light beam from Mozilla, among other things, showing all of the trackers that were following me around the web and actually working to sort of visualize those trackers into, like, things in XR experience that you can see. And I sort of fell into a really wonderful community around self-sovereign identity and started building and architecting and thinking through what a self-sovereign identifier and identity could look like for a person or a company. And that led me to a very interesting place, which is safety and security, which is that I'm building now products that help put safety information, specifically safety fixtures, like fire poles and exits, into our long term memory. Our brains work almost identically actually they do work identically to our own GPS that we've created. We're spatial creatures, and so if we actually use everything we've learned from spatial computing and VR and AR, we can start to hack the right information into our minds. I'm working on building that for shipyards, C-SPAN, which is the largest shipbuilders in North America, and working with the BC government on using our credentials, that's a part of the self-sovereign identity stack, to create a new insurance category that makes safety practices auditable for the individual.
[00:09:08.577] Kent Bye: Great. So in covering this space, it's the realm of all the different privacy factors. There's a lot of them. And I think that maybe the best thing to do is, first of all, just start with the personally identifiable information and why is it important to keep that private. We already have the web and what could happen, but I feel like part of what's happening with these immersive technologies is that we have all sorts of new information that could be added together, aggregated, and get all sorts of specific information about us. And so, just curious to hear maybe a baseline of the existing threats and how, as we start to approach this, what is the first line of defense or ways of thinking about personal identifiable information and how that relates to the privacy threats within spatial computing.
[00:09:54.006] Diane Hosfelt: So, for me, to start with, You know, Mozilla has a manifesto, which is like, I've decided that anything I write is now going to be a manifesto because of that. And one of the principles of this manifesto is that privacy is a right, and individuals' privacy and security on the internet should be a right. It shouldn't be, you know, a privilege, something that just people who can afford to pay to avoid trackers, et cetera, deserve. And so, for me, the key starting point is that when we look at immersive technologies and we look at mixed reality experiences, the sensors and the derived data that we use that are required in order to create these experiences are inherent privacy threats. And there are sometimes technical solutions to this to guard it. But you have to be able to look and analyze the space around a user in an immersed environment. Otherwise, you're going to make them sick or you're going to hurt them. So you have to have this data. which means there has to be some way that's better than just saying, oh, well, we trust companies to do the right thing to safeguard this. And that's the key starting point for me. We need this data. It's a threat. We have to have it in order to do this. So how can we do that without ruining any chance we have at privacy? Because I do think we can have both.
[00:11:41.130] Samantha Matthews Chase: I think it first states what privacy matters and why privacy matters. Because the threat is larger in the sense that it's kind of existential. Because if there's this digital twin of you, it's a collection of your preferences and your influences. you start to get this notion of otherness, of other self, and live more in that, and let that be what guides you. And I think there's a lack of autonomy that comes with that, and that breeds otherness, separation, lack of cohesion in terms of how we share our time, because everybody's got their own little timelines and bubbles. So I think when we talk about privacy, we have to think about what we want to keep private, And maybe it's not bad to have an entire second self, but if you have no access to have any autonomy over that, that matters. Because what we're talking about is all of this data being out there and somehow thinking through this exercise of how that could harm us. as our physical selves now, instead of thinking about ourselves as both here and there, and as a collection of all of that into one place. And if we're not given any access to collect ourselves into one place, I think we start to give up. We get this cognitive dissonance where we just accept. And I think that's a bigger threat than anything.
[00:13:06.638] Taylor Beck: So when you're thinking about things like Diane was talking about, about the information we need to actually execute spatial computing, the mapping of environments, and then you think about how that could relate to information about a person, the way we kind of approached it at Magic Leap is thinking about how can we bifurcate really the information about people from the information about the device. So we've made a lot of architectural decisions in our time to basically take all the personal information from an individual and put it in an AWS instance, and then take all the spatial mapping data and put it in a GCP instance, so that we're running our entire backend architecture based on the device without knowledge of the individual, and then adding the disincentive cost of egress to actually put that information back together.
[00:13:51.138] Matt Miesnieks: I guess we've got a similar in that it's our companies cares about spaces and places and we're not couldn't care less about individuals so you know one of the common things we sort of working through is when people think about scanning a space you think of that scan as just a singular piece of data and really there's different layers there's dense point clouds, sparse point clouds, geometry meshes, textures, semantic labels. And then all of that data could be aggregated from multiple devices and multiple sessions and who knows what. So trying to sort all that out and figure out what do you store, what do you expose, what leaves the device, what goes on the cloud. There are all the questions that are like practical architecture challenges each day. And I guess the principle we're taking on it is trying to figure out this definition of what can actually be reverse engineered into something that's either recognizable by a human or usable by someone else's software. And if it can be reverse engineered to either of those things, then we really don't want to touch it. We'll say the texture to make a space human understandable, so it looks like a sort of photorealistic model, we'd rather that texture be stored by the application developer or the content owner, not by us. But there is some SLAM level data that's useful for stitching and aligning spaces together. synchronizing the positions of phones and devices into a space that can be stored and cannot be reverse engineered through any known science back to something that a machine or a person can use. You know, we're really pushing the limits of not just the technology, but of the science in this space. But what I think as a startup, what we realize is that what we really get to choose is our incentives. And we made a choice as a team to not pursue an advertising driven model. You know, if we pursued ads, we could make all this stuff free and we could just aggregate it. But that creates incentives for us to behave in a certain way with the data where We're more building the company as an infrastructure company, like Amazon Web Services, or Twilio, or Stripe, or that type of company, and choosing those type of incentives where our customers are the people that are using the service, and our incentives should be aligned then to sort of treat that data with the appropriate level of respect. So that's, I guess, a little bit philosophically how we're coming at it, but also for us, the idea of personally identifiable information, we have to look at GDPR, It was like, well, what does this even mean in our context? And we just had to make some guesses there, with our lawyer's advice, but they didn't know either.
[00:16:40.022] Diane Hosfelt: I think it's interesting that you bring up the regulatory aspect of this, because I think that a number of us here are, I don't know if it's like fans, a hobby, something that we have to think about. And generally speaking, privacy regulation is I believe, colloquially called a trash fire. But it's something that if you're going to talk about this space, you also have to back up and you have to ask, what is privacy and what is personally identifiable information? And the answer to that It's pretty much, I don't know, nobody knows. How do we define privacy? There's no decent legal definitions of privacy, realistically. Privacy is contextual, it's societal, it's personal. Which is to say that privacy is hard. And personally, identifiable information, particularly in this age of machine learning and AI, is also very hard. And there's a huge overlap between AI ethics and MR ethics in that we're collecting a huge amount of data. What do machine learning algorithms like? They like a huge amount of data. we might take data and think that, oh, well, you can't get anything out of it, but you can. And we don't realize it until we've already started down that path, which is why it's so important to ask these seemingly obvious questions. What is PII and what is privacy? While your technology is still emerging, before the industry really has sunk billions of dollars into a path and we have to try to retrofit better decisions into that because that's really hard and it doesn't really work all the time.
[00:18:44.828] Samantha Matthews Chase: Don't you think we've already sunk billions of dollars into something that's not working though? Like, really, though, if we think about architecting privacy, I would like to say the positives of where we are right now in mixed reality and this sort of undefined space is that we have a chance to actually visualize and articulate what that looks like for us. Like, if we're talking about mediating reality between a digital space and a physical space, that's literally what we're talking about. We're talking about slamming an environment and tracking motion and scanning an entire environment at the same time. That isn't the only way to do it. There's an entirely different way to do it. And we don't have to think about things in terms of users and logging in and all of these. Basically, browsers don't respect people. Like, no offense to any of the people here, respect people. And, like, that's because you don't work for companies that are building for something for that. You're building to try and take over an environment or know an environment. And when you could, hypothetically, right now, use all of this visualization that we can overlay onto the world to map things better. to create better spatial indexes. And actually, it makes more sense to have the people who are the property owners of the places that you guys are slamming be the ones who give you their digital information about their property. Because we can scan an environment and host it. So it makes more sense if we want to talk about actually architecting a privacy layer to what does a privacy layer look like? Well, it looks like somebody has to have a little bit of autonomy to be able to decide what it means for them, at least, before we even talk about what it is for people. And that doesn't exist yet. There's no layer for that. And we have laws like Castle Doctrine, and we have laws that prevent people from coming in our doors. We have locks on our doors. And all we need to do is use AR to start to digitally visualize this. for everyone else and create autonomy both visualized as bodies and avatars, but also for places.
[00:20:50.293] Kent Bye: I want to just jump in and sort of give how I see the different dynamics here. Because we have a number of companies whose business model is based upon surveillance capitalism, where they're advertising based, they're trying to, as Tristan Harris says from Center for Humane Technology, they're creating these voodoo dolls of ourselves that are trying to both create this predictive model, but when I talk to a behavior neuroscientist, the line between prediction and control starts to get blurred and go away, that if you can predict something, you can then control something. So we have the situation where you're gathering all this really intimate data on us already, on our behaviors online, and then once you start to hook in biometric data about what we're looking at, what we're paying attention to, what our emotional reactions to that are, And if there's certain identifiers that are being able to connect that information to us, and if it's stored, then all of a sudden they're able to potentially start to reverse engineer our psyche to be able to understand our values, our preferences. And then we're living in this world where we have all these companies that are trying to get in the middle of whatever we're doing and trying to persuade and control us. So I do feel like that there's this one model, which is we own our own data, we're not giving that data over to somebody that then they're trying to control and manipulate us through that data. There's different layers of homeomorphic encryption or differential privacy that could put some sort of barrier so that it's obfuscating it in some ways. Or we find some way to navigate the nuances of the third party doctrine and give it to a third party and then suffer the consequences of all that data that we're giving over, eroding our Fourth Amendment rights to that information to be private. So I feel like that there's all these new realms within immersive and spatial computing technologies that are opening up these new vectors and venues for these companies to start to potentially access this data and tie it to our identities. And so I know that a number of you are trying to come up with different stopgaps or firewalls to either separate or to find different architectures. Sam, I know you've been doing a lot of like in the self-sovereign identity. And I know Taylor, with Magic Leap, you're trying to like make sure that personally identifiable information isn't be able to kind of be tied back to people so that they could start to then storehouse that data on us. So I'm just curious to hear a little bit more of the design decisions in terms of the technical architecture that you are making in order to start to consider those boundaries.
[00:23:13.693] Taylor Beck: Sure. So I mean, as a platform, we have two privacy problems. We have the privacy of the data that we collect and then the privacy of the data that we expose to applications. We've made our decisions around our architecture, about how to run a device-based architecture. But then we have the problem of applications needing access to our APIs. So what can we do to minimize that access? So we've taken a very crawl, walk, run approach to opening up our API suite. We don't have identity APIs that are available for publishing as yet. We don't collect a lot of personal information about people anyway. So there's not much that we would be able to expose even through that. Then we thought about how do we provide great granularity of choice, allowing users to make meaningful decisions about what's being exposed to an application. Then for what we can't control technically, we try to control contractually through our application submission requirements, where you have to provide a justified business purpose or a supporting application purpose for accessing an API. You can't just request the entire manifest and get it granted to you for an application. And then, additionally, some of the other things that we've done really is to minimize the amount of data we actually expose to that which would an application need. Sandboxing applications so you're not bleeding API data in the background, and a number of other kind of security features around ensuring that only data that's supporting an application working is what's generally exposed.
[00:24:32.190] Matt Miesnieks: I can give a very practical and specific example of a problem we are wrestling with right now as a team. You know, we have a type of neural network in our system that, in real time on a phone, will let the system draw a 2D mask around people in real time and sort of select them out. And Apple released a version, a similar thing, in ARKit 3 preview. And you may have seen like little clips on Twitter of sort of people with funny like filters and stuff over them. Now that technology or that capability is really useful for building reliable slam systems. If I want to localize my phone in this space, knowing to only pay attention to feature points that are on the space and we can just ignore all the people, everything that's likely to be moving around or changing. It makes the system more robust, more reliable, and just work better. It means you can scan a model of a space, and from a privacy point of view, people never even get past the camera sensor. It means we can just ignore people altogether. We don't have to blur them or remove them later. They never even get captured. But at the same time, it means that you could be running this and I could be just erasing people. If I had a pair of glasses on, I could be erasing people as I look at them. I could hook that up to one of those makeup apps that lets me sort of change skin color and just change people's skin color in real time as they walk around. And all of a sudden you go from this API, which is exactly, like the API is exactly the same thing. It's a trivial sort of call. And on one hand it improves privacy and makes the system robust, but then you could also use it to do this sort of dystopian social type of behaviors in your glasses. And we've got this and we're sort of scratching our heads. Should we expose it? How do you expose it? And even those diminished reality where you sort of erase people, like there's use cases for that if you want to take selfies at a tourist spot and just get rid of the other tourists in the background so it works nicer for you. How do you do that? How do you control it? How do you design for the intent? And we honestly don't know. We're still figuring out and trying to think through what are these unintended consequences of what is a simple API. I mean, as a practical example of the type of architectural challenges and the different ways something could go that's exactly the same bit of code, they're the sort of problems we're wrestling with.
[00:27:00.584] Diane Hosfelt: I think that that example is incredibly powerful, because it really highlights the only way that we can possibly address this is by talking about these things before someone does it, right? Because you anticipated it, and you're thinking about it. And so for me, when we're talking about architecting for privacy, It's so common for applications and platforms to have this dedicated privacy team, which is great. It's important. You need a privacy team, right? But it's not the sole responsibility of that team. Privacy should never be siloed. everyone who is responsible for creating the product, whatever it is, needs to be thinking about privacy, needs to be thinking about how can my technology be misused? How can it be abused? How can I make this safe for people? And that means your UX people, your back-end developers, your in the clouds dreamers, your CEOs and founders. Everyone needs to feel responsible that their product doesn't make the world a way worse place by, you know, enhancing biases or making abuse and harassment online worse, right? We don't need it to be any worse. It's terrible.
[00:28:31.054] Kent Bye: And Sam, just in terms of, I know the people as well, I'm curious to hear a little bit about what does it mean to potentially own your own data and the architectures around that and whether or not something like homeomorphic encryption or differential privacy, how that fits into the equation. But I know that you've done a lot of stuff in the self-sovereign identity community, but it's still sort of emerging in terms of these blockchain approaches and just kind of give us a sense of what's it mean to have access and own your own data?
[00:28:59.375] Samantha Matthews Chase: Well, first I'll say that you can't own anything, so you can control your data and you can control access to it. And it's sort of a tough thing because when you think about ownership of something, then you think about maybe getting paid for it and I can't remember his name, but he's one of the GDPR guys. And he's like, just because you could sell your kidneys on the black market doesn't mean you should. And that's a pretty good way to think about selling your data or owning data in that sense. I think we need to think about it in terms of more the sense of turning it into a better system of asking questions and being able to answer those questions. When I go to the bar, I have to show my eye color, weight, height, and home address to a stranger. That's not what the driver's license was ever intended to do. And that's just laziness and the part of people building these systems. So what they really want to know is if I'm over 21, and I am. And so when we think about what identity is, and autonomy is, and control is, and how you start to control things and have autonomy, you have to first think about what the thing is you want to control. If data is just raw data, you might want to be a little bit careful with that, because it can answer a lot of questions, right? And maybe only you want to have access to that, and maybe only in a world where there is a place that you can look at it on your own and not fear somebody looking over your shoulder. But until then, data should never be raw data. It should just be the answer to a question. And so that's what a verifiable credential is. It's a replacement for these legacy systems that we use with our identity and certificates. And it's a way of saying like, yeah, this would be very hard to spoof. And you can use blockchain. You don't need to. But a verifiable credential is just a way of turning a piece of data into a certificate. And so with One of my products, for example, whenever somebody plays the game, that makes a credential that's stored on a ledger, but it can't be identified to anyone. It just takes the phone's IMEI number and made sure that it was in the right place that the game was played, that it had to be played for it to be approved. And so my system is only asking, is this a unique person? Are they in this place? Did they do the exercise? Because that's all I need to know. And so what I would ask people to start thinking about is, what do you need to know And when you think about identity, it's like, well, actually, what do you need me to show you about myself? Because identity is kind of this sense of otherness. It's really, at best, a collection of a whole bunch of little things you can prove about yourself and a whole bunch of other things that people might say about you. So a login is challenging, because you've got to have a different identity for all these different places. You have a work computer here. We have all these different contexts, but only one body. And so we need to just think more in the sense of a wallet and sharing our information in terms of having a conversation. What do you need? I can give it to you. And I think if we want to talk about areas for people to focus on, if they really want to fix this, it would be on machine-readable terms, which is a working group out of IEEE. That would mean that you could state your terms and then have other machines just read them and know that. And it would be using verifiable credentials in place of secure data stores or storing people's data. And that's a group with the W3C. And then decentralized identifiers, which is an emerging standard. but it's a way to kind of anchor these different identifiers in different routes of trust. And so I would recommend that people start to think about things that way, I guess.
[00:32:46.718] Kent Bye: And I'm curious to hear from other people as well, because I know that some of this decentralized self-sovereign identity and own your own data, these are relatively new architectures. And so as your companies, there may be trade-offs or risks or different philosophies that you may take. I'm just curious to hear how you've sort of evaluated this new emerging possibility and then what you've done instead.
[00:33:08.591] Taylor Beck: Sure, so I thought a lot about self-sovereign identity and what kind of applications we might be able to use with it. And I also think about a lot of the other requirements that we have from a regulatory perspective about data that we have to get. And so when you're approaching self-sovereign identity, it needs to be kind of a two-way street. There has to be regulatory changes that were mandated to collect certain information. I have to collect birth date from you because I have to know that you're over the age of 13 so I don't collect data about children. So getting the solution in place but also getting the regulatory changes in place is important. I also think a lot about what impact self-sovereign identity might have, and I get a little bit concerned. One of the good things about the current system, although it's relatively broken, is that companies need to disclose everything they're going to do with your data in the form of a privacy policy. Now, it could be a very easily readable privacy policy, or it could be an obtuse, destructive piece of legalese like Airbnb's.
[00:34:11.135] Diane Hosfelt: Not to call on anyone specifically.
[00:34:13.336] Taylor Beck: Sorry. But you have to make that disclosure. If you change what you're doing, you have to tell people. You have to provide a material notice or a change of your practices. I feel like if we move towards a self-sovereign identity, we're moving towards a more libertarian model, and the onus of responsibility of our own data is going to move to the individual. and it's gonna be a lot harder to make informed decisions about things. I think about asynchronous data collections and then remarrying the data later as one of those things. So company A can collect your eye tracking data on the first of the month and your object recognition data at the end of the month and then put them together and you've made two choices that were informed in their narrow scope but not able to make choices of how that data kind of lives on. So I think there's a model that we can move towards where using things like Ledger, being able to give people kind of informed transactions about how they interacted with companies and how data's been shared, but it's gonna take some well thought out approaches.
[00:35:14.548] Diane Hosfelt: So self-sovereign identity is not something that I think about. It's not my area. It's not what I've been working on. But the comment that you made about putting the onus on individuals, yeah, that's a problem. We see it with permissions, right? Generally, permissions on the web. More importantly for this context, permissions on the immersive web. Permissions are really, really hard, it turns out. And we're basically, we're asking, when we give you a permission prompt, we're asking you to make a decision that you can't possibly be well informed about. I don't really care how good you are at technology or in privacy. There's no way that you would, who here would have anticipated that the ambient light sensor on your cell phone could be used as a side channel to extract your phone's pen? I work in security and I never would have thought that, but then researchers do it and all of a sudden you're like, oh, well, now we're taking something, we're putting it on our head, we're taking it all around our house and all around the world with all of these new sensors. And it's non-obvious, one, what vectors will appear in the future, and it's even less obvious how these things will interact with each other. What new side channels are we creating for features that we consider foundational to our device security and our data privacy? And so asking individuals to take on all of the responsibility for that, I'm trying to think of a word that's better than stupid.
[00:37:07.024] Samantha Matthews Chase: But it's similar to like... But I think, sorry, just to, the idea is not to put the onus on individuals, it's to give individuals the ability to state their preferences and have that be represented somehow. It's not libertarian to want to have, have to accept terms of service and cookies every single freaking time. It's annoying to do that. And so, like, a self-sovereign identity is not about, like, no one's going to have one. They're going to have a messaging app that, like, respects their privacy and they got to set it up once. And then whenever they need something, they just send their little messaging app that they trust because it was built on the principles of this. It's not that they're like, I am my own person and, like, screw the government. It's, like, not that at all.
[00:37:50.910] Diane Hosfelt: To be clear, I'm not speaking directly to self-sovereign identity because it's not something that I have much knowledge about, but just the general ideas of individual responsibility for managing privacy and identity. They're a very dangerous path to walk. It's hard for security professionals to anticipate this, and we don't often until somebody's like, You know, this looks really weird. We should see if we can use it to do something even weirder, right?
[00:38:24.497] Kent Bye: Yeah, I just wanted to pop in because I feel like this is sort of speaking to a larger dialectic here between the centralization and decentralization. We have the centralization where people do have these adhesion contracts where you read a bunch of terms of service. They're basically taking this colonial mindset where we're going to come in and seize all this data, and they're going to own it and use it against you. But they're also, at the same time, providing services. And so I think a lot of people have been, up to this point, mortgaging their privacy in order to get access to free services. And that's been a more utilitarian approach. But at the same time, it is this undermining of our own autonomy, because they're using that data against us in different ways. ways that we may not be fully, it's like an asymmetry of power between the amount of information that they have as a centralized entity and what they can do with that data versus that moment in the decision when we give that consent. And so I see that we're in this time period where we're actually trying to tip the scales away from this sort of centralized colonial mindset of just trying to seize all of our data to be able to manipulate us in different ways. And I do think that if there's one thing that I see from this self-sovereign identity and these alternative architectures that are a little bit more decentralized is that it is trying to give us a little bit more autonomy. But with that autonomy, we do have a little bit more responsibility.
[00:39:36.872] Samantha Matthews Chase: I don't even think that they're trying to give you more autonomy, I think they're just trying to own another thing. Like, most of the identity companies are like, oh, probably start turning on a little switch at some point so I can just skim a little off the top there. Everybody is the same and decentralized and decentralized because everybody's trying to own a space instead of build a layer or build a protocol or work on a data shape or work on a link between databases or anything that might be useful.
[00:40:07.685] Kent Bye: And I think that part of the big open question right now that the entire industry is facing is, what are going to be the business models that are going to really sustain this? And I feel like, if anything that I see, it's moving away from this harvesting of our private information and doing this surveillance capitalism and moving more towards where you're paying up front for that data. I know, Matt, in some sense, you're in the front lines of this because you're providing a service of trying to move into this new model. I'm curious to hear where you see this sort of tension of the existing business models and how 6D AI is trying to approach this.
[00:40:43.939] Matt Miesnieks: Yeah, I think the question around ownership versus control is the right question. I think control is really the question that needs to be answered. Again, I'll try and keep it really practical in terms of the types of problems that we are working with right now, not sort of hypothetical a year or two out. So one constraint, I guess, is that in computer vision and AR and MR, the foundational technologies and algorithms, the science, is moving very, very fast. And a lot of the user experience is 100% driven and enabled by the quality of an implementation of the algorithm underneath the system. And so what you end up with is data structures and algorithms that are pretty much one and the same thing. And so at the bottom layer, the data itself is proprietary. That's Google's ARCore, or Apple's ARKit, or our system, or Magic Leap system. We all have proprietary data and algorithms underneath it all. So that makes things like interoperability and openness and sort of like, well, you just can't. simply connect our system to Magic Leaps or open it up to Apple's and everyone just worries about the user experience over the top of it all. So that might be solved at some point in the future, but right now the open solutions just are generations behind, you know, what's being built inside the companies. So then you get into the sort of question around control of a space and we looked at, okay, you scan your home, you know, someone goes and scans it and You might be renting, so what control do you have over that scan versus the landlord, versus a family member who visits, versus the teenager's bedroom, versus the six-year-old's bedroom, versus guests that come around for a party? We then said, how does the system even know if you're in an apartment that behind that wall is someone else's apartment? And so that's completely different rules altogether. And we came to the point, like today, we didn't know how to solve technically, didn't know how to solve that problem. So the way we're coming at it is, like I alluded to before, we're just not taking anything from the device that can be turned into something that's actually usable or recognizable. But customers are asking for that sort of information and they want to put it in their applications and there's tons of really good use cases that would benefit people by applying that sort of information. How do you enable control and give those sorts of permissions in spaces that are very, very hard to define boundaries around? From a technology point of view, we're still scratching our head trying to figure it out.
[00:43:35.135] Taylor Beck: Yeah, we faced a lot of the same kind of questions. And we made the decision from an architectural standpoint to just not associate mapping data with any device or person. It's all our maps are organized around hashed BSS IDs because we use the Google Geolocation API. for a location and breaking that up into cells that are signal strengths attached to them. But there's nothing about any specific person or location, find location that will change since iOS decided to deprecate BSSID. But one day, we couldn't find a good solution for figuring out ownership of space, particularly with multiple people in a household. And then who owns specifically the routers and all those things.
[00:44:22.180] Matt Miesnieks: I guess one thing I did forget to add that we felt was a good interim solution was that a device shouldn't be able to retrieve the data about a space without the device being physically present in that space. So I can't sit here on a computer and look up your home. I'd need to actually be in your home and do the query and match what my sensors are seeing with what's in their hands.
[00:44:46.141] Taylor Beck: Yeah, we've taken similar kind of approaches, thinking about doing CV feature validation against maps to ensure that people are not spoofing location and harvesting maps. We're thinking about putting together some anti-teleportation methodologies, what we call them, rate limiting your ability to move far distances at a given time. I want to teleport. just to kind of avoid those things. But yeah, it's a challenging problem.
[00:45:12.576] Samantha Matthews Chase: That's the problem I'm solving. It's basically people know and control most of the space on the earth and there's documents and deeds and trees and spatial indexes that document all of it. It's all run by Esri, pretty much. And if you're a city or a municipality or anything like that, that goes through their geospatial index or information system. The thing that you're talking about is that there's really no layer of truth that isn't proprietary spatially at all. And so until we have that, your stuff won't talk to his stuff, because you're all both trying to query a space by slamming it and using it against your own cloud. You're separating it from the person, to the here, to the there. And it's a whole lot of technology happening at the world, at people, at the place. Whereas there are owners of places, building managers, people who could be given a product, and usually those people that are going to buy your Magic Leap or use your product are the people that pay for their Wi-Fi in their place. And it's very easy to add a little attachment to the back of a Wi-Fi router that says, this is my place, this is the geofence of my place, I'm going to scan it with 60AI or Magic Leap and this is my geofence of my area. And then the building can be your REST API and tell you the location. You don't need to be a big spy helmet trying to solve the world's problems from your one little device. This is what we're trying to talk about here. Shouldn't the earth and the places we're in and the people who manage those have the same digital twin presence that everyone else is interacting with? Wouldn't it make sense if there was one single point of truth, a geospatial index that wasn't owned by Esri, Google Maps, and buying keyhole satellites and all that, maybe if there was just one point of truth that everybody could call to, the same way the web works, then your stuff would talk to each other.
[00:47:13.429] Diane Hosfelt: I'm going to jump in here, because I've been waiting very patiently. Yeah, sorry. It's supposed to be funny.
[00:47:24.307] Kent Bye: Wait, were you going to reply to that?
[00:47:26.828] Diane Hosfelt: I was going to back up from that.
[00:47:29.710] Kent Bye: OK, let me just put one point on that, and then you can move on. So I talked to Mark Pesce, who is doing a mixed reality service. And I think that this is an open question of who owns the space. He was trying to propose a standard to be able to have something that isn't proprietary in some way. So I know that Mark Pesce has been Suggesting that but it sounds like from what I'm hearing is that it's an open problem There's ways to maybe solve it with certain patchwork solutions to put together and that it's something I think will be continued to be talked about in terms of like Who owns a space and who has access to it and Mark Pesce is an interview I did with him He's sort of talking about the mixed reality service as one potential solution for that. So just wanted to put that out there
[00:48:14.043] Diane Hosfelt: As we go back and forth and talk about these things, I like to back up a level and state that privacy engineering, and really anything to do with privacy, is always a conversation about trade-offs. That's the same when we're talking about anything centralized versus decentralized. And so for example, I'm going to talk about some social VR spaces, if that's... on the appropriate time for that. So, shameless plug, Mozilla has Hubs. Hubs is a social VR space. You can go around. It used to be that you could kind of just embody a robot and throw rubber duckies at your friends. But now you can do more than that. You can draw really cute cats and you can have meetings at Hogwarts. It's great. The way that we do hubs is decentralized. We do have accounts, but we don't really store any information in the accounts. They're literally just to give you an identifier. And if you wanted multiple accounts, you could easily do that. So what happens if, say I have a room, I've created a room that makes me the room owner, and then I share a private link with you, and you join my room, and then you decide for some reason to share it with somebody else. And they join the room, and it turns out that they are just a really horrible person, and I want to boot them. So I kick them out of the room. But I can't permanently kick them out of the room because they can always rejoin because we don't maintain any like, this is you. And so there's some where you can now put requirements on rooms, right? Like you'll have to authenticate via Discord. But that's one of the big problems when we're talking about centralized versus decentralized, is that in a decentralized world, To avoid having that hub in the middle that hands out identities and does management, you're assuming certain amounts of risks. And one of those is, for example, here, the inability to permanently kick-ban someone from your space. And so one of the mitigations for that is we don't have public rooms. As a general rule, our model is we make the links private, and then if you wish to open it up, you share that. And you can open up the space more than that, but it's all a question of risks and trade-offs. And that's essentially, when we're talking about location, we're talking about all of this, it's an open question because there is no answer. And I think that that's something that we should all keep in mind while we're, you know, discussing these things is that we're not going to arrive at a solution because there is no solution. All we can do is identify the factors that are, you know, the different variables of the problem. the different solutions, the trade-offs between them, and mitigations that we can take if we identify that there's something unacceptable in one of those outputs.
[00:51:42.081] Kent Bye: Cool. In about 15 minutes or so, I'm going to open it up for questions. So if people have questions, we have a microphone, and we'll have some time, about 20 minutes at the end, to be able to answer some questions. But I wanted to ask about this trade-off between, We have a limited amount of compute resources on these devices. And so then you have certain approaches, like homomorphic encryption or differential privacy, where you may want to try to do things, like if you had infinite amount of resources, we could do all sorts of things. But you're kind of working with a very limited amount of GPU and CPU. Curious to hear a little bit of other types of architectures that you've been looking at, and then seeing maybe in the future this might be potential, or what's available now to use. Because there's a bit of overhead that you could do to be able to do a little bit of extra processing to make sure that things are working, and that you want to maximize your privacy. But having that seems like a trade-off where they're not at the place where it's a free thing. There's a trade-off and a cost there. I'm curious to hear a little bit more as you start to evaluate some of those different trade-offs.
[00:52:51.220] Taylor Beck: Well, whenever there's compute available on a device, there's like a mad rush for it. So, privacy doesn't generally get a good chunk of that compute, but we do... The approach we've taken is we process all raw frame data locally on device in a segregated computer vision processing unit, then isolate it in a secure envelope where it's only attached to our main compute by a proprietary data link so that we can't ever get access to it. We're getting all the raw frame data and all the first order algorithms processed locally. Whenever we can, we endeavor to process locally so that we don't have to see it. And the device does run completely locally. It doesn't need to connect to our services. It's got very limited memory, but it will function. So if you're not moving around in tons of different spaces, it's good to go. From a forward-looking perspective, we're always trying to optimize the amount of local processing we can do and getting enough compute to actually execute that. We've looked at differential privacy. Most of the libraries out there don't work with our datasets, so we've actively tried to engage academics and non-profits to help us try and identify methodologies for exercising differential privacy.
[00:53:58.617] Kent Bye: Is that a matter of having enough machine learning data on that? Or is it just more of algorithmic?
[00:54:03.120] Taylor Beck: I'm not the data scientist, so I don't know why it doesn't work. OK. OK. And then we've talked about homomorphic corruption, but the compute costs and latency are just not scalable for us.
[00:54:14.367] Diane Hosfelt: What was the compute cost on it? No. Again, it's not much. I was just wondering what order of magnitude it was. Several.
[00:54:22.192] Taylor Beck: Several. That's all I got on that.
[00:54:26.682] Kent Bye: Yeah, and Matt, I don't know if there's anything, because you're basically creating an application on top of the hardware that's already there. So you're not controlling the hardware. But from your architectural design point, I'm just curious how you sort of start to think about and try to separate things out and not have things talk to each other.
[00:54:44.294] Matt Miesnieks: We're not doing any special encryption or anything. It's more the algorithm and the data structures, anything that leaves the phone. To the best of our knowledge and our research team at Oxford, and we've sort of checked this with peers at Magic Leap and Microsoft and others, we're not aware of it being possible to sort of take, like if the NSA came and took our servers away, they wouldn't be able to figure out how to turn this data back into like a geometric model of your living room. Yeah, and that's about as far as we've gone. We don't expose anything from our libraries apart from a mesh of just the surrounding area, and that's up then to the application developer what they do with that. But that's as far as we've... I'm sorry if it's taken it. Yeah.
[00:55:34.895] Samantha Matthews Chase: To answer the question in a sort of broader architecture point, you could build fog networks. Ideally homes and smart homes become the next computers. And people can have a device that just has a little bit more memory on board that can maybe encrypt your shitty LED light bulbs or Wi-Fi light bulbs and things like that. Because it really just comes down to what, if you can compute a hash, or not. And most of those little things can't. So, home fog networks are something that I hope like Mozilla IoT is really, really cool. Super cool little team. They're killing it. Samsung took away their smart home open source platform. And Mozilla IoT has stepped in and there's a really cool little personal boxes you can buy. U-Boss is one. It just comes pre-built with an xCloud server. And starting to see, ideally, hopefully people start to take choices that consider their IoT devices as a part of a home and not individual things. And then hopefully we start to see more connection in the sense of these IoT devices pairing up and offering each other, sort of like opening each other up to offer encryption to each other. I think that would be really cool.
[00:56:46.233] Matt Miesnieks: One thing I forgot, you reminded me of the fog thing, is something that's unique about 6D that no one else does is we do all our processing on the phone. We don't do any processing on the cloud. So if you look at Google or Microsoft, their spatial anchors type product, all of that takes some sort of image data from the phone, puts it up the cloud, processes it, and gives you a result back. We do it all locally, so there's no image data or nothing that's recognisable that ever even leaves the phone.
[00:57:18.970] Kent Bye: So I want to talk about privacy in the sense of a comprehensive framework for privacy. Because I feel like at the beginning of this year, I went to the American Philosophical Association. And the founder of the philosophy of privacy, Dr. Anita Allen, was giving a whole lecture to all these philosophers. And she was saying that this is a big open philosophical question, that we do not have a comprehensive framework for privacy. And then it was in that moment when I heard her sort of put forth what the work that is yet to be done philosophically, I was like, oh. If there's not sort of a comprehensive framework that's out there, that means that it's kind of left up to all these individual companies, that you have to make these moral intuitions and your own decisions as to where that line is. And I feel like we're kind of in a space where there's a lot of major players that have been making some really bad decisions of transgressing over those lines of privacy, that we're starting to actually define what that line is for us. And I feel like there's certain aspects of biometric data and our financial information taking our mortgage information and adding it together and where we live, our home address, our location, our private communications, our hobbies, our sexual preferences, and abortion is actually a big part of privacy law in the United States, but also like our medical information. So to come up with some framework to say, okay, this should be public and this should be private, how do each of you navigate their own sense of cultivating that moral intuition? through the lens of the transgressions that may have happened, or if you think it's an impossible thing to try to come up with some sort of comprehensive framework for privacy.
[00:58:53.713] Diane Hosfelt: So are you asking from a philosophical perspective, a regulatory perspective, a legal perspective?
[00:59:02.985] Kent Bye: Well, I think philosophy feeds into the legal. In some ways, the lawyers and the policymakers are not getting any guidance. And part of what I saw in this tension is that you have, on one hand, philosophers who are in their ivory towers that like to go back and really figure things out and get it perfect. And then they are like, here it is, ta-da. But yet, we're kind of in this realm where the technology is about 10 to 20 years ahead of where the philosophers are. And so it's like maybe philosophy needs to be a little bit more agile. because you want to make it good enough, but then at the same time, you don't want to put it in stuff that's going to be bad law. So I feel like it's kind of a dilemma where things are moving so quickly, but that I would rather have something rather than nothing, or a different approach than what we have right now, which is this huge lag of the philosophical frameworks around it, the regulatory oversight, and these companies who are moving fast and breaking things, and they're breaking a lot of things.
[00:59:58.385] Diane Hosfelt: So I'm going to hand it to Taylor in just a second because I know that he has far more experience with GDPR and regulatory nuances than I do. But one of the important scene-setting things that I want to mention is that basically all of privacy law in the US is sector-based. So we like to pick one thing, and then we regulate that. But then if something isn't directly comparable, or even until someone makes a court case saying that that's directly comparable, it's completely unregulated unless it falls under a previously regulated sector. And then in Europe, of course, you have more comprehensive approach. And there's a lot of historical reasons why this is all true. But more practically,
[01:00:53.525] Taylor Beck: So I would argue that there is kind of a philosophical framework. The fair information practice principles are developed in the 70s by the FTC and then expounded upon by the OECD in the early 80s is the basis for pretty much all privacy legislation. Things like transparency, consent, access, purpose specification, use limitation, security. That's really the philosophical framework on what regulations are all built on. How those are operationalized, it varies. When you talk about non-regulated industry in the US, it all falls under the FTC's unfair and deceptive trade practices. Enforcement, so say what you do, do what you say, don't do something else. Which a lot of people have trouble with. Because how many of you engineers actually read your company's privacy policy when you're building something? So I think you can expound upon that fair information practice principles as a basis for looking at a philosophical framework of privacy. It's more operational than it is thinking about what the definition of privacy is, but it's meant that way because privacy is the ever-moving target, what you consider to be a privacy invasion. At one point in time, may not be in another, or under one circumstance, it's a good, in another circumstance, it's a bad. So you need flexibility in how you kind of approach privacy. and not be rigid in your definition of it.
[01:02:18.782] Matt Miesnieks: I mean, as a small, like, 15-person startup, I don't think companies like us can really be trusted to figure this stuff out. Not because we're, you know, up to bad things, but, I mean, even if I've got the most pure soul imaginable and we make every decision with the best intentions and the best of our knowledge, it's still going to be the, you know, whatever we end up implementing is going to be the result of our own thoughts and ideas. companies, you know, maybe us, maybe not us, but you know, this happened with Facebook and Google and these other big companies that were small at a time. If things work out for us, business-wise, we could end up with whatever our thoughts and, you know, whatever we implement could end up being something that implements all society. So, you know, my, you know, I hate saying this as an entrepreneur, but I actually think, you know, smart regulation that sort of requires us to make decisions that are flexible or changeable later, or at least, there's some real fences around what we do, I think would be a smart thing for society to have in place.
[01:03:30.731] Diane Hosfelt: I have a lot of thoughts about this. So Kent, obviously, just stop me when you need to. And part of this is because my mother was a lawyer. So everything was a negotiation and everything was a contract. And I got very good at looking for loopholes. And there are a lot of loopholes here. And so when we're talking about comprehensive privacy as a framework, we have the principles. Principles aren't particularly enforceable. So yes, you have the FTC. The FTC, their ability to enforce and their punitive powers are both limited and very widely based on what's happening in the government. So one problem is, going back to the incentivization that's been mentioned by both Matt and Sam, You want to have positive incentives for doing non-shitty things, but you also need negative incentives for when people do shitty things. And a large fine, that's still just a fraction of your yearly revenue, that's not much of a negative incentive. And so that's part of it. We have to be flexible and yet also be able to enforce. It's really hard to achieve a balance that isn't going to put startups out of business because of regulatory overhead, but that will also help larger companies actually apply to both the spirit and the letter of the regulations or the laws. So, a really interesting thought experiment that I like, it's not really an experiment, because people are doing it right now. So, say you're interviewing with a company, and they want to know if you're a good fit, right? So, they're like, all right, here, just put on this VR headset. And while they're interviewing you, you know, they're getting information from, I don't know, say like your gaze and your eyes, and maybe they can measure your galvanic skin response, et cetera, et cetera. All of these things that are eerily similar to what polygraphs use. Now, I hate polygraphs. I think they're stupid, I think they're useless, I think they're pseudoscience, because they are. But unfortunately, people still try to use them. It is illegal in the United States for a private company to terminate or punish an employee solely on the basis of a polygraph result. Now, if they have other things, they can still do it. And it's always gray down there, right? But is that going to immediately apply to MR technologies and data that can be used to achieve the same ends, quote unquote? Probably immediately not. What will happen is someone will get terminated because they were doing their promotion test in a VR headset, and then they were fired for being a liar based on it. And then they'll sue. And then after a couple of years, they'll make the argument. And then we'll have that in place. But that's a long, complicated, unpleasant process, right? So how are there ways that we can anticipate how things will apply and put these things in place so this doesn't happen, that we prevent it before somebody has to lose their job and everything. And there are a ton of examples in education and employment over the misuse of this data that I could talk about until I lose my voice, which is what usually happens. So I'm going to leave it at that. But again, privacy is hard, and everything is complicated.
[01:07:35.731] Kent Bye: Cool. Sam, did you have something you wanted to say?
[01:07:40.008] Samantha Matthews Chase: I'll just say the more philosophical viewpoint that each of us needs to have a realm of privacy in order to explore who we are as people. We need to have a way to be able to self-reflect, a way to be able to self-reflect and test out some of these ideas of self with our family. And then from there, we need to be able to test that out with our community. And if we don't think of privacy in terms of those realms, and we think of them merely as frameworks, which they are, they require, but if we don't start from the sort of realms of privacy that exist from a human outwards, I think we'll get it wrong every time.
[01:08:19.206] Kent Bye: Cool. I think it's a good time to open it up for questions. So if you have a question, please make your way over to this microphone over here, and then we have about 18 minutes or so.
[01:08:30.829] Questioner 1: Hello, thank you very much for all of your candid and really interesting thoughts to share today. Something that's come up a couple of times is the idea that everybody in the company is responsible for, or should be responsible for thinking about these privacy and ethical concerns from the CEO on down. And in particular, Matt, I believe, is that your name? Did I get that right? I'm terrible with names. You were talking about a specific instance where you were considering the potential unintended consequences of an API that serves good as well. What are the systems or structures that you have within your organization that help the people at all levels explore those kinds of questions or bubble them up?
[01:09:17.575] Matt Miesnieks: So we're like a 15-person company, so we almost have no systems or structures for anything. Right there with you. What we do care about instead is more around values and how do we communicate those values to our stakeholders, to new hires amongst ourselves. We've written them up, put a lot of them publicly. Is it a manifesto? It's not a manifesto, but it could be. It leans that way.
[01:09:50.722] Questioner 2: It should be.
[01:09:52.800] Matt Miesnieks: Yeah, and I think it sort of gets back to Sam's point earlier. I think we just try and say, look, what sort of people do we want to be? And if you've got a shared set of values that we all believe in, then it's through that lens that we should look at not just the privacy decisions, but every type of decision that we make in building our products. And at some point, we'll hire HR, and we'll hire these people to help us put real policies and systems in place. But we're making it up as we go, and none of us really know what we're doing. And we're trying to just do what we think is the right thing, and all of us have a shared understanding of what we think the right thing is. So that's about as honest an answer as I can give.
[01:10:40.683] Taylor Beck: Magically, we use a privacy by design model. So, basically, privacy is integrated into the software development lifecycle, starting at the product requirements document all the way through to pushing into production. So, the engineering team, we're a part of regular development, so the engineering teams, the product teams are constantly in consultation with the privacy team. to understand what product we're building, how we're going to build it, and then figuring out what requirements we may need for specifically that product or any dependencies across the platform to ensure that kind of moves forward.
[01:11:13.868] Questioner 2: Hi, really interesting conversation. Thank you. I'm also interested in ethics of access to sort of the new world that's being created and things like phone booths don't exist in the world in many places anymore. And just paper maps are degrading. It's hard to rely on them because they're so bad. So I'm wondering what sorts of things you guys see disappearing from our world in the near future and how that might put pressure on people to opt in even more than we already have to, and how that might strain some of the systems that you're putting in place to control privacy. Will this accelerate? Will elderly people who aren't trained on technology be left out? What are the ethical implications there?
[01:11:58.499] Diane Hosfelt: That's a really good question and point and a different dimension to the ethics of access than I've usually thought about, so thank you. Because usually I approach this from the whole, there are some devices that I can't really wear because I have longer hair. And I know that there are a lot of women of color who can't wear devices because of their hair. And particularly as we're seeing devices used in education, we really need to make sure that everyone can use these devices if we're using them for education, right? Like, we should always make sure that people can use technology regardless. So that's how I've usually thought about it. So I don't know what exactly I think will be disappearing. But I will say that I believe that mixed reality devices can improve access and accessibility in a number of ways, such as doing real time captions. for deaf or hard of hearing people, helping to provide additional input for people who are vision impaired while they're like walking around, you know, indicating like, oh, hey, you might have this crosswalk light, but there's a car running the light, right? Especially as we have, you know, quiet electric cars that you might not hear, things like that. But if anyone else has thoughts on
[01:13:32.563] Matt Miesnieks: taking extreme, like every sign, every bit of text, every public image, every bit of decoration that's visible could all disappear at some point in the future. That's not too hard. I mean, it's terrible to imagine it, but it's not too hard to imagine it.
[01:13:53.174] Samantha Matthews Chase: I think the infrastructure question is really interesting, especially because I think we have a huge infrastructure problem. And I think it's really difficult to have a conversation that isn't about devices when you talk about mixed reality. But we often just talk about devices, and we talk about the frameworks for those devices. But half of mixed reality is the reality part. And I think that things like phone booths and things disappearing are problematic because it really just becomes about access in general, socioeconomic being the biggest problematic one. And so I think if we could start to use spatial computing to better make our reality more coherent, to make databases talk to each other, to be able to mark pollutants, to be able to cross-reference things and make our lives better. That's the mixed reality that I'm really interested in and that I think that has a lot of space to develop if just like 10% of the people at this conference stopped working in gaming and Hollywood and computer vision for that stuff and started working on actual problems where the real world exists outside of the technology world, which is just a small bubble. a very small bubble. There is an entire world out there that needs infrastructure and the game engines that we're building and we're using should be used to make that.
[01:15:12.730] SPEAKER_04: So thank you guys for this whole discussion. This has been absolutely wonderful. I was wondering if you guys could speak a little bit to domain and definitions. Because it seems like throughout this conversation, we talk a lot about privacy. We talk a lot about mixed reality. But we haven't talked at all about the key points of what is privacy and what is domain. Because we have different privacy expectations for private and public domains. And privacy on servers, even if The issue becomes very interesting, right, because we have all of this data on all of our private servers. All of these companies are harvesting all of this data, and it's treated as private. But all of this data is absolutely in the public sphere, and we have a very different public expectation for what privacy is. And speaking to the phone booths and the things that are disappearing, those were public-facing private areas, which are also now disappearing. So you walk into this public space with this private information, and I was hoping you guys could speak a little bit to some of the difficulties and domain-related issues for dealing with these ethics in this very, very gray domain.
[01:16:19.753] Diane Hosfelt: I think that to start with, we just have to say there's no way to define privacy, unfortunately. I was recently reading a law book, because that's apparently what I do now, information privacy law. And it starts out with just a bunch of a chapter with a bunch of different legal definitions, essays on privacy. And half of them are privacy is good. And the other half of privacy is bad. And they're all compelling. So, what is privacy? Like, yeah, we should define it, but I don't think we can.
[01:16:59.482] Taylor Beck: Yeah, I mean, the initial definition... But isn't that kind of required?
[01:17:03.366] SPEAKER_04: I mean, as a first step for all of this. You would think. Well, no, but I mean, like, absolutely. But coming from a philosophy background, We can't do any meaningful work here until we have a definition to work from, whether it be legal or otherwise. So there needs to be, I think, as an industry, we might want to start a working group or something to try to make a definition to work from.
[01:17:23.484] Taylor Beck: Yeah, there's been hundreds and hundreds of lawyers who've tried to work out a definition for privacy, and the best they could really come up with is the right to be left alone. It just isn't really actionable.
[01:17:35.694] Kent Bye: I think this is why I want to say that there isn't a comprehensive framework for privacy, and this is why. The definition is so open, so I think it's an open philosophical question, but yeah, go ahead.
[01:17:46.740] Taylor Beck: I mean, like we noted before, privacy is very contextual, so you can't make a solid definition unless you outline all of the different contexts to design it.
[01:17:54.664] Diane Hosfelt: Which will take more time than the heat death of the universe.
[01:17:59.185] Kent Bye: We'll leave that to the philosophers. So I've asked over a thousand people what they think the ultimate potential of VR is, and they usually answer into one of the domains of human experience, one of the contexts. So I think that the challenge, I think, for me at least, I think is that if you start to map the cartography of the human experience in all the different contexts, if you have contextual computing, then you have to be able to identify what that context is and then be able to have it. So privacy is connected to context, and so it actually becomes more of an issue of defining what those contexts are, and then defining what the bound of the human experience is, and then try to figure out some abstracted way to figure out all those possible contexts. So that's the approach that I did at AWE, Augmented World Expo, and the keynote that I did. It's like a whole, like I said, an iterative approach, because I think that it is an open question, but I think that we are better suited to have at least something to push against rather than nothing.
[01:18:49.032] Diane Hosfelt: Sometimes I literally just sit at my desk and stare into the void thinking about this.
[01:18:57.621] SPEAKER_04: Thank you, guys.
[01:18:59.739] Questioner 3: So thanks very much for a great panel. A lot of the large corporations out there that produce digital content in the form of things like movies or music or even software are very zealous about guarding the content and not allowing it to be redistributed. I'm wondering if all the panel might be interested in commenting on a future in which, say, we regard ourselves as producers like that. and the hardware protection mechanisms that are built into all of the devices that we possess and use to view this media and are used on the sensor data that's collected about us from our devices. And so that we have a more even power balance between the people who are trying to sell services to us and the fact that we're streaming data out to them and we don't allow them to save on their local databases. So, for example, if you want to mail somebody something, how often do you need to access their address? Can you ask for it on a case-by-case basis and then have them refuse it potentially and end the relationship?
[01:19:49.625] Taylor Beck: So, what you're talking about is the basic digital rights management for identity. Yeah, for people instead of corporations. Yeah, it's a challenging problem, especially when you think about how much data is actually used to operate any ecosystem, and when you start thinking about those contexts, there's this thing we call in privacy, it's called consent fatigue, that people give up after being asked three times. I'm trying to think about an ecosystem where I would go through to execute something. You think of a simple example, like asking for an address to mail you something. But if you think of a more complex system, is that going to end up being like 50 asks, 100 asks? You know, how the lifecycle of that data works across an ecosystem. It gets challenging. I mean, is there an opportunity to do it in the future? I mean, it's something we're all thinking about. And I'll leave it to Sam to really go into it.
[01:20:41.605] Diane Hosfelt: More philosophically, I do not want to be DRM'd because DRM is, in my personal opinion, the worst.
[01:20:53.314] Questioner 3: But you'd have an option to opt out of that by having an end user licence agreement for yourself. That basically has to take the lot.
[01:20:58.502] Samantha Matthews Chase: The problem, the reason this can't exist is because data can't be named. So the granularity of data is, it stops at the browser and the website and then all those cookies and all the things you're looking at, that's not yours. And there's no way for how that's collected to be translated into something else. So there's no way to mark your data into a system and get any sort of retribution for it. And until there's data shapes and shared languages, query languages, it's just not possible. Yeah, I guess that's kind of what I was getting at, is you need to throw the incentive for that back onto the industry, rather than expecting them to... Well, I think, no, you have something like GDPR happen, where they're like, here, we're going to do consent, and then you have the California Consumer Protection Act, and they're saying things like consent, but, yeah, nobody's really giving consent, and we know that the ways in which we're asked isn't true informed consent. and so everybody's in this pickle right now, and that's sort of what a lot of these groups in are working on. If you're interested, I highly recommend, and also just to sort of touch on asking what Matt could do, and like, oh, maybe I should hire an HR department, but I would really recommend that any, if you don't have an HR department, join a working group. Join a standards committee. I really think that everybody who is building in this space should also be actively engaged in standards. There's like, if you like, highly argumentative, low agreeableness, but very passionate and caring people, they're the best. But that's also where you'll find a lot of, it really always comes down and boils down to design decisions and ethics. And so there's a lot of groups, IEEE, W3C, Oasis, there's quite a few that work on this stuff. There's been things that have gone very far and not made it out into the light of day for obvious browser war reasons. But there are people working on it actively, and I would suggest you try it. The data transfer project is a big one. That's Twitter, Google, somebody else. And that's basically, they're all working on trying to create shared data shapes for this very reason. Great. Thanks.
[01:23:01.825] Kent Bye: Cool. Just a few more minutes left. We have time for one more question.
[01:23:04.859] Questioner 4: Thank you. Sorry. I want to offer a metaphorism to summarize and got a question. So a lot of what was said fits into the kind of idea that the principles we need are for things with evolving semantics where we don't know, like, We as companies don't know, the consumers don't know, the society and regulators don't know the exact semantics of the objects we're talking about from the get-go and they evolve. And you're doing that in a land of exponentials where all the activity happens suddenly and late in the game. It sort of feels like... Maybe in the Facebook case, like at the beginning with a small number of users, maybe it was okay that it was kind of unregulated and it was evolving semantics for what social network data meant, but then it suddenly grew towards the hundreds of millions and then to the billions, and maybe the regulation and the frameworks for regulation were kind of late in addressing it. There also seems to be a principle in the privacy law, dating back as far as we can go philosophically, about matching expectations. The main responsibility we have is to communicate what the right expectations are in this land of highly evolving semantics, which is an ongoing and difficult job. What startups need to do with their early adopters is different from what large organizations, large companies need to do later. So I guess the analogy I wanted to give, it feels like out of a marina or harbour you're launching a number of different vehicles that move at different high speeds and some of them look like the Red Bull jumping off a balcony and diving into the sea and others look like a Hyperloop version of a yacht that's speeding out and you're trying to avoid like the people making the ships from dumping too many people into the ocean that are on top of the ship, crashing into other ships and crashing into the harbour. And you're trying to do that all at once. And so I think in summary, the question I want to ask anyone who cares to answer is, How best can we treat the process of matching the expectations of the people who interact with our products and the people who interact with them and the side effects on society? How best can we treat that challenge as an overall strategy at every phase of startup through to the big company? Thanks.
[01:25:23.095] Taylor Beck: So it's not easy. I spend an inordinate amount of time trying to distill down complicated computational concepts into easily understood language so that I can manage my users' expectations. Trying to explain SLAM mapping to someone at a 5th grade reading level is kind of difficult. It really boils down to having that principled approach and trying to be as transparent as you can about what you're doing. We've made that effort to try and convey as much information as possible and provide good granularity of choice to manage the platform. That being said, we've also tried to reach out to the broader privacy community because it's hard for us to do a full red team analysis of every possible bad way data can be used that we expose through our API surface area. So we're trying to start looking at standards and talking to academics, talking to non-profits like the Future of Privacy Forum to double check our work. Are we managing expectations correctly? Is the solution we put together the right solution? What could we do better? So trying to get feedback on our own practices is how we've kind of tried to approach it.
[01:26:39.610] Diane Hosfelt: All of that. I think that in general, when you're talking about engineering anything for privacy, you need to be thoughtful, you need to be respectful, and you do, while you might not be able to exhaustively enumerate threats, You need to think about how this could be used by a bad actor. With a social environment, what happens when people are harassed? Do you have appropriate moderations? And it's that first step by considering that perspective and trying to be respectful of experiences different than yours. It sounds like very little, but it does accomplish more than you would think.
[01:27:29.335] Kent Bye: Cool. And so with that, I just wanted to thank my panelists for joining us today. And it is a lot of open questions.
[01:27:34.879] Diane Hosfelt: Thanks for moderating, Kent.
[01:27:39.383] Kent Bye: So that was Matt Misinks. He's the co-founder and CEO of 6D.AI. Diane Hossfeldt, she's on the mixed reality team at Mozilla as the privacy and security lead. Taylor Breck, he's the privacy operations lead at Magic Leap. And then Samantha Matthews-Chase, she's the co-founder at Vim Agency. So I remember different takeaways about this panel discussion is that first of all, well, there is some fiery interactions on this discussion. And I think Samantha Matthews Chase, she's really bringing forth like this self sovereign identity, data sovereignty. And in some ways, there's a bit of a concern of With that freedom, you have a certain amount of responsibility. So, you know, when you have things managed and stored by other people that are taking care of where that information is, then what are the security implications of you, for example, carrying around all of your really intimate data around you on your body? So there's these different trade-offs because do you actually want to have a way for people to give access to all of your information if they are somehow able to get that information directly from you and break your encryptions? This is I think in some ways what we're seeing with people who have a lot of cryptocurrency money if they get held up on the street and if they get access to their key then they're able to basically take their money and there's no way to turn it back. there's these different trade-offs. When you have this complete autonomy and sovereignty over your data, then you may be vulnerable in other ways. And so I think that was, from a safety and security perspective, some of these different trade-offs that may come up. So I think there was some interesting dialectics there of talking about things like self-signal identity and owning your own data. I think they're important architectural keys, but there are still these different trade-offs. Also, in general, privacy engineering is hard. There's no perfect answer. And when you take a step back and try to define privacy like a universal definition for privacy, it does seem like there's going to be specific things that are going to be culturally dependent and what people are favoring. I ended up doing an interview with Dianne Hossfeldt where we go into a little bit of those different cultural differences. But just as an example, in the UK, they really focus on dignity of the individual. And so they have something in the larger context of GDPR, of being able to own your own information and control what is getting out and what is not getting out. But then there's this aspect of like, there's the libel laws that are in the United Kingdom that are completely different than they are in the United States, where in the United States, you really are preferencing that freedom of expression, rather than the dignity of the individual. But in that same sense, in the United States, you are saying that these companies can kind of come in and seize this data and start to own it on your own behalf. And so While in the GDPR and the European Union, they're really trying to draw this line of like you own your own data Whereas the United States that line hasn't been drawn yet. And so we have this surveillance capitalism model that's been really proliferating But you know when you take a step back and look at the philosophical definition There is no universal philosophical framework to be able to describe privacy in every single context for every single individual there does seem like there's going to be a specific ways that you can start to break it down, especially when you focus on specific context. And so one of the things that Taylor was talking about was the fair information practice principles. And so there's like eight different principles that are trying to limit the amount of data that you collect, the data quality principle, which is like the data that you're collecting should be relevant to what you're trying to answer. So you're not just trying to sweep up a bunch of information just arbitrarily, The purpose specification. So, you know, being able to define why you're collecting this data rather than just hoarding all this data and not knowing what you're going to do with it. And just on the off chance that you'll figure that out later. So being much more intentional, the use limitation principles. So actually disclosing to the user, like why you're collecting it and what you're going to be doing with it upfront before you start to actually collect all of it, the security safeguards principle. So once you collect the data, there's a certain responsibility for you to keep it protected and private. The openness principle is just disclosing specifically what you're collecting for people to have a little bit more transparency as to what exactly is being collected. The individual participation principle, so that's just giving different rights to the individual as to what type of information you can get from other people. Being able to communicate what information that you're collecting on people, like to be able to get some transparency specifically on that. And if you're denying that, being able to describe why that's being denied. And then if you want to get rid of it, just the right to be forgotten. So you can actually get rid of the data that is collected on you if you choose for that to happen. And then finally, the accountability principle is just that if you're collecting all this data, then you're accountable for complying with all the different regulations for how you actually manage all of that. So those are those eight high level principles, and those are really helping the privacy and security folks have these high level principles to be able to then figure out what the policies are, but trying to implement each of these different policies. But in terms of like a comprehensive framework for privacy, then you're starting to get into trying to define all the different contexts. So context was a thing that came up over and over again of how context dependent privacy is not only on the cultural context, but also the individual context that you're in. So in the talk that I gave both at AWE as well as what I did at Greenlight, XR strategy was to try to map out the cartography of an individual's contexts that they're in and try to look at ethics and privacy through that specific lens. So some of the other things that Samantha Matsui's Chase was talking about, like the decentralized identifiers, which is a lot of new specifications that were at the W3C, as well as the IEEE standard for machine readable personal privacy term. So can we specify in machine code, like what we want to happen to our data and to be able to help mediate the control that we have based upon these different services around this IEEE standard that is still emerging. So that's something that she had mentioned is something to consider as well. Another aspect was just how difficult it is to specify third party API. So once you start to create these different tools and techniques, then what are you going to be making available to other people to start to use versus what is going to be so sensitive as to for you to have a little bit more tightly controlled access over that. So for your first party collaborators, either for yourself or other people that are. working in more direct relationship with you, having access to some of these APIs that may have a little bit more sensitive information. They were talking about the different mitigating ways to be able to keep from you spoofing your location. So if you're talking about the AR cloud, then you would only want to reveal what information is stored in the AR cloud relative to if you're actually there and be able to prove that you're actually there. and to prevent you from teleporting around. So yeah, just thinking about this air cloud issue of how to mitigate against people from spoofing their location to get access to information that they shouldn't have if they're not actually in that physical location. And then just also just talking about how to segment different information. And so how information as it comes together, the different sort of aspects of like, let's say your biometric data information or your location information. And so seeing how when you start to combine information in different ways, then what kind of risks that opens up for mixed reality applications. And so magically was talking a little bit about like using these different data stores of using like AWS over here and using like Google Cloud over here. So using different cloud services to be able to segment out that information in different ways, you know, and then Matt, in some ways, saying that as a small startup, they're really quite limited. And so they're really relying upon these larger community discussions to help set some of this standards and whatnot, and I think in some ways he would welcome having the government be able to regulate different aspects of this so that it's not just up to a lot of these companies or startups to be able to define all of that. I do think that this is an area where there tends to be a little bit more industry that's innovating and pushing forward those envelopes, and then seeing what kind of harm is created once things go wrong, and then the regulation comes in to try to balance that a little bit. But to not do too much regulation beforehand to prevent innovation from happening in the first place, which is a lot of what Danny O'Brien was talking about in EFF, as well as in another conversation that I had with Diane a few months later, also just looking at these different dynamics of not wanting to create regulation that's going to prevent anybody other than the big major companies from being able to operate in this space. So, you know, for me, the big takeaway of this conversation is that privacy is hard. It's hard to pin down. It's hard to precisely define, but that once you start to isolate it down into specific contexts, you really have to have some sort of operational definition to start to work from. So this fair information practice principles, I think is probably a good start. It seems to be a pretty standard thing. There's the future of privacy has all sorts of different privacy references. And then there was the Menlo report and then a 33 page companion report from the department of Homeland Security. There's also the Ethical OS, which is trying to set forth some larger, high-level principles of these different trade-offs. So with all of this stuff with privacy, there's different trade-offs. There's no perfect answer. That's what makes the privacy engineering so difficult, is that whatever architecture you choose, you're trying to balance off imperfect solutions and there's going to be harm caused somewhere. It's impossible to eliminate all harm and to eliminate all risk. And in order to provide some of those benefits, you can't get some of those benefits without some of the risks, I guess, is the bottom line. And that's what makes this so challenging. And also, for me, just so interesting, because as people are starting to create these systems, trying to define what those frameworks are is a really hard problem. And I think it's worth trying to do it, because if you don't think about it, then you could start to be sleepwalking us into these dystopic futures where we have these technologies that are the worst surveillance technologies that we can possibly imagine. In talking to Daniel O'Brien, he said that, you know, as we move forward, it's kind of like a downhill road. So it's actually becomes easier for freedom of expression to make these proliferation of technologies, encourage lots of people to express what they're thinking, what they're feeling. But yet, it's an uphill battle when it comes to privacy, because the more that the technology Improves and pushes forward then the more that the aspects of our sovereignty and autonomy and our privacy are eroded and so I think it's important for the civil liberties aspects and perspectives of the electronic frontier foundation and some of the perspectives that Samantha Matthews chase was bringing on in terms of the self-sovereign identity and these different architectures to be able to help preserve and protect some of those civil liberties and some of those Aspects of our autonomy and our sovereignty super important to be able to consider how those different types of architectures could be included as well So, that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast, and if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a list of supported podcasts, and as an independent journalist and oral historian, I rely upon donations from people like yourself in order to continue to have conversations like this and to try to talk about the things that are Difficult things to figure out and we need to have much more people thinking about them and talking about them and hopefully with this series It'll help to catalyze even more Deeper discussions about all this stuff around privacy and privacy engineering and talking about these different ethical frameworks within XR So if you are enjoying this and want to see more of it, then please do support me here on patreon You can become a member and donate today at patreon.com slash voices of ER. Thanks for listening
