Rikard-SteiberApril 4th was the one-year anniversary of the HTC Vive, and HTC released the subscription service for Viveport for $6.99 to try out five new VR applications per month. I had a chance to catch up with Viveport president ​Rikard Steiber at GDC to talk about the genre differences between Viveport and Steam, future support of VR headsets beyond the Vive, their support for delivering content to China, as well as some of their arcade licensing options.

LISTEN TO THE VOICES OF VR PODCAST

Viveport is the first subscription service for high-end and room-scale virtual reality, and HTC has a chance to leapfrog other content companies like Amazon, Netflix, Facebook, and Google’s YouTube. Time will tell whether or not HTC will get into producing original content like other film subscription services have, as well as how much new and high-quality content Viveport is going to introduce to the platform each month. But for now, paying $6.99 to try out five new VR experiences a month is a great deal worth considering.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

Shari-FrilotShari Frilot started the New Frontier at Sundance in 2007, and programmed the festival’s first VR experience in 2012 with Nonny de la Peña’s Hunger in LA using an early Oculus Rift prototype made by Palmer Luckey. Frilot has since programmed around 75 VR experiences since 2014 that explore storytelling, empathy, and emotional presence, but she sees that it’s going beyond empathy. She says that being in VR gives us “the ability to see ourselves in a way that we could never do alone,” and that VR embodiment may allow us to overcome our unconscious biases. In speaking about embodying a number of different creatures in The Life of Us she says, “you can watch yourself tap these primitive instinctual responses and you watch yourself go into another place of being able to socially engage with somebody” beyond the normal labels of white dude or a black lesbian.

I had a chance to catch up with Frilot at Sundance this year where we talked about the power of story to change someone’s reality, the role of Sundance in the modern history of consumer VR, interdisciplinary insights into storytelling from over 10 years of New Frontier, how VR could change how we see and understand our underlying value systems, and how VR could help us reconnect the body to the brain in a new way.

LISTEN TO THE VOICES OF VR PODCAST

Here’s the short documentary that Frilot references in the podcast about “Scientists Have Found a Way to Make Paraplegics Move Again”

Here’s the keynote that Nonny de la Peña’s gave at SVVR where she talks about Hunger in LA and some of her early pieces that premiered at Sundance.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

KentBye-Avatar-2016I had the chance to give the keynote at the Silicon Valley Virtual Reality Conference where I provide some historical context for virtual reality tracing the evolution of communications mediums in order to contextualize my elemental theory of presence. It traces the history of the science and philosophy, and shows how VR is providing a direct experience of how much of our reality is subjectively constructed with our entire bodies. Hopefully this talk will help contextualize where we’ve been and where we’re going with VR, as well as a holistic framework to doing experiential design.

LISTEN TO THE VOICES OF VR PODCAST

Here’s the video of the talk:

And here’s the slides from my presentation:

Here’s the link to this presentation on Facebook if you’d like to share it there.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

nate-mitchellI had a chance to catch up with Oculus’ Nate Mitchell at GDC where I asked him about privacy in VR. Oculus has delegated the design and maintenance of their privacy policy to their parent company of Facebook so that Oculus can focus on providing the best VR experiences and growing the VR ecosystem. He acknowledges that there are “a lot of potential pitfalls over the future of VR and AR around user privacy” because VR has a “double-edged sword” of providing incredibly compelling immersive experiences, but that “used in the wrong way or in the wrong hands, you can be tracked probably more than you would normally expect to be.”

I learned more about the relationship dynamic between Oculus and Facebook in that Oculus isn’t thinking too much about how to use the data gathered from VR for advertising purposes, but the language in Oculus’ privacy policy is being shaped and directed by Facebook who is much more interested in using data gathered from virtual reality for advertising purposes. Mitchell claims that privacy is a top priority for Oculus, but a close reading of their privacy policy indicates it serves the needs of Facebook over consumers.

Mitchell and I also talked about Oculus’ announcement of lowering the price of the Rift + Touch by $200, their twelve new games premiering at GDC, as well as a number of important issues concerning the future of virtual reality. There are a lot of exciting new possibilities that could come from Oculus’ support for WebVR and the Khronos Group’s OpenXR initiative, but we also had a chance to talk about some of the challenges that Oculus has faced this year including some of their tracking regressions and some of the limitations of front-facing camera set ups when it comes to abstractions of embodiment.

LISTEN TO THE VOICES OF VR PODCAST

There are a lot of complicated issues surrounding privacy in VR, and Oculus has delegated the design and maintenance of their privacy policy to their parent company of Facebook. In Oculus’ letter to Al Franken, they say, “We also take advantage of Facebook’s expertise in other areas, including its large team of privacy and security professionals to help design and maintain privacy and security in our products. These collaborations allow Oculus to focus on what we do best: delivering the absolute best VR products and experiences.”

When I asked Mitchell about Oculus’ stance on privacy in VR. He said, “We are committed to really protecting user privacy. That’s one of our #1 focuses, which is why we have a super detailed privacy policy. And it goes hand-in-hand with that we are committed to being really transparent with users about what generally is being collected, and anything we’re doing with that. So that’s part of the reason why I think we have such a rich privacy policy to begin with. Also being part of Facebook, obviously, helps with that. They have an incredible team dedicated to user privacy, and they’re on the bleeding edge of that. And so that’s been great for us.”

I have to disagree in Mitchell’s assessment that privacy has been one of Oculus’ top priorities. Oculus’ top priority has been to deliver amazing VR experiences, and having a “rich privacy policy” that specifies everything that can be captured and recorded just means that it reflects the values and interests of Facebook. Facebook wants to collect and store as much data as they can, and tie back to a singular identity so they can sell advertising. On January 11, I sent an email to privacy@oculus.com to “access data associated” with my account, but I never heard anything back from them after two and a half months. If it really was a top priority for Oculus, then I would have expected to have received a response, and that there would be more systems in place for the type of transparency and accountability that is promised within the “Data Access and Deletion” section of their privacy policy.

Oculus is mostly taking a passive approach to privacy in VR where they’re prioritizing the needs and concerns of Facebook, which is reflected in how much data sharing rights are being provided to Facebook. The following is a sampling of data that when combined together could allow Facebook to determine personal identifiable information about you including your IP address, certain device identifiers that may be unique to your device, your mobile “device’s precise location, which is derived from sources such as the device’s GPS signal and information about nearby WiFi networks and cell towers,” “information about your physical movements,” and “information about your interactions with our Services.” Facebook will know that it’s your VR headset, where you’re located, and different actions that you’re taking from capturing everything you’re doing in VR and correlating it with your identity even if you’re anonymously interacting within the context of a VR experience. Once eye tracking and other technologies that can determine facial expressions are added, then there will be even more biometric data that could be able definitively identify you or whomever is using your VR headset.

Their privacy policy contains an open-ended statement about recording communications that could potentially allow Facebook to record and store all VoIP conversations: “When you post, share or communicate with other Oculus users on our Services, we receive and store those communications and information associated with them, such as the date a post was created.” Oculus denies in their letter to Al Franken that they’re recording conversations by saying, “VoIP communications are not being recorded. We do not store the content of these communications beyond the temporary caching necessary to deliver these communications to people who could be in different parts of the world.” But it’s unclear as to whether or not the privacy policy as it’s written would prevent Facebook from starting to record conversations at any time.

There have been a number of previous denials from Oculus saying that they’re not sharing data with Facebook yet, but there is actually nothing in the privacy policy that prevents this sharing from happening. For example, in Oculus’ response to Al Franken’s question as to whether Oculus is sharing information with third parties including it’s related companies they said, “Oculus does not currently share location information with third parties or related companies.” Their privacy policy certainly allows this sharing to happen at any moment, and so Oculus is basically just saying that we’re not sharing this data yet.

In response to data collection privacy concerns last year Oculus said, “Facebook owns Oculus and helps run some Oculus services, such as elements of our infrastructure, but we’re not sharing information with Facebook at this time. We don’t have advertising yet and Facebook is not using Oculus data for advertising – though these are things we may consider in the future.” Again, Oculus is diverting attention from what their privacy policy already allows by emphasizing that they’re not exercising their rights yet.

It’s almost as if Oculus is using their perceived operational independence from Facebook as a compartmentalized buffer to divert any focus on what their privacy policy is already enabling. Making statements that access to VR data streams haven’t been turned on yet do not carry much legal weight when there’s absolutely nothing stopping them from being turned on at any moment.

For example, Oculus’ privacy policy says “When you post, share or communicate with other Oculus users on our Services, we receive and store those communications.” Oculus responded to Franken that “VoIP communications are not being recorded.” But the real question is does Oculus’ privacy policy enable Facebook to start recording VoIP at any moment? Does Facebook/Oculus mean “we’re not recording VoIP yet“? Or do they mean “we never intend on recording VoIP because we would never do that?” They did not make a strong statement that they would never record VoIP, and so I have to assume that any time that I communicate with anyone on Oculus’ services that this data could be captured, stored, transcribed, shared with Facebook, tied to my personal identity, combined with information from commercial third parties in order to create a Facebook’s super profile to sell me ads either on Facebook or eventually on Oculus’ services.

In a candid moment, Mitchell said to me, “There are a lot of potential pitfalls over the future of VR and AR around user privacy. There’s never been a technology that brings so much of you into the experience, which is sort of that double-edged sword that’s the power of VR. But yeah, used in the wrong way or in the wrong hands, you can be tracked probably more than you would normally expect to be. Right? And I think that that’s only going to become more and more important as we develop new technologies that bring even more of you into the experience. And users are going to want to know and understand what’s actually happening under the hood.”

The problem with Oculus’ privacy policy is that it already provides Facebook a lot of leverage to capture and track a lot of information about you “probably more than you would normally expect to be” from just these two provisions of “information about your physical movements” as well as “information about your interactions with our Services, like information about the games, content, apps or other experiences you interact with, and information collected in or through cookies, local storage, pixels, and similar technologies.” This could already include head gaze, what you’re looking at, what you’re interacting with, and what interests you. These data streams could already be recorded and be sent to Facebook.

Oculus says that they’re using 60-second averages of physical movement data to debug their tracking. Mitchell said, “Almost all any of the live tracking we’re doing, almost all of it, is all really diagnostics focused. So if there’s a problem with your hardware, like a batch of hardware for example, we want to know that so that we can deliver a high-quality experience, and make sure that if there’s an issue with your system and reach into support, you can send us logs. And we can say, “Hey, clearly there’s a problem the Rift sensor” or something like that.”

Oculus is clearly using this data to debug and improve their technology, but it’s unclear whether Facebook could use this “physical movements” provision in order to record all sorts of eye movements, facial movements, and potentially more biometric data in the future. It’s a vague enough provision to potentially allow Facebook to capture a whole range of biometric data including eye tracking, galvanic skin response, heart rate and heart rate variability with ECG, muscle tension & facial expressions with EMG, and brain waves with EEG. This type of biometric data is usually gathered within a medical context protected by HIPAA or a marketing research context with explicit consent and privacy protections.

It’s also problematic that Oculus’ privacy policy is recording all of this data, tying it back to your personal identity, and storing it forever. The third-party doctrine is a legal theory that says that any data that you give to a third party “does not have any reasonable expectation of privacy.” This means that the government can request access to any data that you provide to any third party without a search warrant or probably cause. So the more biometric data that Facebook is collecting on us and storing forever, the less likely it is that we can have any Fourth Amendment privacy protections over any of this data. Facebook will know what you’re looking at and how you’re emotionally reacting to it, and there’s nothing stopping an abusive government from getting access to this same level of intimate data.

There are huge privacy implications that are coming with the technological roadmap of VR, and Facebook is sort of using Oculus as a technological shield to be able to develop this technology independent of the deeper advertising implications of the data that is going to be made available. When I asked Mitchell if the business models need to evolve beyond this type of privatized surveillance, he said that these types of new models are not something that Oculus is thinking extensively about right now. They’re mostly focusing on getting as many people in VR as possible. Oculus is working on the low-level implementation of VR while Facebook can think about what they’ll be able to do with all of this data.

In wrapping up his thoughts on privacy, Mitchell said, “So in summary: Very committed to user privacy. It’s something we take very seriously. It’s something we’re really focused on. We’re committed to taking care of user’s privacy. And you’re asking the right questions, keep asking them. I think right now, everything is in a good place across the industry. But that could change, and that’s something for folks like you to keep chatting about.”

Indeed this is something that the entire VR community needs to keep talking about, and it will change towards a direction that’s not a good place unless some of the deeper open questions listed down below are addressed. I’d also recommend listening to these interviews below about privacy in VR for more in-depth discussions.

Overall, in my assessment, Oculus has delegated privacy considerations to Facebook and it is clearly not a priority for them, despite Mitchell’s claims. If you have any questions regarding Oculus’ privacy policy, then I’d encourage you to follow up with Oculus via the email privacy@oculus.com. I haven’t personally received a response yet, but it’s a way to provide some direct feedback to Oculus. Hopefully they can start to implement more processes for transparency and accountability, as well as engage in deeper and more involved questions about the future of what will and will not be recorded when you’re within VR.

Other recommended interviews about Privacy in VR:

Here are some of the open questions that should be asked of virtual reality hardware and software developers:

  • What information is being tracked, recorded, and permanently stored from VR technologies?
  • How will Privacy Policies be updated to account for Biometric Data?
  • Do we need to evolve the business models in order to sustain VR content creation in the long-term?
  • If not then what are the tradeoffs of privacy in using the existing ad-based revenue streams that are based upon a system of privatized surveillance that we’ve consented to over time?
  • Should biometric data should be classified as medical information and protected under HIPAA protections?
  • What is a conceptual framework for what data should be private and what should be public?
  • What type of transparency and controls should users expect from companies?
  • Should companies be getting explicit consent for the type of biometric data that they to capture, store, and tie back to our personal identities?
  • If companies are able to diagnose medical conditions from these new biometric indicators, then what is their ethical responsibility of reporting this users?
  • What is the potential for some of anonymized physical data to end up being personally identifiable using machine learning?
  • What controls will be made available for users to opt-out of being tracked?
  • What will be the safeguards in place to prevent the use of eye tracking cameras to personally identify people with biometric retina or iris scans?
  • Are any of our voice conversations are being recorded for social VR interactions?
  • Can VR companies ensure that there any private contexts in virtual reality where we are not being tracked and recorded? Or is recording everything the default?
  • What kind of safeguards can be imposed to limit the tying our virtual actions to our actual identity in order to preserve our Fourth Amendment rights?
  • How are VR application developers going to be educated and held accountable for their responsibilities of the types of sensitive personally identifiable information that could be recorded and stored within their experiences?

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

jonathan-pardoThe developers at Harmonix didn’t set out to create an open-ended music creativity tool with Rock Band VR, but once they realized how compelling it was to simulate the feeling of being a rock star on stage in VR, then they completely pivoted the production of their game. They discovered that it wasn’t compelling to force users to focus on any single gameplay mechanic, and so they focused on allowing users to look around to cultivate deep feelings of immersion and stage presence.

Rather than rewarding precision of playing the perfect studio session, they wanted to recreate what it felt like to give a live musical performance that allowed users a lot of agency in expressing their own musical creativity and giving an entertaining embodied performance. I had a chance to catch up with Jonathan Pardo, QA Audio Lead for Rock Band VR, to talk about their design process of how they were able to incrementally teach players how to play their game as well as some of the fundamental components of music theory for what chord combinations tend to work well within their set of 60 different songs.

LISTEN TO THE VOICES OF VR PODCAST

One of the most impressive things about Rock Band VR is that they’ve actually recorded live guitar sounds of every chord type and rhythm with the proper guitar sounds for all 60 of their songs. This means that you can push a few buttons while strumming and have it sound like you’re playing the proper chords within the context of playing within a band. There are seven different chords with the following finger combinations: Single notes (1), Muted Power Chord (1,2), Bar Chord (1,2,3), Power Chord (1,3), Muted Arpeggio (1,2,4), Arpeggio (1,3,4), & Octave (1,4). This instructional video from Harmonix gives a great overview of the basic game play mechanic:

Harmonic realized that the most immersive experience would be to not have any artificial gameplay visualization, but this mode would be virtually impossible for anyone to organically figure out how to play the game if they weren’t a game developer or professional musician. So the created a Performance Mode with more instructions and guidelines. The Virtuoso Mode can be unlocked after a tutorial, and provides more explicit chord following to help train your muscle memory and understand some of what chord combinations work well together. Then the goal is to eventually be able to play the Monster Mode, which is the most immersive since there isn’t any gameplay visualizations happening but you have to know all of the music theory and listen to the band and follow along — much like a real musician would have to do in a live performance.

Fans of the original Rock Band will be happy to hear that they’ve also included a Classic Mode that can be played in VR:

Harmonix has a lot of musicians who have played a lot of live shows, and so they’ve created 15 different environments in order to simulate what it feels like to play on stage in front of a wide range of audience sizes and contexts. They change how it sounds in each environment and have different pedals to modulate the sound that are activated by looking at the pedal and using your whammy bar.

Even if you don’t want to learn and play all of the different chord combinations, Pardo says that there are other things that you can do in order to earn points by doing an embodied performance that cultivates your sense of stage presence that include things like turning the guitar upright, getting on your knees, doing a head bang, or jumping up and down.

Adding more social features is a top priority for Harmonix, but there are various network latency challenges that will make it difficult to synchronize live performances. Pardo says that they often will hold back on features if they know that it’ll make a worse networked gameplay experience, and that there are some features like live streaming of the audio that previous console versions of Rock Band disable due to sync problems due to latency. They’re actively exploring network workaround solutions as well as how to track other instruments, but these challenges explain why Rock Band VR is only launching with guitar support.

While Harmonix didn’t originally set out to create a live performance simulator emulating what it feels like to be a rock star, the unique affordances of VR slowly led them down this path. Pardo says they kind of accidentally created a music creativity tool, and that it was more about designing a game to be fully embodied and present in the moment on stage rather than the type of precision that you’d want if you were in the recording studio. The game play is not easy, but neither is being able to play music. Musicians should have an advantage in learning to play Rock Band VR, and it will be exciting to how games like Rock Band VR will help train and inspire gamers into learning how to play actual instruments.

You can learn more about the release of Rock Band VR and the compatible guitar controllers in this Oculus blog post.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

chris-berkaAdvanced Brain Monitoring is a 17-year old neurotechnology company that has been able to extract a lot of really useful information from EEG data. They’ve developed specific EEG Metrics for drowsiness, inducing flow states, engagement, stress, emotion, and empathy as well as biomarkers for different types of cognitive impairment. They’ve also developed a brain-computer interface that can be integrated with a VR headset, which has allowed them to do a couple of VR medical applications for PTSD exposure therapy as well as some experimental VR treatments for neurodegenerative diseases like Dementia.

I had a chance to catch up with Advanced Brain Monitoring’s CEO and co-founder Chris Berka at the Experiential Technology conference where we talked about their different neurotechnology applications ranging from medical treatments, cognitive enhancement, accelerated learning, and performance training processes that guide athletes into optimal physiological flow states.

LISTEN TO THE VOICES OF VR PODCAST

Advanced Brain Monitoring operates within the context of a medical application with an institutional review board and HIPAA-mandated privacy protocols, and so we also talked about the ethical implications of capturing and storing EEG data within a consumer context. She says, “That’s a huge challenge, and I don’t think that all of the relevant players and stakeholders have completely thought through that issue.”

They’ve developed a portfolio of biomarkers for neurodegenerative diseases including Alzheimer’s Disease, Huntington’s Disease, Mild Cognitive Impairment, Frontal Temporal Dementia, Lewy Body Dementia, Parkinison’s Disease. They’ve shown that it’s possible to detect a number of medical conditions based upon EEG data, which raises additional ethical questions for any future consumer-based VR company who records and stores EEG data. What is their disclosure or privacy-protection obligations if they are able to potentially detect a number of different medical conditions before you’re aware of them?

The convergence of EEG and VR is still in the DIY and experimental phases with custom integrated B2B solutions that coming soon from companies like Mindmaze, but it’s still pretty early for consumer-based applications for EEG and VR. Any integration would have to require piecing together hardware options from companies like Advanced Brain Monitoring or the OpenBCI project, but then you’d also likely need to roll your own custom applications. There are a lot of exciting biofeedback-driven mindfulness applications or accelerated learning and training applications that will start to become more available, but that some of the first EEG and VR integrations will likely be within the context of medical applications like neurorehabilitation, exposure therapy, and potential treatments for neurodegenerative diseases.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

John-BurkhardtI recently attended the Experiential Technology Conference where there were a lot of companies looking at how to use biometric data to get insights into health & wellness, education for cognitive enhancement, and market research. Over the next couple of years, virtual reality platforms will be integrating more and more of these biometric data streams, and I wanted to learn about what kind of insights can be extrapolated from these data streams. I talked with behavioral neuroscientist John Burkhardt from iMotions, which is one of the leading biometric platforms as a service, about what metrics that can capture from eye tracking, facial tracking, galvanic skin response, EEG, EMG, and ECG.

LISTEN TO THE VOICES OF VR PODCAST

I also talked to Burkhardt about some of the ethical implications of integrating these biometric data streams within an entertainment and consumer VR context. He says that the fields of advertising and brain washing often borrow from each other’s research, and he is specifically concerned about whether or not it’ll be possible to hack our fixed action patterns, which are essentially stimulus response behaviors that could be operating below our conscious awareness. Most of the work that iMotions does is within the context of controlled research and explicit consent of participants, but what happens when entire virtual environments can be controlled and manipulated by companies who know more about your unconscious behaviors than you do?

Burkhardt says that there is a behavior that he would consider to be unethical for how this biometric data are captured and used, but the problem is that no one really knows where that threshold lies. We might be able recognize it after it’s already crossed, but it’s hard to predict what that looks like or when it might happen. We’re not there yet, but the potential is clearly there. An open question is whether or not the VR community is going to take a reactive or proactive approach to it.

Burkhardt also says that these types of issues tend to be resolved by implicit collective consensus in the sense that we’re already tolerating a lot of the cost/benefit tradeoffs of using modern technology. He says that it’s just a matter of time before someone creates a way to formulate a unique biometric fingerprint based upon aggregating these different data streams, and it’s an open question as to who should own and control that key. The insights from biometric data streams could also evolve to the point where big data companies who may be capturing it could be able predict our behavior, but potentially even be able to directly manipulate and control it. He also says that it raises deeper philosophical questions like if someone can take away our free will with the right stimuli, then do we even have it to begin with?

As I covered in my previous podcast with Jim Preston, privacy in VR has many utopian or dystopian outcomes, but it’s likely to fall somewhere in between of being complicated and complex. There are lots of potential of new forms of self awareness of being able to observe our autonomic and unconscious internal states of being as well as changing the depth and fidelity of social interactions. But there also risks for this type of data being used to shape and control our behaviors in ways that cross an ethical threshold. It’s something that no individual person or company can figure out, but is something that is going to require the entire VR community.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

jim-PrestonWhen I was at the GDC VR Mixer, Jim Preston struck up a conversation about his concerns about privacy in VR. He works at the VR eye tracking company of FOVE, but wanted to speak to me on his own behalf about some of the deeper philosophical questions and conceptual frameworks around the types of intimate data that will become available to VR headsets. As more and more biometric data streams are integrated into VR, then there a lot of complicated and complex ethical questions that he thinks will take the entire VR community to figure out.

LISTEN TO THE VOICES OF VR PODCAST

Preston says that VR is challenging the long-standing enlightenment model of mind-body dualism, and that VR is able to do a sort of “redirected thinking” in being able completely control all aspects of someone’s else’s reality. This is a lot of power to put into performance-based marketing companies who have an extraordinary amount of data about our private lives, and he has concerns that this data could start to be used to drive consumer behaviors in unconscious ways.

The technological roadmap for VR includes integrations with new biometric data streams including eye tracking, facial tracking, galvanic skin response, emotional states, our voices interactions, and eventually EEG brainwave data. This data has typically had tight privacy controls either within the context of medical applications or market research that requires explicit consent, but it’s being captured within the context of an attention-driven consumer market where there many other vectors of private data that have been collected and connected to your personal identity.

Here are some of open questions around the future of privacy in VR:

  • Do we need to evolve the business models in order to sustain VR content creation in the long-term?
  • If not then what are the tradeoffs of privacy in using the existing ad-based revenue streams that are based upon a system of privatized surveillance that we’ve consented to over time?
  • Should biometric data should be classified as medical information and protected under HIPAA protections?
  • What is a conceptual framework for what data should be private and what should be public?
  • What type of transparency and controls should users expect from companies?
  • Should companies be getting explicit consent for the type of biometric data that they to capture, store, and tie back to our personal identities?
  • If companies are able to diagnose medical conditions from these new biometric indicators, then what is their ethical responsibility of reporting this users?

Preston has a nuanced view of what VR is going to enable in that he thinks that it’s not going to be either a total dystopian or utopian future, but that our future is going to be complicated and complex. Much like centaur chess teams of humans and AI are able to beat any other AI program, then this type of co-operation between humans and machines are going to enable all sorts of new amazing capabilities while also introducing new challenging problems.

The future integration is biometric data into immersive technologies have an array of complicated and complex questions that go beyond what any single company or individual can figure out, but Preston says that this is something that the VR community as a collective should talk about and attempt to answer some of these open questions.

I’ll be featuring some more information from biometric experts from the Experiential Technology Conference on the Voices of VR podcast as well as an interview with Oculus’ Nate Mitchell. For my previous coverage on privacy in VR, then be sure to not miss Sarah Downey’s take on privacy in VR and the relationship between the 1st and 4th Amendment, as well as Tobii Eye tracking’s recommendation for explicit consent for recording eye tracking data, HTC’s Dan O’Brien, these two interviews with Google with some open questions about Google Earth VR & Tilt Brush, as well as my interview with Linden Lab’s Ebbe Altberg.


Support Voices of VR

Music: Fatality & Summer Trip


Support Voices of VR

Music: Fatality & Summer Trip

I believe that the principle of Embodied Cognition is probably one of the most significant and important concepts to understand about virtual reality. Cognitive science researchers have been connecting the dots the importance of our bodies when it comes to perception, the subjective construction of reality, and how we process and think about information. We use our entire body and surrounding environment in our cognitive processeses, and virtual reality is bringing our full bodies into computing in a way that takes full advantage of the insights coming from embodied cognition research.

EleVR is a VR research collective that has declared 2017 as the “Year of the Body.” “Mathemusician” and virtual reality philosopher Vi Hart was a self-proclaimed body skeptic seeing it as an inconvenience to take care of in the persuit of higher forms of beauty with math and music, but after some preliminary experiments with embodied visualizations of physics she started to have a direct experience of the power of Embodied Knowledge.

I had a chance to catch up with EleVR’s Vi Hart and M Eifler to hear about their VR experiments and research into embodied cognition from creating interactive math museums built around 3D Venn Diagrams, visualizing hyperbolic space, and exploring the boundaries of container schemas and metaphors for understanding the concept of home and a place to rest.

LISTEN TO THE VOICES OF VR PODCAST

Venn Diagram Museum

Hyperbolic Space in VR

Real Virtual Physics

Check out my previous episodes about Embodied Cognition:


Support Voices of VR

Music: Fatality & Summer Trip


Support Voices of VR

Music: Fatality & Summer Trip

Johan-HellqvistThe eye tracking company Tobii had some VR demos that they were showing on the GDC Expo Hall floor as well as within Valve’s booth. They were primarily focusing on the new user interaction paradigms that are made available by using eye gazing to select specific objects, direct action, but also locomotion determined by eye gaze. I had a chance to catch up with Johan Hellqvist, VP products and integrations at Tobii, where we discussed some of the eye tracking applications being demoed. We also had a deeper discussion about what type of eye tracking data should be recorded and the consent that application developers should secure before capturing and storing it.

LISTEN TO THE VOICES OF VR PODCAST

One potential application that Hellqvist suggested was amplifying someone’s eye dilation in a social VR context as a way of broadcasting engagement and interest. He said that there isn’t explicit science to connect dilation with someone’s feelings, but this example brought up an interesting point about what type of data from an eye tracker should or should not be shared or recorded.

Hellqvist says that from Tobii’s perspective that application developers should get explicit consent about any type of eye tracking data that they want to capture and store. He says, “From Tobii’s side, we should be really, really cautious about using eye tracking data to spread around. We separate using eye tracking data for interaction… it’s important for the user to know that’s just being consumed in the device and it’s not being sent [and stored]. But if they want to send it, then there should be user acceptance.”

Hellqvist says our eye gaze is semi-conscious data that we have limited control over, and that this is something that will ultimately be up to each application developer as to what to do with that data. Tobii has a separate part of their business that does market research with eye tracking data, but he cautions that using eye tracking within consumer applications is a completely different context than market research that should require explicit consent.

Hellqvist says, “It’s important to realize that when you do consumer equipment and consumer programs that the consumer knows that his or her gaze information is kept under control. So we really want from Tobii’s side, if you use the gaze for interaction then you don’t need the user’s approval, but then it needs to be kept on the device so it’s not getting sent away. But it should be possible that if the user wants to use their data for more things, then that’s something that Tobii is working on in parallel.”

Tobii will be actively working with the OpenXR standardization initiative to see if it makes sense to put some of these user consent flags within the OpenXR API. In talking with other representatives from OpenXR about privacy I got the sense that the OpenXR APIs will be a lot lower level than these types of application-specific requirements. So we’ll have to wait for OpenXR’s next update in the next 6-12 months as to whether or not Tobii was able to formalize any type of privacy protocols and controls within the OpenXR standard.

Overall, Tobii’s and SMI VR demos that I saw at GDC proved to me that there are a lot of really compelling social presence, user interface, and rendering applications of eye tracking. However, there are still a lot of open questions around the intimate data that will be available to application developers and the privacy and consent protocols that will inform users and provide them with some level of transparency and control. It’s an important topic, and I’m glad that Tobii is leading an effort to bring some more awareness to this issue within the OpenXR standardization process.

Subscribe on iTunes

Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip


Support Voices of VR

Music: Fatality & Summer Trip