For XR, the Eyes are the Prize

About: This article was originally published on Motherboard (aka vice.com) in May 2019. This version contains new information and graphics.

Avi Bar-Zeev spent the last 30 years working on AR/VR/MR, helping companies like Disney, Microsoft, Amazon, and Apple make decisions about how to unlock the positive potential of new technologies while minimizing harm. He helped start the original HoloLens project at Microsoft. Nothing in this article should be read as disclosing unannounced products or ideas from any company.

Humans have evolved to read emotions and intentions in large part through our eyes. Modern eye-tracking technology can go further, promising new wonders in human-computer interaction. But this technology also increases the danger to your privacy, civil liberties, and free agency.

Bundled into VR headsets or AR glasses, eye-tracking will, in the near-future, enable companies to collect your intimate and unconscious responses to real-world cues and those they design. Those insights can be used entirely for your benefit. But they will also be seen as priceless inputs for ad-driven businesses, which will learn, model, predict and manipulate your behavior far beyond anything we’ve seen to date.

I’ve been working to advance Spatial Computing (also known as AR, VR, MR, XR) for close to 30 years. Back in 2010, I was lucky enough to help start the HoloLens project at Microsoft. The very first specs for that AR headset included eye-tracking. My personal motivation was a vision for the next great operating system and user interface, one that dispenses with “windows” and “mice,” to let humans interact naturally in our 3D world.

I am happy to see some form of these ideas included in the HoloLens v2, and also Magic Leap and Varjo headsets, to name a few. I’ve seen articles hailing the coming cornucopia of eye-tracked user-data for business. The gold rush to your eyes is just beginning. But there’s an important part of the conversation that is still largely absent from public view:

How can we foster the best uses while minimizing harm?

Now is the time to begin building awareness and expectations. Eventually, I anticipate companies will have to concede to a government somewhere in the world (most likely in Europe) that eye data is like other health or biometric data, meaning it must be secured and protected as least as well as your medical records and…