For XR, the Eyes are the Prize

About: This article was originally published on Motherboard (aka vice.com) in May 2019. This version contains new information and graphics.

Avi Bar-Zeev spent the last 30 years working on AR/VR/MR, helping companies like Disney, Microsoft, Amazon, and Apple make decisions about how to unlock the positive potential of new technologies while minimizing harm. He helped start the original HoloLens project at Microsoft. Nothing in this article should be read as disclosing unannounced products or ideas from any company.

Humans have evolved to read emotions and intentions in large part through our eyes. Modern eye-tracking technology can go further, promising new wonders in human-computer interaction. But this technology also increases the danger to your privacy, civil liberties, and free agency.

Bundled into VR headsets or AR glasses, eye-tracking will, in the near-future, enable companies to collect your intimate and unconscious responses to real-world cues and those they design. Those insights can be used entirely for your benefit. But they will also be seen as priceless inputs for ad-driven businesses, which will learn, model, predict and manipulate your behavior far beyond anything we’ve seen to date.

I’ve been working to advance Spatial Computing (also known as AR, VR, MR, XR) for close to 30 years. Back in 2010, I was lucky enough to help start the HoloLens project at Microsoft. The very first specs for that AR headset included eye-tracking. My personal motivation was a vision for…