Holographic telepresence and eye contact.

A screenshot of me moving a virtual camera around
A screenshot of me moving a virtual camera around

Eye contact is one of those things we often take for granted in face-to-face conversations, unless it goes wrong. We sorely miss it in telepresence, like video chat: Zoom, Skype, Teams and more.

It’s been twelve years since I built one unusually impactful prototype around this issue. It used a pre-release Kinect, about 12 months before the consumer device launched. I had just seen someone in the XBox R&D group use a Kinect to make a live 3D point-cloud. I got excited about the potential for 3D video.

I’d previously used a technique called “Parallax Occlusion Mapping” in OpenGL to…

The “Top 100” (or whatever) original voices you should listen to

This list is inspired by the people who bring fresh, original thinking to the field of Spatial Computing aka XR/AR/VR/MR. They may not market themselves as “influencers” or “visionaries” or even count their followings. You may not yet know their names. But when they speak, we should always listen, because they’re likely sharing something important that we haven’t heard before.

And to the extent we have heard something before, there’s a decent chance it came from one such original, then got digested, emulsified and regurgitated for larger audiences, too often without credit. …

VR enthusiasts got upset at Facebook’s recent announcement that their headsets will begin to test ads. One VR developer who had originally signed up to include ads in their game reversed their decision due to the controversy.

VR enthusiasts were also shocked last year when Facebook announced their consumer headsets will require real-name Facebook accounts, or risk a ban.

These moves were all expected. Let’s stop being surprised by Facebook’s ad-driven business model and try to understand it better. Like all business models, it selects for and rewards decisions that increase revenues. In this case, employees and features that increase…

Facebook says they have 10,000 employees working on XR, or about 1/6 of the company’s 58,604 employees.

I’ll do some simple math based on public information. I’ll include my notes in brackets if you want to try other variations or tell me where I’m wrong.

Using their public projections from Q1 2021, we might reasonably expect they spend about $6 billion on XR this year for salaries, hardware and overhead.

[ assuming 1/6 * 4 * (Q1’s “cost of revenue” + “R&D”) = $6B ]

Compensation alone for 10,000 mostly Silicon Valley employees could exceed $5B a year, depending on…

I’ve worked for some really big companies to help define their AR & VR programs early on. Skepticism of powerful players is always wise.

But it really makes a difference when a company’s core incentives, their income and KPI, are aligned with the needs of consumers, long-term.

I avoid companies whose business models serve other masters. Even assuming best intentions, as Upton Sinclair said: “It’s difficult to get a person to understand something when their salary depends on them not understanding it.”

Ad-driven companies work, figuratively, like Casinos, although ad buys are really betting on us. …

When I raise concerns about ad-driven business models, most people shrug. Some may even get defensive. Ads pay a lot of salaries. They pay for so much of the online world that we consider essential. So why kick the golden goose?

Hear me out. Ads themselves are mostly harmless. We might complain about some overly loud videos for medicines we don’t need, or when ad-laden webpages jump around like monkeys on cocaine. We might vaguely suspect our cell phones spy on our conversations (they rarely do).

If you’re visiting a giant free buffet every day, you probably aren’t going to…

[Neuroscientists: feel free to correct anything here with better information for non-scientists to learn. Others: please read this as a primer.]

Some people soundly reject the premise that eye-tracking is somehow better (today) than other forms of technological mind-reading. I can understand this reaction. Eye tracking (ET) is not literally reading minds — neurons to be precise — where the best science fiction imagines lifting our thoughts directly.

We could argue that our eyes are only the place where our central nervous systems (CNS) are directly exposed to the world. The eyes are at the very least a window into…

A slurry of recent articles assert that Facebook increases polarization and perhap radicalization among its users. Facebook is circling the wagons to counter this narrative, even asking its employees to help push back. After all, if they claim to be the company that brings people together, it would be unfortunate to be widely seen as having the opposite effect.

The first step to understand how Facebook may or may not polarize us is to look at how recommendation engines work. …

The Road to Individual Autonomy

You’re walking down the street. You notice random teenagers gawking at you and then shying away. And not just a few. What’s going on? You ask a savvy friend for help.

They eventually figure out what happened. When you use Trawler’s (the leading fictional social network) AR glasses and add the “BodyNet” app, your clothing — as far as you can tell, just yours — becomes transparent to the person wearing the glasses. It’s like x-ray vision.

Nude. You. What?!

You don’t even have an account with Trawler, let alone your own AR glasses. …

About: This article was originally published on Motherboard (aka vice.com) in May 2019. This version contains new information and graphics.

Avi Bar-Zeev spent the last 30 years working on AR/VR/MR, helping companies like Disney, Microsoft, Amazon, and Apple make decisions about how to unlock the positive potential of new technologies while minimizing harm. He helped start the original HoloLens project at Microsoft. Nothing in this article should be read as disclosing unannounced products or ideas from any company.

Humans have evolved to read emotions and intentions in large part through our eyes. Modern eye-tracking technology can go further, promising new…

Avi Bar-Zeev

Design and Technology Leader (fmr. HoloLens, Apple, Google Earth, Second Life, Disney VR)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store