What is ‘Mixed Reality’?

It depends on where you’re coming from…

--

To XR veterans, it was originally defined as “the whole spectrum from AR to VR,” not just something smack in the middle. But it’s been redefined by industry lately, and somewhat depends on which direction you’re coming from:

1) For those coming from VR development, MR means we’re adding in this magical new thing called “reality” (beyond simple guardian features) so that it doesn’t seem like we’re always somewhere else. All of a sudden we have our full bodies again, real clothes mostly, we know where the coffee table really is, and we can even type on keyboards. Yeah. It’s a revolution.

2) For those coming from AR (the smart-phone kind), their devices always used video pass-through, but now it’s on their heads instead of in their hands. So they’d just call this “hands-free AR” and wonder what the fuss is about.

3) For those coming from “true AR” (optical pass through, which ultimately leads to all-day glasses), it’s still just AR. Why did anyone need to rename it based on the use of cameras and displays vs. waveguides? Who cares what the display tech is as long as it works? And, in fact, the best AR experiences are the ones with a very light touch out in the world.

For everyone else (the 99%), MR is just a simpler way to explain what’s going on. “Oh, you’re mixing real and non-real stuff. Ok. I get it.”

This is probably why Microsoft was one of the first to apply the term MR to the HoloLens… which was optical pass-through… Oh so confusing. But at least it unified AR+VR devices. MR also won out over calling it “hallucinations” which is way worse. Now it’s about time for Microsoft to start all over again by calling this stuff “multi-modal AI.”

Alas, the word ‘Virtual’ still doesn’t mean much to folks, especially if it seems real enough to us. Whatever experience as real is real. Virtual is imaginary.

The word ‘Augmented’ can be misleading too, because it’s not Reality that is augmented, but rather our perception and interactions in the natural world. So that’s one reason people like me and Dr. Helen Papagiannis, Ph.D. like to use the term “augmented humanity” instead. But not enough of us do so yet.

So at the end of the day, we’re left with a few words that work mainstream:

“Mixed Reality” is about what’s displayed and interactable, bringing 3D imagery and other senses out into the natural world.

“Spatial Computing” is about the 3D interaction paradigm itself, bringing HCI off the screen and into the natural world with us.

--

--

Avi Bar-Zeev

XR Pioneer (30+ years), started/helped projects at Microsoft (HoloLens), Apple, Amazon, Keyhole (Google Earth), Linden Lab (Second Life), Disney (VR), XR Guild