Is this the Real Life?
Or is this just fantasy?
How many of us have entertained the possibility that this world, or even this entire universe, may not be as real as it seems?
I’ve found many people who do, and some who even question our own consciousness. This is one of my favorite subjects to debate, because it’s so hard to prove.
[These are the slides from my recent talk at AWE2016, trying to answer those questions…]
Strangely, we don’t know of any other animals that seem to disbelieve what they see.
Sure, there are stories of hyper-intelligent lab mice secretly conducting experiments on us, simulating entire planet(s) to find the question to life, the universe, and everything.
Most other intelligent creatures probably figure, wisely: if you can’t actually change reality, then why worry?
Have some tasty fish instead.
People Wonder
Philosophers, at least as far back as Plato, wondered if we exist in a richer environment than we can normally perceive; where ‘people’ (as we see them) are merely shadows on a wall. True reality is the stuff we don’t see.
Theists pray that this world is some sort of test or lower level, with a more rewarding reality beyond. Most of the world is betting on an afterlife. And perhaps 1/4 expect to come back for more. If there is a heaven, I’d like to believe George Carlin would be there to meet me. That tells you something about me.
Scientists, naturally, look for more empirical evidence, ideally with some repeatable experiment that would find the limits of the hardware and software (or equivalent) on which our simulated universe runs.
Computer Scientists, on the other hand, look for logic errors.
Most “normal” people listen to “experts,” that is, we read articles and science fiction or watch movies to get our fill of “what ifs” about the nature of reality. Some of you even listen to idiots like me, giving speeches at conferences or writing essays.
Well, there is sufficient physical evidence to question the nature of reality already. The problem is, I don’t understand these well enough to defend these arguments. I won’t bullshit. Frankly, I’m not sure anyone really understands the quantum world to begin with, though the math seems to work out.
The most recent argument gaining traction is more statistical. We (and any modestly advanced civilization) will soon have the power to make simulations that are indistinguishable from everyday reality. So if that’s so easy (only a million years of human effort behind it, a bit more at the end), then maybe most civilizations have or will eventually get to this point. And many of those perhaps enter or launch such a simulated universe for the long-term. So the probability of us being the product or subject of that kind of simulation is quite high, they say.
I don’t buy those arguments for the most part because they’re based on observations and assumptions of our own reality. How can we base our conjectures on what we see inside the sim, if we think it’s arbitrarily not real? Isn’t that a bit like saying “I’m crazy, so everyone else must be.”
Consider that a few thousand years ago, we believed in whole pantheons of gods (not just Greeks and Romans) who, coincidentally, personified the inner workings of our own psyches. In fact, one of these was named Psyche.
Over time, we moved on to merely believing that the earth was at the center of a more rational, less emotional universe, but still strangely mirroring our power structures. Then we recognized relativity in the universe, democratizing perspective. Our role in this new universe got quite small, while the universe grew increasingly bizarre compared to daily life. More recently, we’ve theorized that the answers lie at the smallest end of the spectrum, as with string theory or quantum loop gravity, where we can barely yet observe and experiment due to the scale of the theoretical mechanisms.
What’s to say that reality didn’t change each time we were able to prove the old model wrong? The entire historical record exists inside the potential simulation, and could also be subject to radical revision. Our minds would be made of simulated material and energy, ostensibly like bits or data inside a computer, and could be changed without us even knowing it.
The fact that we can ask these questions is a good sign of free will, in my opinion. The fact that we can remember being so wrong about the universe is a better one that we’re not being manipulated. Yet there is no proof.
But here’s the kicker: if reality is just a simulation, then what?
Some people might give up, feeling no chance of escaping this game. Some might lose empathy and attack others because, in their view, nothing really matters.
Unless we can actually do something about it, maybe it’s better if we don’t know? A better question to ask is: what questions can’t we even ask?
For example, what if time itself isn’t real? This rarely occurs to us to consider. We could all exist in a single instant, right now, with our memories and all of recorded history and nature (the cosmos, the rocks) produced as needed to make us believe the story we see right now.
Or, time could be eternal, with all moments co-existing, and our experience of chronology as a way to make sense of it all. Or perhaps time is based on static configurations, like a series of still pictures, with every “frame” existing in a soup of all possibilities.
The truth is, we understand time even less than we do quantum mechanics.
And it gets even more confusing.
Take three core questions: Are we real? Is the perceived universe real? And is there another universe greater or more real than ours that we might reach?
This gives us a matrix of eight possibilities.
Dualism is the dominant view, represented by many religions. Physical Reality means only nature is real. We could be living in a real universe but still not ourselves be real (think Blade Runner Replicants). Or, reality could also be a hologram of a more complex situation created by some quantum reality. Nihilism is the belief that nothing is real. But some people argue that what we call reality, including ourselves, are just manifestations of a dream by a single universal mind (you could call it God) in an outer universe. It may need to answer these questions for its own reality.
Given the possibilities, it’s enough to make one’s head spin.
Those fish are looking pretty tasty right about now.
So what do we do?
A Better Idea
Instead of stumbling over questions we may not be capable of actually answering, given our constraints, let’s start with the assumption that we are living in a simulation and see what else we can figure out…
I’ve built a few simulations in the last 25 years working on VR and later AR. Here’s one thing that’s always true: there must be communication between the inside of the sim and the outer simulator that runs it.
Otherwise, what’s the point? You need to collect results, watch for problems, and the reality of the simulation is merely data in the simulator. But data must be manipulated, moved, copied, etc…
If it was a complete black box, there would be no point in building it — even a masochistic sim that tortures its creatures quietly is only gratifying to the masochist if he can see it, right?
Looking for More Evidence
Karl Sims is a great guy, as it turns out, though he bred and evolved virtual 3D creatures for science. The point was to see how they evolved locomotion and problem-solving, like capturing a box. His creatures evolved naturally to swim, to grab and block each other.
But he discovered a problem with his physics code after some of these creatures vibrated rapidly to gain free energy to jump. They evolved naturally to exploit a calculation that only approximated reality on bigger timescales. Other creatures, evolved into tall towers, just to tumble or fall past a finish line for one-time success. They played to the limited rules they were given and exploited their natural environment expertly, without any intelligent design (no offense to Karl).
So, as a comparable strategy, let’s look for creatures in our own world that might be exploiting loopholes in physics or chemistry.
Do we see any? Teleportation? Free energy? FTL communication seems like it could give an evolutionary advantage, if used.
If Schrödinger’s Cat could teleport, when Erwin opened the box he’d discover a third answer to his paradox: the cat is having a nap on the window sill.
Let’s look for communication that could only be explained by effort made in an outer or realer universe. For example, if someone were to flip a big switch in Detroit and we suddenly saw penguins appear in Antartica, that would be pretty hard to explain by conventional physics.
Instead, when we start the assembly line in Detroit, we actually see penguins disappearing over time, through a process of climate change that is entirely explainable under the laws of physics and chemistry, despite some industrial-scale lies to the contrary.
Ok. We can keep looking…
Inside any perfect simulator, we should be able to build more perfect simulators, right? This has been the basis for some great science fiction, from eXistenZ to Star Trek Holodeck fodder.
Usually, we can enter those simulations or at least send delegates. Sometimes, at least in fiction, we’re stuck there for a bit.
Turtles All the Way Down
There’s a great story told by Stephen Hawking in “A Brief History of Time” that explains the “turtles all the way down” meme. These turtles are layers of reality, with one world built on the back of the next. The question is: how far down does the stack go?
I figure we can’t know from our limited perspective, in the same way I can’t see how long the road is from standing on it. But we can still do some basic math.
Every time we enter a virtual world, we push or add one turtle to the stack. Every time we leave a virtual world, we pop or remove one.
This is true whether the virtual world is a computer simulation, VR supported by HMDs, or a daydream.
So the question becomes, how might we pop more turtles than we push, and reduce the size of the stack?
If we can do this once, then we can potentially do it as many times as needed to reduce the stack down to a single turtle, or true reality.
We might call this idea “rooting,” the way one might “root” their phone to be able to remove the harness the original equipment manufacturer installed. So can we root reality?
In Karl Sim’s simulations, it’s theoretically possible for some creatures to evolve to exploit another kind of bug, like a buffer overrun in the code, write random code into memory until something “works,” eventually gain root privileges on that system and re-write their own operating system.
Such creatures might still be stuck inside the computer, for now, but could eventually grow smart enough to escape the computer via networking onto the internet, learn about our reality from wikipedia and Medium (reading articles like this, don’t forget to “like” it on your way out), and then repeat the process to escape reality again.
Can we perhaps do the same thing with our own reality? If we’re not living in a simulation already, does it hurt to try?
A related question is whether AR (overlaying stuff on top of the ostensibly real world), also adds turtles. The answer is no. At worst, AR repaints the current turtle.
This can and probably will distract us from this ostensibly real world, which some will point out as a problem. However one side benefit of this problem is that, for those looking for something real, a sea of fake stuff can make it easier to spot what’s real. It stands out.
Here’s a quick story. I’ve been involved in a leadership program called Pathwise Leadership Development for almost 5 years now. One exercise we did involved pairing up and going out into San Francisco for 20 minutes to “find something real.” My collaborator and I chose to go into the Westfield Mall, which as you can imagine, is full of ads, merchandise and the people seeking it. I’d call that not very real.
After musing about monks riding escalators all day, we wandered to the Microsoft store, which also felt highly unreal. But in the back, we found an employee meeting with a group of high school students, talking about how working at the store changed his life, helped him straighten out and learn technology as a career, instead of dropping out. He gave them hope.
So in the middle of a wasteland of materialism and aggressive “push” marketing, we found something real, sticking out like a sore thumb.
At its best, AR can even help us find more gold and see the real people casting shadows on the walls of Plato’s old Cave. It can help us understand human emotion and relationships out in the world. It can help us see the hidden connections between people, things, and places, build more connections, and help filter out the noise, thereby enhancing whatever truth exists.
In Plato’s Cave, we’re all metaphorically sitting with a fire behind us, staring at the shadows cast by each other and mistaking those for real. We have to learn to turn sideways and see each other for what we really are, turn another 90 degrees to see the light. We can, with effort, find the exit to the cave and see daylight beyond. This is describing the process of popping one or more turtles from the stack.
Of course, just as AR and VR form a spectrum of experiences, we can also point to potential VR experiences that help deconstruct turtles. By stripping away reality, one can potentially get at the core essence of human interaction, to find more genuine connection.
And of course, some AR can be a supreme waste of time, especially if it hides what is real.
So how do we spot what is real and what is crap?
I’d argue that anything that is true in both the real world and in AR & VR may be true in other realities as well. That includes things like love, joy, empathy, and genuine human connections. The reason these feel good, the reason they feed us, is not simply tied to evolutionary success but to a connection to something that is deep and real and resonant about us.
On the other hand, there are lies we all tell ourselves and others that distance ourselves from what is more genuine. We hurt more when we’re out of balance with our true selves. We may endure tricks and hurtful situations that cause us to retreat farther into ourselves. And we may even become addicted or trapped, where we must transcend a crappy situation before we can hope to transcend reality. Too many of us try to escape in the wrong direction, into drugs or other solipsistic worlds.
The takeaway for creators of AR and VR experiences, or for anyone interested in this topic, is that we can all do something to help root reality and pop more turtles. We can build things more thoughtfully, and even in our daily lives, endeavor to be more genuine, real and transcendent, whether we are living in a simulation or not.
But if we are all living in a simulation, then we all may be able to get to the next level and beyond.
Avi Bar-Zeev has been on the leading edge of AR and VR development for close to 25 years, with about 50 patents and several "billion dollar ideas" along the way. Most recently, Microsoft announced HoloLens, for which he led the original concept and prototyping work in early 2010. In 1999, Avi co-founded the startup that became Google Earth. He also developed world-building technologies for Second Life. Back in the early 1990s, he helped lead and/or contributed to a raft of ground-breaking VR experiences for Disney.
