25 years of Quantum Foundations

“It was about three o’clock at night when the final result of the calculation lay before me. At first, I was deeply shaken. I was so excited that I could not think of sleep. So I left the house and awaited the sunrise on the top of a rock.”

— Werner Heisenberg, describing his excitement after giving birth to the new theory of matrix mechanics (later known as quantum mechanics) on the North Sea island of Helgoland.

In the mid-1920s, humanity’s understanding of the universe was on shaky ground.

The picture of electrons orbiting neatly around the nucleus of an atom and emitting continuous energy had fallen away. Experiments had proven that atoms only emit light at specific discrete wavelengths.

Similarly, the entire concept of an absolute spacetime was gone. As explained in Einstein’s famous special relativity paper of 1905, the speed of light was constant, but time was relative to the observer’s motion.

Out of that turbulence that shook the entire scientific community, something revolutionary was born: Quantum theory and its formalized mathematical structure known as quantum mechanics. 

It wasn’t an easy birth. Quantum mechanics generated controversy. 

How can the wave function describing a particle be in a superposition of many possibilities (like a cat being both alive and dead) when you are not observing it, and then collapse into a definite observed state (a single outcome) just because we observe or measure it? (This is known as the measurement problem.) How can the fates of two particles be intertwined so that measuring a property (such as spin) for one particle is reflected in the property of its far-away partner (entanglement or what Einstein derided as “spooky-action-at-a-distance”)? Was everything really as probabilistic as quantum mechanics implied? Must the uncertainty principle  be accepted at face value? Does God really play dice, as Einstein famously put it?

There were many scientific and philosophical questions. What is the role of the observer in quantum experiments? Why is there a difference between the laws that govern a quantum particle like an atom, versus a soccer ball? Is there a boundary between the quantum world and the classical one? What does quantum theory actually say about the nature of reality? Should physicists just accept it (shut up and calculate, as one scientist famously put it), or try to interpret it, make sense of it? Is this all there is, or are there some hidden variables lurking beneath that we just don’t know about yet? 

These questions sparked vigorous debates in the mid-1920s, and for decades after.

Since then, countless experiments have proven quantum theory to be one of the most successful theories in the history of science. Superconductivity, lasers, semiconductors and quantum computers wouldn’t work at all if it weren’t for the quantum rules that underlie these technologies. 

Progress is being made at Perimeter

After 100 years, we are still struggling to make sense of it all. But scientists in the field of Quantum Foundations at Perimeter Institute feel they are making progress and getting closer to the answers. They are chipping away at it, trying to constrain what is or isn’t possible, each in his or her own way.

Quantum Foundations was one of the earliest fields of study to find a place at Perimeter Institute when it opened some 25 years ago. Early on, it drew Lucien Hardy, a Royal Society University Research Fellow at the University of Oxford known for his thought experiment, Hardy’s paradox. 

When Hardy was in high school in Britain, he had listened to a BBC radio program led by physicist Paul Davies that introduced him to luminaries in quantum physics —people like John Stewart Bell and Alain Aspect. These people were running experiments on entangled photons and ruling out many hidden variables. As strange as it seems, they proved that the underlying quantum world is truly different from our everyday intuitions. 

“I was hooked. I was already interested in physics, but they were asking intriguing questions of a kind that I had not heard before,” Hardy says of that radio program. It led to him studying the subject in university. 

In 2002, shortly after Perimeter Institute opened (initially in an old post office building that was nicknamed Spacetime Square in Waterloo), Hardy joined the faculty with excitement. Here was a brand-new institute devoted entirely to theoretical physics, and colleagues eager to discuss the deep questions in which he was interested. 

Over the years more researchers joined, such as Robert Spekkens and Elie Wolfe. Today, young postdoctoral researchers in the field include Nicholas Ormrod, David Schmid, Tein van der Lugt, Christopher Jackson and Roberto Dobal Baldijão as well as PhD students Yìlè Yīng, Marina Maciel Ansanelli, Daniel Centeno Díaz,  María Ciudad Alañón and Pedro Lauand who are part of the next generation of quantum foundation explorers. 

They all went into quantum foundations for similar reasons: They were intrigued by its counterintuitive nature and wanted answers. 

Wolfe, a research scientist at Perimeter, describes it as being a bit like a kid seeing a magic trick performed on a stage, and “wanting to peek behind the curtain to see how it is done.”

The tool for peeking behind the curtain is mathematics.

As Hardy puts it, “you’re trying to find some underlying mathematical structure, the thing that makes it tick.” 

Hardy thinks the answers lie at the intersection of quantum theory and Einstein’s gravity theory known as general relativity. Each of those fields contains an underlying mystery. In quantum theory, we have superposition, where particles can be in simultaneous possibilities at once, at least until an observer measures them. In Einstein’s gravity theory, spacetime is malleable. Clocks tick slower in high gravity regions, or on fast moving spaceships.

Quantum theory doesn’t align neatly with the current understanding of gravity. At Perimeter there is a whole research area, quantum gravity, devoted to meshing together these two great theories of science.

As Hardy points out, the Schrödinger equation that describes how quantum states evolve, can be traced forwards and backwards. It is “time symmetric.” Yet when the wave function collapses, there is an irreversible change — the arrow of time goes one way, and it can’t be reversed. Why? 

The answer might have to do with “mixed states” rather than “pure states,” where you may have incomplete information about the quantum system. 

“In any real application of quantum theory, like in quantum information theory, you ultimately end up with these mixed states rather than pure states, and there are deep reasons to think that this is important,” Hardy says.

The answers might lie in the nature of spacetime itself

Hardy thinks quantum theory might, like general relativity, be time malleable. He is currently writing a book to repackage quantum theory. He is taking an operationalist approach that considers the measurements of the quantum system to be the fundamental reality, independent of the observer.

Meanwhile, researcher Robert Spekkens is taking more of an epistemological approach, focusing on the knowledge of the agent or observer. 

Spekkens did his undergraduate degree in both philosophy and physics. After he got his PhD, he came to Perimeter, one of the few institutions hiring in the field of quantum foundations. Around 2003, Spekkens developed an idea that a quantum state actually represents an incomplete knowledge of reality rather than reality itself. 

Spekkens says an analogy is to think of it as two meteorologists giving you different probabilities about whether it will rain or not. One might say there is a 50 per cent chance of rain; another might say there is a 75 per cent chance. Neither is telling you about the actual rain event – what they are presenting you with is probabilities, based on incomplete information about whether it will rain or not.

One could also think of it as a who-done-it mystery story with two different investigators working under two different assumptions. “They are looking for clues in very different places,” Spekkens says.

An observer approach to quantum theory

That’s the spirit of Spekkens’ observer-approach to quantum theory. “Quantum theory, more than any other theory, involves the observer in a significant way, so you have to be sophisticated about the nature of causation. That is part of disentangling what is really going on, from what is merely known by observers,” Spekkens says. 

Spekkens developed a toy model that took this view of quantum mechanics. The model is based on a foundational principle that for every quantum system, what you know is equal to the knowledge that you lack.

His model can’t reproduce all of quantum theory, but within the bounds of this model, many phenomena typically associated with quantum theory — such as entanglement — are present. It implies that as observers, we simply can’t see the entire reality. A paper on this can be found in Physical Review A: Evidence for the epistemic view of quantum states: A toy theory

Spekkens’ thinking about probabilities and what-causes-what ultimately led him to become interested in causal inference around 2009. He now heads the quantum causal inference initiative at Perimeter.  

Causal inference at Perimeter

Causal inference is a statistical method used in areas like economics, or in disease modelling, to disentangle correlation from causation. A simple example is that just because the number of ice cream sales and the number of drownings at the beach are correlated, that doesn’t mean the ice cream sales caused the drownings. There are other factors, like hot weather, related to both, and you need to disentangle the various factors.

To Spekkens, the statistical approach in causal inference sounded a lot like Bell’s inequality tests, a milestone in theoretical physics. Prior to Bell, it was thought that maybe there were some “local hidden variables” (properties that are predetermined and influencing a particle’s behaviour in its immediate surroundings) that are lurking in the background and could explain entanglement. But Bell’s experiments proved that entangled particles had stronger statistical correlations than any local hidden variables could explain. In the years since, many similar experiments have been done and the conclusion holds up. 

That similarity between causal inference statistics and Bell’s inequality tests intrigued Spekkens. “I had to familiarize myself with what had been done in causal inference and think about quantum problems from a new perspective,” he says.

Spekkens thinks it’s important to consider the interpretations of quantum theory and not just take the approach of pragmatically accepting its predictive power and use it to create new technologies. “It matters what interpretation you have, because that sets up the kinds of questions you are going to ask,” Spekkens says.

Perimeter has, from the start, built upon the century long history of quantum theory. In October, Spekkens was a lead organizer of a conference: 100 Years of Quantum: Perspectives on its Past, Present, and Future. The lectures from that conference can be found in the Perimeter Institute Recorded Seminar Archive.

Today, Perimeter Institute researchers are taking that foundation and building in new directions. 

Wigner’s friend paradox

One of the ideas they are exploring is the Wigner’s friend paradox, which is a thought experiment that illustrates the measurement problem. One scientist, inside a lab, performs an experiment on particles that are in superposition until the measurement is taken. At the point of the measurement, it collapses into one definitive state, either zero or one. But for a friend outside the lab, the particles are still in superposition, at least until she learns of the measurement. Until they collaborate, the theory suggests that two contradictory realities seem to exist simultaneously. How can that be?

Perimeter postdoctoral researcher Nicholas Ormrod (along with Oxford colleague and Perimeter alum Johnathan Barrett) have been attempting to build a more coherent observer-independent interpretation of quantum theory.

As described in a recent New Scientist article, they are trying to blend a “consistent histories interpretation” of quantum mechanics (that focuses on the many ways that a quantum system might evolve over time) with relational quantum mechanics, which has been advanced by Carlo Rovelli, a  Distinguished Visiting Research Chair at Perimeter, taking the view that the properties of a quantum system can only exist in the interactions, or the relationships, with other quantum systems. 

Ormrod and his colleagues are trying to put some mathematical flesh on these ideas, proposing that reality could be made up of networks of causal bubbles. Ormrod described his ideas in a Perimeter Institute Recorded Seminar Archive video.  In the Ormrod/Barrett paper, Wigner and the friend are in different causal bubbles with spacetime itself emerging from these networks of causal order, just like the points of a spider web emerge from the intersecting threads.

Another Perimeter postdoctoral researcher and teaching fellow, David Schmid, became interested in quantum foundations by reading a lot of popular science that discussed the various counterintuitive results and debates that raged around quantum theory.

“To be honest, a lot of it sounded too far-fetched to me. I didn’t really believe it. I was skeptical and I thought, I want to study this, either to understand why people believe these things, or show that it is not a fair conclusion.”

He was drawn to Perimeter because of Spekkens’ work , especially the toy model that Spekkens had developed to include the state of the observer’s knowledge in quantum theory. His research currently involves constructing “no go theorems” that constrain what we know about the nature of reality.

After 100 years of quantum theory, it may seem to lay people outside the field that there has been little progress, or that a way forward is impossible.

However, most scientists involved in quantum foundations feel that there has been a lot of progress, especially as a result of the “no go” theorems that have come out of the Bell inequality experiments and the experiments around the Wigner’s friend paradox to deal with the measurement problem.

Elie Wolfe, who co-leads the quantum causal inference initiative at Perimeter, said those no-go theorems narrow the range of possibilities for what quantum theory means. “I think we have at least clarified what the questions are,” he says.

The future of quantum foundations

Schmid, as someone newer to the field, is hopeful. “We are narrowing down the weirdness of quantum theory,” he says. Many people who are outside the field of quantum foundations still think about the various features — such as entanglement, or the particle-wave duality, or teleportation, or superposition and the collapse of the wave function — as being profoundly strange. But to Schmid, these feature are less mysterious than they were 100 years ago. “I believe that will ultimately pay off and allow us to concentrate our efforts on those questions that still need an explanation.”

PhD student Yìlè Yīng was also drawn to Perimeter in part because of Spekkens’ ideas about the role of the observer in quantum theory. She was intrigued by that. “What does it say about my experience, my observation?”

She is hopeful that there will one day be more consensus. “Even if, in the end, there is something about it that somehow humans can never understand, we should try as hard as we can. We are far from reaching the limit of our knowledge, and progress in quantum foundations is being made every day,” she says.

There are a number of reasons why it is important, she adds.

“There is this tension between relativity theory and quantum theory, and I think it is important to resolve that tension.”

Also, as she says, we know that quantum mechanics is important in modern-day technologies, and it is important to understand it at a fundamental level in order to make future progress.

But more than that, there is the pure joy of it. The members of quantum foundations group at Perimeter are constantly involved in animated discussions about the most recent papers and ideas. “We have lunch together, we brainstorm together,” she says.

Just like Werner Heisenberg sitting on a rock on Helgoland at 3 a.m. because he was unable to sleep after doing his final calculations, that kind of excitement and joy reverberates within the quantum foundations group at Perimeter.