Psychological Science Is Not Broken -- It's Maturing

Daniel Engber’s recent piece in Slate’s “Science” section (“Is Science Broken? Or is it self-correcting?” August 21st, 2017) argues that science is currently failing, and refusing to recognize this failure only serves to empower skeptics. “Science is broken,” Engber proclaims. “Self-correction doesn’t always happen, and science journalists mustn’t be afraid to spell that out.” Engber concludes that relying on this false framing of self-correction makes science weaker and less persuasive. Admitting the brokenness of science, he argues, will ultimately strengthen the field in the eyes of those who doubt its merit.

Engber’s thesis is misguided, and I will tackle that particular issue in a moment. Perhaps more troublesome, however, is that Engber does not offer a single constructive idea for how to help strengthen scientific integrity. Rather, Engber simply points to a flawed system and says it is flawed. “Our vaunted engine of discovery is sputtering and… it’s time we brought it in for repairs,” he claims. What ought we repair? Where is the metaphorical break in our scientific motor? These questions go unanswered, and for good reason – the breaks are a feature of science’s design.

Allow me to unpack this idea a little further. Engber does not suggest that science is in a state of “utter, irreversible decrepitude—that every published finding is a lie, or that every field of research is in crisis.” Instead, he claims that self-correction is used as a tool for blunting the effects of controversy, and it “has since been wielded, on occasion, as a magic wand to wave away egregious missteps.” He hones in on psychology, specifically, using recent work detailing a replication crisis in psychological science as evidence that many lessons should have been learned long ago. He declares, “It’s not that all of science is a sham but that it’s not self-correcting fast enough.”

First of all, using psychology as a foil against science is a fairly cheap trick (and I say this as a psychologist in training). The scientific community has been studying human anatomy for over 3500 years, with modern medicine emerging 200 years ago. Aristotelian physics is nearly 2500 years old, with Galilean astronomy emerging 500 years ago and Newtonian physics emerging 300 years ago. If science is defined by robust, meticulous measurement, then scientific discovery is necessarily beholden to the amount of extant preceding work in a given field. Physics and psychology seem qualitatively different for this very reason. While psychological science seemingly dates back to the 18th century, Immanuel Kant at the time famously derided the notion of experimental psychology. In Metaphysical Foundations of Natural Science, Kant remarks, “The empirical doctrine of the soul can also never approach chemistry even as a systematic art of analysis or experimental doctrine.” The first real challenge to this line of thinking was Gustav Fechner’s Elements of Psychophysics published in 1860, and Wilhelm Wundt did not establish the first psychological laboratory until 1879. At approximately 150 years of age, psychological science is an infant among elders.

For someone like myself, this is precisely what makes psychological science so invigorating. We stand on the shoulders of giants in scientific history, but we are also in unchartered waters. This is perhaps what Engber aimed to discuss – what are the pitfalls of a science that is so profoundly youthful? This would be a fascinating, more appropriate discussion, which requires first elaborating that the common comparison of hard versus soft sciences is a false distinction. There are, of course, older and newer sciences, and with age comes the comfort that your foundations have already been tested. For instance, we learn in grade school how to prove mathematical formulations such as the Pythagorean Theorem (the square of the hypotenuse in a right triangle is equal to the sum of the squares on the two legs). But how is something proven in mathematics? Any proof must be formulated based on other theorems and corollaries, themselves proven ahead of time. If any foundational aspect of the Pythagorean Theorem were disproven, such as the Triangle Postulate (all angles of a triangle add up to exactly 180 degrees), then the Pythagorean Theorem may itself be disproven. Given the age and prevalence of Euclidean geometry, we often rest assured that many minds have tried to disprove these theorems and corollaries. It is worth noting that Elisha Scott Loomis’ The Pythagorean Proposition contains 367 different proofs of this theorem. In the presence of such abundance, and in the absence of counterfactual evidence, the scientific community considers the Pythagorean Theorem a reliable foundation.

Fast forward 2.5 millennia from Pythagoras to the present day. Mathematicians continue to develop proofs for novel concepts, many of which rely upon the proven theorems that preceded them. Physicists have developed and tested theories to explain what Newtonian physics could not, leading to Einstein’s Theory of Relativity. Then, in explaining what relativity could not, physicists developed modern quantum mechanics. Newton was not wrong, nor was Einstein – their theories and formulations were simply incomplete. Their work explained, using the best physical knowledge available, novel truths about how the universe works. Depending on the context, Newtonian physics may often be sufficient to describe and evaluate a phenomenon – if an apple falls from the tree and hits you in the head, you can use Newtonian physics to accurately describe its trajectory. But in other circumstances, we turn to relativity, or to quantum mechanics. And eventually, physics will develop a new theory to explain what these theories cannot.

In comparison to these venerable fields, psychology is a toddler. Few theorems have been tested and supported 20 times, and therefore very little can be empirically proven. Adding to that complexity, even the most well-replicated phenomena in psychology remain context-dependent, because we pull the levers of our psychological environment far more freely than we control the natural world around us. I cannot alter gravity, and I cannot will a broken bone to heal in an hour. I can, however, make you more attentive, comfortable, energetic, or confused. I can make a small group of people more cooperative, more productive, or more combative. And at a macroscopic level, cultures shape the way we experience the world, such that a perplexing health phenomenon amongst Italian immigrants in rural Pennsylvania does not seem to generalize to similar populations elsewhere. This does not make the phenomenal finding any less valid – we simply need to understand and articulate how psychological science varies among different contextual constraints. For instance, when scientists published a 1990 meta-analysis indicating that men and women tend to utilize differing leadership styles, they did not declare this difference as a law of the natural universe. Rather, the finding demonstrates that, in the complex socio-cultural environment of America in the 1990’s, a particular tendency was observed. Sure, some men had more participative leadership styles, and some women had authoritarian styles. But on the average, the reverse phenomenon existed at that particular cross-section in time and culture. This conclusion is valuable, even if it is not a mathematical proof.

So let us now circle back to the current replication crisis in psychology and why Engber believes “our vaunted engine of discovery is sputtering.” Psychological science has boomed in accordance with demand. With more researchers and more money to spend, the field can formally test ideas that have been taken as gospel, and this is a very good thing. The only way to for any science to evolve is by testing what we believe to be true, rather than accepting long-held theories as self-evident. If and when new evidence supplants the current line of thinking, the psychological community will embrace it. That is what science is at its heart: ruthless pragmatism.

If Engber has a better frame of reference for contemporary psychological science, I urge him to make his suggestion known. The scientific community embraces constructive direction; after all, such direction would provide us with new hypotheses to test. But to declare science as broken is to ignore scientific history. Though we can all agree we would like for science to move faster, failure remains a feature, not a mishap, of the scientific process. Psychology is no exception.

A failure is not always a mistake, it may simply be the best one can do under the circumstances. The real mistake is to stop trying. ~B. F. Skinner