Perhaps the biggest threat to secular progressivism’s claim to scientific validation is the growing awareness of just how little of that validation can be replicated experimentally. In 2005 Dr. John Ioannidis, co-director of Stanford University’s Meta-Research Innovation Center, wrote a now-famous paper showing that much of what has passed for “settled” research in the social sciences and related disciplines is, in fact, the product of unreliable statistical testing, the use of small sample sizes, unwarranted credence given small effects, unshared experimental data, and other scientifically dubious methods.
Ever since Ioannidis’ paper, it has become painfully clear that even many of the most influential experiments in sociology, political science, social psychology, economics, climate science, anthropology, education, and medicine cannot be duplicated. Studies supposedly confirming the progressive belief that discriminatory behavior stems from unconscious stereotyping, for example, turn out to be based on highly unreliable methodology.
In 2015, Science tried to replicate the findings of 100 articles published in three prominent psychological journals during 2008 and got significant results for only 36, compared with the significance claimed by 97 of the originals. A similar study one year later in the Finance and Economics Discussion Series of the Federal Reserve could not reproduce the outcomes of a majority of prominent economics articles.
Ioannidis himself believes that up to half the discoveries published in peer-reviewed social science and medical journals are probably wrong, an opinion he shares with The Lancet’s respected editor-in-chief, Richard Charles Horton. National Association of Scholars (NAS) President Peter Wood has argued that many of the regulations, laws, and social programs routinely passed by the U.S. Congress on the presumption that they reflect rigorous research have no real scientific basis.
Theoretically, of course, there are reasons other than the desire to please a government funder why a particular experiment might not be reproducible: researcher incompetence, sloppiness, damaged equipment, the psychological pressure to find positive results, and even the hope that a faked result will someday be proved correct. It is also true that not all social science, education, and medical studies are done for some state or federal agency. Non-profits, pharma and other companies, professional associations, and public employee unions also sponsor university research – although their dependence on politicians for regulatory approvals, tax breaks, member licensure, employee benefits, and other operating advantages does not quite make them ideologically neutral.
But perhaps the best indicator of how the experimental irreproducibility crisis threatens secular progressivism is how lopsided the political response to solving it has been. In February of this year the Independence Institute, a prominent free market think tank, sponsored the first national conference on “Practical Solutions for the Irreproducibility Crisis.” Among its seemingly commonsense recommendations were to spend more money on replication studies, to prioritize grants to researchers who pre-register their protocols and meet new best-practices standards, and to require government-funded experimenters to make their data and research protocols publicly available.
On the other hand, the social media response of the left to these suggestions has been almost hysterical, labeling them everything from misogynistic to white supremist to fronts for climate change denialism. Two graduate students set to speak and the Independent Institute conference itself were forced to withdraw out of concern for career sabotage, according to organizers.
Such attacks continue, even though the authors of the most detailed expose of academic bias, The Irreproducibility Crisis of Modern Science, have gone out of their way to show that some liberal causes, such as reversing climate change, could conceivably benefit from stricter research standards. While “the short-term thrust of [experimental reproducibility] reforms may seem to favor the political agendas of American conservatives,” write David Randall and Christopher Welser, “every American who cherishes the scientific pursuit of truth should seek to solve the problems that beset contemporary science.”
What the left undoubtedly fears from the effort to end funder bias is an intellectual setback far more damaging than just losing the ability to favorably footnote preferred legislation. “Not all irreproducible research is progressive advocacy,” as NAS president Wood puts it, and “not all progressive advocacy is irreproducible; but the intersection between the two is very large.”
The good news is that experimental reproducibility is so fundamental to science that no ideology or self-serving relationship between government and the academy can prevent past studies from eventually being rerun under more rigorous conditions. As a 2017 Nature editorial put it, “Replication studies offer much more than technical details. They demonstrate the practice of science at its best.”
Neither is it possible for new studies to remain as corrupted by self-interest as they have previously been. Indeed, the psychology community is already encouraging study authors to include suggestions on how to reproduce their results as well as to publish their protocols. The biggest cultural change in the coming years could well be inspired, not by political activists or party platforms, but by the pages of the latest scientific journals.
