Daren Bakst and Marlo Lewis have edited a series of science policy recommendations for reforming the Environmental Protection Agency (EPA) that literally put the irreproducibility crisis of modern science first. In the first chapter of Modernizing the EPA, Marlo Lewis argues that the EPA’s policy mistakes derive from permitting and fostering practices associated with the irreproducibility crisis of modern science. He says that the way to reform environment policy is to institute new federal statutes that require substantial improvement of the EPA’s reproducibility and transparency requirements.
Bakst and Lewis, and the Competitive Enterprise Institute (CEI) more generally, raise the bar for advocacy concerning environmental policy. Every advocate for environmental policy, whatever his position, now should meet the standards for reproducibility reform advocacy established by CEI.
The National Association of Scholars (NAS) has championed reproducibility and transparency reforms generally because they are vital for the proper functioning of scientific research. The irreproducibility crisis is the product of improper research techniques, a lack of accountability, disciplinary and political groupthink, and a scientific culture biased toward producing positive results.
[RELATED: Top NSF Officials Used Duplicate Research to Boost Their Careers]
Other factors include inadequate or compromised peer review, secrecy, conflicts of interest, ideological commitments, and outright dishonesty. Many supposedly scientific results cannot be reproduced reliably in subsequent investigations and offer no trustworthy insight into the way the world works. A majority of modern research findings in many disciplines may well be wrong—and every professional incentive in modern scientific culture discourages scientists from taking the time to reproduce scientific research to see if there’s any truth to it. Scientific research that can’t be reproduced is presumptively false; scientific research that hasn’t yet been reproduced is just interesting speculation.
The irreproducibility crisis is not just a general crisis in science. It is particularly a crisis in federal science policy.
Technocrats and radical activists embedded in government service have weaponized the powers delegated to federal science regulatory agencies, as well as the authority accorded to putatively nonpartisan scientific experts, to advance their policy goals without transparency or accountability to elected policymakers or the public. Activist bureaucrats actively commission the false positive research results produced by the irreproducibility crisis in a host of scientific and social scientific disciplines to justify the mass production of illiberal, radical regulations throughout the federal science regulatory agencies. The federal government, moreover, is the largest single funder of scientific research in the world, and federal funds not only distort American regulatory policy but also subsidize the wholesale production of irreproducible research in American universities. Federal science policy supercharged the irreproducibility crisis and only federal science policy reform can bring it to an end.
Bakst and Lewis wrote before the Trump administration began to reform science policy, by means notably including the Executive Order “Restoring Gold Standard Science,” which directs federal agencies to undertake transparency and reproducibility reforms along the lines that Bakst and Lewis suggest. Bakst and Lewis substantiate the necessity for the Trump administration’s science policy initiative, as well as providing a road-map for ways to carry our reproducibility reform in detail. Their book, in other words, explains why “Restoring Gold Standard Science” still needs to be translated into detailed regulatory form and how it should be done.
[RELATED: To Rescue Science, Phase Out Research Grants]
To return to the book itself: NAS’s own conclusions align closely with CEI’s concerns—and indeed, Lewis cites our work in Modernizing the EPA’s first chapter, both as he narrates the nature and the dimensions of the irreproducibility crisis and as he suggests legislative solutions. We endorse Lewis’ and the CEI’s conclusions. CEI’s broader advocacy also addresses a wide range of other policy concerning the EPA, much of which does not concern the irreproducibility crisis. The NAS is centrally concerned that the procedures of modern science continue to seek out truth, unaffected by politicization, group think, sloppy statistics, or any of the other aspects of the irreproducibility crisis.
Lewis and the CEI recommend a series of statutes to Congress which would serve excellently to reduce the irreproducibility crisis. These include:
Prohibit the EPA from funding PM research or using such research to determine air quality standards or other critical metrics, unless all research materials are sufficiently transparent to facilitate independent validation. (32)
Make data access a condition for receiving an EPA research grant or using a study to determine air quality standards or other critical metrics. (34)
Facilitate independent review before a study is selected to inform rulemaking. (35)
Require the EPA to weigh studies according to their reproducibility. (36)
Prioritize quality over quantity in weight-of-evidence assessments. (37)
Require each rulemaking to include a table showing whether the studies cited meet reproducibility criteria. (37)
Require correction for the effects of Multiple Testing and Modeling. (38)
Require p-value plotting to detect publication bias and data manipulation in meta-analyses. (38) (Also see pp. 41-44, 48, 50, 71-73, 82.)
Lewis and the CEI substantiate these headline proposals with thoughtful substantiation and detail. Their policy proposals are practicable and would do a great deal to improve the quality of EPA science policy. While some of these proposals parallel the NAS’s own Model Science Policy Code, they include a great many suggestions we did not make. NAS has learned from Lewis and CEI about new and better means to address the irreproducibility crisis.
[RELATED: WATCH: Science Bargains with Trump, Protesters Storm NSF, and Slugs Go Solar]
CEI, going forward, might incorporate more fully Louis Anthony Cox’s work on probability of causation, including his 2024 article in Critical Reviews in Toxicology titled “Interventional Probability of Causation (IPoC) with Epidemiological and Partial Mechanistic Evidence: Benzene vs. Formaldehyde and Acute Myeloid Leukemia (AML),” and his 2021 article “Toward Practical Causal Epidemiology” in Global Epidemiology, which might be applied fruitfully to EPA policy. Its work on weight-of-evidence policy might be expanded to a more substantial policy brief to reform the use of “expertise” in science policy. Lewis also should expand his brief analogy between proper science policy and “the Framers’ extended commercial republic, which curbs the ‘violence of faction’ not by trying to abolish faction, the causes of which are ‘sown in the nature of man,’ but by multiplying the number and variety of factions.” (41) This merits not only expanded policy treatment but also more philosophical treatment: it could be the kernel of an influential scholarly work.
The EPA establishment, and environmental activists, are predisposed to regard the CEI’s initiatives, and ours, as simply political “weaponizing” of legitimate concerns about reproducibility and transparency. There has been sufficient evidence of scientific error and malfeasance that the “it’s just a few cases of individual malfeasance” defense is no longer tenable: enough scientists have erred that government policy must make reforms. They would do better to adopt reproducibility and transparency reforms themselves—and advocate for their preferred policies from that vantage point. America would be better off if all partisans about environmental policy did so from a presumption that science should be conducted with absolute transparency and reproducibility.
For the moment, however, CEI holds the field. It has proposed excellent reproducibility reform legislation to Congress, and its opponents have proposed—nothing. Environmental activism will not be intellectually respectable until it embraces the principles of reproducibility reform. And until it does, environmental activists should blush for shame when they claim to speak in the name of science.
Follow David Randall on X and visit our Minding the Science column for in-depth analysis on topics ranging from wokeism in STEM, scientific ethics, and research funding to climate science, scientific organizations, and much more.
Image: “EPA Breidenbach Laboratory” by Antony-22 on Wikimedia Commons
