Compliance Without Cure: The Bureaucratic Performance of Scientific Reproducibility
The Appearance of Accountability
In the years following the so-called replication crisis—a period in which landmark findings across psychology, biomedicine, and social science collapsed under the weight of independent scrutiny—American science's principal funding bodies responded with the instruments most natural to bureaucratic institutions: requirements, checklists, and mandated disclosures. The National Institutes of Health issued new guidelines demanding rigor and reproducibility statements in grant applications. The National Science Foundation expanded its open data provisions. The Department of Energy began requiring data management plans as standard components of funded research proposals. On paper, the reform era had arrived.
Yet a curious asymmetry persists. The volume of irreproducible findings has not meaningfully declined. Preregistration rates, while rising, remain low in many fields. The incentive structures that reward novelty over verification, positive results over null ones, and publication quantity over methodological depth have not been dismantled. What has changed is the administrative surface of science—the forms researchers complete, the boxes they check, the language they deploy in compliance documents. The question this raises is not merely practical but deeply philosophical: can procedural compliance, however earnestly enforced, substitute for the kind of epistemic culture change that reproducibility genuinely demands?
Proceduralism and Its Philosophical Limits
The philosophy of science has long distinguished between the formal rules governing inquiry and the tacit norms that actually shape scientific practice. Michael Polanyi's concept of tacit knowledge, developed across mid-twentieth-century writings, held that scientific competence cannot be fully codified—that researchers learn through apprenticeship, example, and embedded community practice rather than through the explicit articulation of rules. If Polanyi was correct, then the belief that reproducibility can be mandated through compliance language reflects a fundamental misunderstanding of how scientific knowledge is actually produced and transmitted.
Funding agencies, by their nature, operate through legibility. They require that research practices be rendered visible, documentable, and auditable. But the conditions that produce irreproducible science are often neither visible nor documentable in the ways bureaucratic instruments can capture. A researcher who designs a study with unconscious flexibility in analysis—what methodologists call researcher degrees of freedom—does not record that flexibility in a data management plan. A laboratory culture that quietly discourages negative results does not announce itself in a rigor statement. The pathology is structural and cultural; the remedy is procedural and administrative. The mismatch is not incidental. It is constitutive.
The Publication Market as Unaddressed Variable
Among the institutional forces most responsible for producing irreproducible science, the academic publication market stands as perhaps the most consequential and least touched by current reform efforts. The incentive to publish novel, statistically significant, theoretically dramatic findings did not originate with researchers' moral failures. It was engineered by a system in which career advancement, grant competitiveness, and institutional prestige all depend on publication in high-impact journals—journals whose editorial preferences have historically favored positive results.
Funding agencies have not, with any systematic force, addressed this dynamic. The NIH's Enhancing Reproducibility initiative, for instance, focuses extensively on the front end of research: study design, statistical planning, blinding procedures. It has relatively little to say about what happens when a rigorously designed study produces a null result and encounters a journal market largely indifferent to publishing it. The result is a kind of epistemological theater: research enters the system with improved documentation of its methodology and exits through a publication pipeline whose incentive structure remains unchanged. The performance of rigor is real. The transformation of conditions is not.
Reform as Institutional Self-Presentation
There is a sociological dimension to this phenomenon that deserves attention. Funding agencies are not merely scientific administrators; they are political actors whose continued appropriations depend on public and congressional confidence in the integrity of American science. When a reproducibility crisis becomes visible enough to attract media attention and legislative scrutiny—as it did in the early 2010s—agencies face pressure to respond in ways that are themselves visible. A new policy initiative, a revised grant requirement, a public workshop on open science: these are legible signals of institutional seriousness. They communicate responsiveness to oversight audiences who are rarely positioned to evaluate whether the reforms address root causes.
This dynamic is not unique to science funding. Scholars of organizational behavior have documented the phenomenon across regulatory agencies, educational institutions, and corporate compliance departments: organizations under external pressure adopt the symbolic apparatus of reform while preserving the operational structures that generated the original problem. In science policy, the consequence is that the reform discourse itself becomes a kind of distraction—absorbing energy, generating documentation, and producing the impression of progress while the underlying epistemological conditions remain substantially intact.
What Genuine Reform Would Require
To ask what genuine epistemological reform of American science would require is to ask a question that funding agencies, as currently constituted, may be structurally incapable of answering. It would require confronting the publication market directly—either by funding replication studies at a scale that makes them professionally viable, or by using grant conditions to incentivize null-result reporting in ways that alter journal behavior. It would require rethinking how scientific careers are evaluated, which implicates universities and professional societies as much as federal funders. It would require attending to the tacit norms of laboratory culture in ways that resist easy codification.
None of these interventions is impossible. Several are being attempted, in partial and experimental forms, by a range of actors including the Center for Open Science, various disciplinary societies, and a handful of progressive journal editors. But they have not become the organizing logic of federal science policy, which continues to treat reproducibility primarily as a documentation problem rather than a structural one.
The history and philosophy of science offer a useful corrective here. They remind us that the norms governing scientific inquiry have changed before—not through administrative mandate but through the slow transformation of what scientific communities regard as credible, admirable, and professionally rewarding. The question is whether the current moment of reform, however procedurally elaborate, is generating the conditions for that deeper transformation, or whether it is primarily generating the appearance of having done so.
