Governing by Inference: The Philosophical Crisis of Algorithmic Rulemaking in American Federal Science Policy

In the history of scientific governance, the question of who decides has always been entangled with the question of how we know. From the Progressive Era consolidation of expert authority within federal bureaus to the mid-twentieth-century formalization of risk assessment methodology, American regulatory science has operated on a tacit but durable premise: that the reasoning behind a binding public determination should, in principle, be legible to those it governs. That premise is now under sustained pressure from a direction few regulatory theorists anticipated—not from political interference or institutional corruption, but from the architecture of computation itself.

Across a widening range of federal domains—the Environmental Protection Agency's modeling of cumulative chemical exposures, the Food and Drug Administration's signal-detection systems for post-market pharmacovigilance, the Centers for Medicare and Medicaid Services' quality-scoring algorithms for hospital reimbursement—proprietary or semi-proprietary algorithmic systems have assumed consequential roles in determining what the regulatory record will say. The outputs of these systems carry normative weight. They shape standards, trigger enforcement actions, and allocate public resources. Yet the methodological architecture that produces them frequently remains shielded from public examination, either by commercial confidentiality claims, by the genuine opacity of machine-learning inference, or by both simultaneously.

The Epistemological Inheritance of Administrative Science

To appreciate the novelty of this situation, it is worth recalling the intellectual tradition from which American regulatory science descends. The procedural framework established by the Administrative Procedure Act of 1946, and subsequently elaborated through decades of judicial review, rests on a conception of reasoned decision-making that is fundamentally discursive. Agencies must explain themselves. The State Farm doctrine, as interpreted by the Supreme Court in 1983, requires that an agency's factual and methodological premises be sufficiently articulated to permit meaningful judicial scrutiny. The underlying epistemology is one of transparent inference: conclusions must be traceable to evidence through a chain of reasoning that an informed observer can follow and, if necessary, contest.

This tradition was itself a philosophical achievement. It represented a negotiated settlement between technocratic expertise and democratic accountability—a settlement that acknowledged the irreducible role of specialized knowledge in modern governance while insisting that such knowledge remain answerable to public reason. The scientist-administrator was granted authority, but authority of a conditional and reviewable kind.

Algorithmic regulation does not simply modify this settlement; in important respects, it dissolves its preconditions.

When the Method Cannot Be Recovered

The philosophical problem is not merely one of proprietary secrecy, though that dimension is serious enough. It is, more fundamentally, a problem of the nature of certain computational inferences. A gradient-boosted ensemble model trained on tens of thousands of variables to predict, say, the likelihood that a chemical compound will exhibit endocrine-disrupting effects at sub-threshold exposures does not reason in the way that a human toxicologist reasons. It does not weigh evidence against a theoretical framework; it identifies statistical regularities in high-dimensional space. When such a model produces a regulatory-relevant output, the question "why did the model conclude this?" may not have a satisfying answer—not because the answer is being withheld, but because the model's inference is not decomposable into the kind of propositional structure that human deliberation requires.

This creates what might be called an epistemic accountability gap: a space between the inputs and outputs of a regulatory determination where the usual instruments of democratic scrutiny—public comment, expert testimony, legal challenge—cannot easily reach. Citizens and their legal representatives can challenge the data that entered the model, and they can dispute the validity of the output, but the inferential process connecting the two may be genuinely beyond reconstruction.

The Displacement of Scientific Judgment

There is a further dimension to this problem that concerns the sociology of scientific authority within regulatory institutions. When an algorithmic system assumes a central role in producing the evidentiary record that underlies a rulemaking, the scientists employed by that agency do not simply defer to the machine. They interpret its outputs, contextualize them, and translate them into regulatory language. But the nature of their interpretive work changes in ways that are philosophically significant.

Rather than reasoning from evidence to conclusion, agency scientists increasingly reason about conclusions that have already been generated by systems they did not build and may not fully understand. Their expertise becomes hermeneutic rather than generative—a matter of reading algorithmic outputs rather than producing scientific arguments. This is not a trivial shift. It alters the epistemic character of the regulatory record and, arguably, the professional identity of the regulatory scientist.

The historian of science Lorraine Daston has written compellingly about the historical variability of what counts as a good scientific reason. What the current moment reveals is that the institutional infrastructure of regulatory science was built for one kind of reason-giving and is now being asked to accommodate another kind entirely.

Toward a Philosophically Adequate Response

The challenge, then, is not simply technical—it is not resolved by demanding that agencies publish their source code, though transparency of that kind would be a necessary condition of any adequate response. The deeper challenge is to develop a philosophical and legal framework adequate to the epistemic situation that algorithmic governance creates.

Some scholars of administrative law have proposed that algorithmic systems used in rulemaking should be subject to mandatory interpretability requirements—that agencies should be permitted to use only those models whose outputs can be explained in terms accessible to non-specialist review. This proposal has the virtue of preserving the discursive epistemology on which the APA framework depends. Its limitation is that it may systematically exclude the most predictively powerful tools, potentially at real cost to regulatory efficacy.

Others have argued for a more radical reconceptualization: that algorithmic regulatory outputs should be treated not as findings of fact but as hypotheses requiring independent corroboration before they can bear normative weight. On this view, the model's output is a prompt for further investigation, not a conclusion. This approach preserves the primacy of human scientific judgment while still allowing algorithmic tools to play a productive role in the regulatory process.

Neither framework is fully satisfying, and the philosophical work of developing a genuinely adequate alternative has barely begun. What is clear is that the current situation—in which consequential regulatory determinations rest on inferential processes that cannot be examined, challenged, or even fully described—represents a significant departure from the epistemic norms that have historically grounded the legitimacy of American regulatory science. That departure deserves far more sustained philosophical attention than it has so far received.