The Invisible Tribunal: Anonymity, Authority, and the Hidden History of Peer Review
An Institution Without a Founding Moment
Americans who have submitted a manuscript to a scholarly journal, applied for a National Science Foundation grant, or sought tenure at a research university are intimately familiar with a particular ritual: the submission of one's work to anonymous judges whose identities are withheld, whose deliberations are conducted in private, and whose verdicts carry enormous professional consequence. This ritual is peer review, and it has become so thoroughly naturalized within the infrastructure of American science that most practitioners treat it as a permanent feature of intellectual life rather than a historically contingent invention.
It is neither permanent nor inevitable. Blind peer review, in the form now considered standard, did not consolidate its authority until well into the twentieth century. Its rise was shaped by institutional pressures, professional anxieties, and philosophical assumptions about impartiality that deserve far more scrutiny than they typically receive. To treat peer review as the obvious solution to the problem of scientific quality control is to forget that the problem itself was once defined very differently.
Before the Gatekeepers: Alternative Traditions of Verification
The history of science contains a rich diversity of verification practices that preceded and, in some cases, survived alongside the modern peer review apparatus. In seventeenth-century England, the Royal Society relied heavily on the testimony of credentialed witnesses—gentlemen of recognized social standing whose word carried epistemic weight precisely because their reputations were publicly staked. Steven Shapin and Simon Schaffer's foundational analysis of this period illuminates how scientific credibility was once a function of social visibility rather than anonymous judgment. The witness was known; the procedure was public; accountability was, in principle, traceable.
Across the Atlantic, the early American scientific community operated through networks of correspondence, public demonstration, and institutional endorsement. Benjamin Franklin's electrical experiments gained credibility partly through European recognition and partly through the replicability that public demonstration afforded. The notion that an unseen committee of specialists should serve as the final authority over a claim's legitimacy would have seemed peculiar, if not troubling, to many eighteenth-century natural philosophers.
Even within the journal tradition, editorial practices varied enormously. Many nineteenth-century scientific periodicals relied on the judgment of a single editor—often a distinguished scientist—who evaluated submissions personally and publicly. The editor's name, and thus the editor's reputation, was attached to every publication decision. Accountability was asymmetric but it was not absent.
The Twentieth-Century Consolidation
The shift toward systematic, anonymous refereeing accelerated in the mid-twentieth century, driven by several converging forces. The professionalization of American science following World War II produced an enormous expansion in the number of practicing researchers and the volume of manuscripts submitted for publication. Editorial boards could no longer rely on personal familiarity with contributors. Specialization deepened, requiring that evaluative authority be distributed among domain experts rather than concentrated in generalist editors.
Federal funding agencies, particularly the National Institutes of Health and the National Science Foundation as they matured through the 1950s and 1960s, institutionalized peer review as the mechanism for allocating research dollars. This was not simply a bureaucratic convenience; it was a philosophical commitment to the idea that scientific merit could be assessed independently of the identity, institutional affiliation, or social position of the applicant. Anonymity was the procedural guarantee of this independence.
Yet this guarantee rested on assumptions that were rarely examined. The model presumed that disciplinary expertise was both necessary and sufficient for valid evaluation—that a qualified specialist, freed from knowledge of the author's identity, would assess claims on their intrinsic merits. What this model obscured was the degree to which expertise itself is socially distributed, paradigmatically constrained, and institutionally situated. Reviewers bring to their task not merely technical knowledge but theoretical commitments, methodological preferences, and professional interests that anonymity does nothing to neutralize.
The Paradox of Anonymous Power
The philosophical irony at the core of peer review is this: a system designed to remove personal authority from scientific judgment has created one of the most consequential and least accountable forms of institutional power in contemporary intellectual life. The anonymous reviewer exercises enormous influence over careers, funding trajectories, and the direction of research programs, yet does so without public accountability, without formal obligation to justify decisions in terms that can be contested, and without any mechanism by which the reviewed party can identify, challenge, or appeal to the reviewer directly.
This is not merely a procedural inconvenience. It is an epistemological structure that systematically privileges certain kinds of knowledge claims over others. Research that challenges dominant paradigms faces the particular disadvantage of being evaluated by specialists whose expertise was formed within those paradigms. Methodologically unconventional work encounters reviewers whose evaluative frameworks may be ill-equipped to assess it. Work emerging from underfunded institutions, from researchers without prestigious affiliations, or from scholars whose names or research sites might signal demographic identity even through double-blind procedures, faces compounded disadvantages that anonymity cannot fully neutralize.
Philosophers of science have long recognized that theory-ladenness pervades empirical observation. It is equally true that evaluation-ladenness pervades peer review. The criteria by which significance, rigor, and contribution are assessed are not given by nature; they are products of disciplinary history, and they encode the values and priorities of those who have historically dominated the disciplines in question.
What the System Cannot See
Perhaps the most consequential invisibility within peer review is not the reviewer's identity but the system's own assumptions. American scientific culture has largely accepted peer review as a proxy for truth—or at least for a reasonable approximation of it. Journal publication in a peer-reviewed venue is treated as evidence of validity; rejection is treated as evidence of inadequacy. Grant applications that survive review are presumed to represent meritorious science; those that fail are presumed to fall short.
This conflation of procedural legitimacy with epistemic validity is historically contingent and philosophically fragile. Peer review can identify obvious errors, enforce methodological conventions, and filter out work that falls outside disciplinary norms. What it cannot do is guarantee that the norms being enforced are the right ones, that the paradigms being presupposed are adequate to the phenomena under investigation, or that the exclusions it produces are epistemically rather than socially motivated.
The history of science is populated by claims that were initially rejected by peer evaluation and later recognized as foundational, as well as claims that sailed through review and were subsequently overturned. The replication crisis that has shaken several scientific fields in recent decades did not emerge from work that bypassed peer review; it emerged from work that peer review certified.
Toward a More Historically Informed Evaluation
None of this is to suggest that peer review should be abandoned, or that the problems it addresses—the evaluation of technical claims by knowledgeable specialists—are not genuine. It is to insist that peer review, like any institution, is best understood when it is understood historically: as a solution to particular problems, encoding particular values, producing particular exclusions, and carrying particular costs.
Alternative and supplementary models exist and deserve serious attention. Open peer review, in which reviewer identities are disclosed, restores a form of accountability that anonymous systems eliminate. Post-publication review, practiced increasingly in some scientific communities, distributes evaluative authority more broadly and allows correction over time. Registered reports, which separate the evaluation of methodology from the evaluation of results, address some of the incentive distortions that current systems produce.
The deeper philosophical task is to resist treating any single evaluative mechanism as coextensive with scientific legitimacy itself. Peer review is an institution with a history, a philosophy, and a politics. Recognizing it as such is not an attack on science; it is the application of the same critical scrutiny to science's social infrastructure that science applies to everything else.
