Mark Humphries, The Absurdity of Peer Review: What the pandemic revealed about scientific publishing, Elemental, June 3, 2021.
Humphries begins by reporting that he "was reading my umpteenth news story about Covid-19 science" and noted that the story cited many pre-prints and, in each case, the mention
was immediately followed by the disclaimer that it had not yet been peer reviewed. As though to convey to the reader that the research therein, the research plastered all over the story, was somehow of less worth, less value, less meaning than the research in a published paper, a paper that had passed peer review.
Imagine reading about the discovery of the structure of DNA with that same reticence we use today: “In a recent Letter to the journal Nature, Cambridge University scientists James Watson and Francis Crick proposed a new structure for DNA (not yet peer reviewed). They claim their “double helix” model, a spiral of two strands of bases, both explains decades of experimental work, and provides a clear mechanism for copying genes. Their proposal drew heavily on data contained in Letters in the same issue of Nature from the teams of Rosalind Franklin (not yet peer reviewed) and Maurice Wilkins (not yet peer reviewed).”
Or consider this modern take on a certain scientist’s annus mirabilis: “The past year of 1905 has been a remarkable for one Herr Einstein, who proposed no less than four theories new to modern physics in a series of papers. His first was on a much-anticipated explanation of the photoelectric effect (not yet peer reviewed), the second on how Brownian motion arises from the collision of invisible particles (not yet peer reviewed), the third on the equivalence between mass and energy (not yet peer reviewed), and the final paper updates Newton’s mechanics to be more accurate for objects moving close to the speed of light (not yet peer reviewed).”
These imagined reports are both eye-wateringly ridiculous.
The papers they describe are works that have formed the foundations of biology and physics as we know them. None of the classic papers on DNA, nor Einstein’s four in his miracle year, were peer reviewed. Indeed, only one of Einstein’s 300 or so published papers was ever peer-reviewed, which so disgusted him that he never submitted a paper to that journal again. It did not matter that the DNA papers and Einstein’s papers were not peer-reviewed. Nor did it matter for the many thousands of the classic papers published in the past century that were not peer reviewed: the work in them is the foundations on which science stands today.
Whoops! Note that peer review didn't become the norm in science until the 1970s. How did science ever advance without it?
So, if peer review doesn't insure intellectual rigor and scientific interest, what's it good for?
I’d wager that it’s a combination of simple inertia, and because journals rely on it for gate-keeping. When it did appear, peer review’s job was to help out editors who received papers beyond their expertise, and wanted some advice on whether to publish or not. Now it has become all about who gets chosen to be published in the most selective journals, who can convince a handful of reviewers their paper is worthy of publishing in that journal, and so dramatically increase the chances of their paper being read: for the primary purpose of the selective journal is to say “hey, read our stuff, it’s been hand-picked just for you.” It is a way for editors to filter papers based on the personal biases of two to four individuals. Without that gate-keeping role, it seems we would have no need of peer review.
This then is why I was so bothered about how Covid-19 research is reported: peer review is no guard, is no gold standard, has little role beyond gate-keeping. It is noisy, biased, fickle.
That is to say, it's good for nothing beyond protecting the reputations of the gate keepers.
No comments:
Post a Comment