Are we publishing too damn much?

There's too much science: “These findings suggest troubling implications… If too many papers are published in short order, new ideas cannot be carefully considered against old & processes of cumulative advantage cannot work to select valuable innovations” https://t.co/os1xxwTCKl pic.twitter.com/vZbCgTDEN3

— Ethan Mollick (@emollick) March 31, 2023

Significance:

The size of scientific fields may impede the rise of new ideas. Examining 1.8 billion citations among 90 million papers across 241 subjects, we find a deluge of papers does not lead to turnover of central ideas in a field, but rather to ossification of canon. Scholars in fields where many papers are published annually face difficulty getting published, read, and cited unless their work references already widely cited articles. New papers containing potentially important contributions cannot garner field-wide attention through gradual processes of diffusion. These findings suggest fundamental progress may be stymied if quantitative growth of scientific endeavors—in number of scientists, institutes, and papers—is not balanced by structures fostering disruptive scholarship and focusing attention on novel ideas.

Abstract:

In many academic fields, the number of papers published each year has increased significantly over time. Policy measures aim to increase the quantity of scientists, research funding, and scientific output, which is measured by the number of papers produced. These quantitative metrics determine the career trajectories of scholars and evaluations of academic departments, institutions, and nations. Whether and how these increases in the numbers of scientists and papers translate into advances in knowledge is unclear, however. Here, we first lay out a theoretical argument for why too many papers published each year in a field can lead to stagnation rather than advance. The deluge of new papers may deprive reviewers and readers the cognitive slack required to fully recognize and understand novel ideas. Competition among many new ideas may prevent the gradual accumulation of focused attention on a promising new idea. Then, we show data supporting the predictions of this theory. When the number of papers published per year in a scientific field grows large, citations flow disproportionately to already well-cited papers; the list of most-cited papers ossifies; new papers are unlikely to ever become highly cited, and when they do, it is not through a gradual, cumulative process of attention gathering; and newly published papers become unlikely to disrupt existing work. These findings suggest that the progress of large scientific fields may be slowed, trapped in existing canon. Policy measures shifting how scientific work is produced, disseminated, consumed, and rewarded may be called for to push fields into new, more fertile areas of study.

When I was an undergraduate at Johns Hopkins I remember hearing Dick Macksey lament the rate of publication in humanities fields. That was back in the 1960s. This is a long-standing and pervasive problem.

Perhaps GPT-X will help us make sense of it all. But, you know, despite what I've just posted about meaning in LLM-generated text, I'm having trouble imagining just when I'll trust LLMs to scan, evaluate, and summarize the professional literature in any field. Though, who knows, maybe they can be made to be more objective than humans.