AIs Predict Research Results Without Doing Research

From the YouTube page:

Scientific literature is growing rapidly, meaning scientists are increasingly unable to keep up with all of the latest developments in research. AI large language models, though, can read and “digest” information much more quickly than their human counterparts, making them the perfect tools to conduct massive literature reviews. Recent research shows they’re also very accurate at predicting the results of studies that they’ve never read before. Let’s take a look.

Hossenfelder references this article:

Luo, X., Rechardt, A., Sun, G. et al. Large language models surpass human experts in predicting neuroscience results. Nat Hum Behav (2024). https://doi.org/10.1038/s41562-024-02046-9

Abstract: Scientific discoveries often hinge on synthesizing decades of research, a task that potentially outstrips human information processing capacities. Large language models (LLMs) offer a solution. LLMs trained on the vast scientific literature could potentially integrate noisy yet interrelated findings to forecast novel results better than human experts. Here, to evaluate this possibility, we created BrainBench, a forward-looking benchmark for predicting neuroscience results. We find that LLMs surpass experts in predicting experimental outcomes. BrainGPT, an LLM we tuned on the neuroscience literature, performed better yet. Like human experts, when LLMs indicated high confidence in their predictions, their responses were more likely to be correct, which presages a future where LLMs assist humans in making discoveries. Our approach is not neuroscience specific and is transferable to other knowledge-intensive endeavours.