Causal impact of content expertise on predictions of biomedical research outcomes

Study ID sspp-2026-0002-v1

General Details

Project Causal impact of Content Expertise on Predictions of Biomedical Research Outcomes
Study ID sspp-2026-0002-v1
Study Title Causal impact of content expertise on predictions of biomedical research outcomes
Authors Chiara Franzoni, Andola Stanaj
Completion Time 10 Minutes
Close Date (UTC) March 15, 2026
Discipline Economics, Management
Field Other
Country Online (many countries)
Abstract
Funding agencies (grantmakers) seeking to maximize the impact of science funding entrust experts (peer reviewers) to evaluate and select research proposals on their behalf. A standard practice in peer review—also referred to as “deferral to expertise”—is to appoint reviewers with content (topical) expertise, i.e. a background as close as possible to the research proposals evaluated, such as conducting research and publishing scientific papers in the same area of study. This practice is not backed by the general findings of the forecasting literature, which indicate no or weak advantages of topical expertise on forecasting. However, in real-world settings such as geopolitics or financial markets, topical expertise is generally difficult to assess. By contrast, in biomedical research, where reviewers produce many publications per year, LLM-based techniques can be used to characterize the knowledge background of each expert. We are conducting a large RCT study in the biomedical field to test the causal impact of content expertise on forecasting accuracy of research project outcomes. Our research question is straightforward: are reviewers with strong content expertise providing more accurate predictions than reviewers in the same field but with weak content expertise? The RCT study has received IRB approval from the NBER Review Board (FWA #00003692; IRB Ref#25_212 of December 4, 2025). The RCT study was pre-registered on the AEA RCT Registry (AEARCTR-0017412; December 26, 2025). The RCT was launched recently and is underway until mid-March 2026. We ask you to predict the key and supplemental outcomes of our RCT.

Incentive Details

Incentive Type Based on forecast accuracy
Calculation Method Discrete: Fixed payment if forecast is within bounds
Recipient Pool Everyone receives the incentives

Forecast Distributions

When uploading forecasting surveys, authors are asked to select several key questions which are often the questions they think are the most important to the study.

View Key Question Response Distribution:

Disclaimer: All data are preliminary and may differ from the published results.

Citation
Franzoni, Chiara, and Andola Stanaj. 2026. "Causal impact of content expertise on predictions of biomedical research outcomes." Social Science Prediction Platform. February 5. https://socialscienceprediction.org/s/yfa2d6