NIH Takes Steps to Improve Reproducibility
For the last few years, concerns have been growing in the biomedical research community that many animal studies can’t be reproduced in other labs. The problem has frustrated industry researchers and raised questions about the basis for some clinical trials. Today, in a comment in Nature, leaders at the U.S. National Institutes of Health (NIH) discuss several efforts to improve the reproducibility of preclinical research.
NIH Director Francis Collins and Principal Deputy Director Lawrence Tabak write that “the checks and balances that once ensured scientific fidelity have been hobbled.” They blame several factors: poor training, an emphasis on provocative conclusions in papers, a dearth of experimental details, and an overemphasis on publications in high-impact journals.
To address these problems, NIH is launching several pilot projects, such as a training module to teach young researchers how to design better experiments. The agency also plans to have grant reviewers use checklists to make sure proposed experiments are properly designed.
Journals also need to be part of the solution, the NIH leaders write. They praise announcements last year by the Nature Publishing Group and Science Translational Medicine and this month by Science that these journals will require more details about experiments and will check a study’s statistical analysis.
To correct a skewed academic rewards system, the agency may change its requirements for submitting biographical sketches, so that they emphasize an author’s contributions rather than publications. It’s an idea that National Cancer Institute chief Harold Varmus has espoused. NIH is also looking at funding more researchers through long-term grants and making the peer-review process anonymous to reduce bias.