Subscribe
 
 

A Radical Revamp of Peer Review?

7 December 2007 (All day)
Comments

UCSF

Cause for celebration?
UCSF's Keith Yamamoto is leading a committee to reimagine peer review at NIH.

BETHESDA, MARYLAND--Scientists conducting a sweeping examination of the peer-review system at the National Institutes of Health (NIH) are considering some radical ideas to revamp the process, they revealed today. At a meeting here of the advisory committee to NIH Director Elias Zerhouni, members debated everything from doing away with the current scoring system on grant proposals to incentives that might improve the quality and motivation of reviewers.

Although peer review is still considered a cornerstone of science, it is experiencing new pressures. The average first-time NIH grantee is getting older, NIH budgets are nearly flat, and science has grown more complex. Furthermore, "we are desperately worried" about new investigators, who find it hard to land grants and may turn away from science, says Mary-Claire King, a geneticist at the University of Washington, Seattle, who was not a member of the peer-review team.

NIH last examined peer review 8 years ago in its so-called Boundaries report, but not all grants were covered. This time, "Elias said, 'Look at the whole thing,' " says Keith Yamamoto, executive vice dean of the University of California, San Francisco (UCSF), School of Medicine, and a biochemist there. In June, Zerhouni set up two working groups--one inside NIH and one outside--to solicit and synthesize comments from the scientific community (Science, 15 June, p. 1553).

There was no shortage of ideas; more than 2600 comments poured in. At today's meeting, Yamamoto, who co-chairs the external working group, and Lawrence Tabak, director of NIH's National Institute of Dental and Craniofacial Research and co-chair of the internal and external working groups, presented ways to reduce the administrative burden of grant writing and reviewing while funding the very best science. One possibility included an "editorial board" model, in which grant proposals that contain certain technical details could be sent to experts who would consider those elements alone and report back to the study section. This could help reduce the size of study sections, which have ballooned to as many as 80 people to accommodate all the expertise needed.

Other ideas include encouraging reviewers to be more direct about proposals that seem hopeless and seek fewer revisions; having each reviewer rank only the top 10 applications rather than scoring them all; and finding ways to get and keep the best reviewers in the system--including permitting reviewers to rate the reviews of their colleagues, which they can’t do now. Some proposed briefing reviewers on work NIH already funds to help avoid overlapping grants. "Are we studying the same protein head to toe?" Tabak asked.

For those applying for grants, possibilities being seriously considered include streamlining applications from 20 pages to seven, essentially doing away with preliminary data and instead focusing on a project's potential impact; and allowing resubmission of grant proposals only rarely, rather than the usual two times permitted today. There are "a lot of tensions that we're trying to navigate," said Mary Beckerle, director of the Huntsman Cancer Institute at the University of Utah in Salt Lake City, who is a member of one of the working groups.

Now, say Yamamoto and Tabak, the goal is to synthesize the ideas that have bubbled up in the two different working groups and present firm recommendations to Zerhouni by February.

Related sites

Posted In: