Improved U.K. Peer-Review or "Scientific McCarthyism"?

18 March 2009 3:59 pm

Hoping to cut down on the number of funding proposals that go through peer-review, the United Kingdom’s main provider of research funding in engineering and physical sciences will ban submissions from "repeatedly unsuccessful applicants," a policy change that could exclude about 5% of its previous grant applicants. Outraged U.K. researchers have hurled charges of “blacklisting” and "scientific McCarthyism” at the Engineering and Physical Sciences Research Council (EPSRC), which announced the change to its funding system last week.

An online petition calling on the EPSRC to repeal the change has already been signed by more than 200 people, including several fellows of the Royal Society, and some researchers plan additional measures. Philip Page, an organic chemist at the University of East Anglia, says he will no longer volunteer to peer-review grant proposals for the EPSRC. “Some colleagues are unhappy about refereeing proposals under these circumstances—i.e. they don't want to be (partially) responsible for others being excluded. That is the reason that a number of people will not referee proposals while this policy is in place,” he e-mailed ScienceInsider.

The furor results from EPSRC’s efforts to deal with what it calls the growing burden of peer-reviewing grant proposals. EPSRC receives the largest number of grant proposals among the U.K. research councils, and that number has grown dramatically in recent years, according to David Reid, head of communications at the council.

The EPSRC has now decided that applicants cannot resubmit a funding proposal unless invited, even if the previously rejected proposal had gone to another funding body. Still, that shift by itself merely brings the EPSRC in line with the policies of the other research councils: The U.S. National Institutes of Health has also had a vigorous debate about grant resubmissions.

What has really riled U.K. scientists is the new EPSRC policy that “repeatedly unsuccessfully applicants” will be banned from any EPSRC funding consideration, for a year or possibly longer.

According to the new policy directive:

From 1 June 2009, we will exclude repeatedly unsuccessful applicants from submitting proposals to EPSRC for 12 months and ask them to take part in a mentoring programme.

This will apply to applicants (listed as the principal investigator on a proposal) who have:

Three or more proposals within a two-year period ranked in the bottom half of a funding prioritisation list or rejected before panel; AND
An overall personal success rate of less than 25%.

We expect this to affect around 200-250 people, accounting for 5% of applicants and 10% of applications.

There appears to be some confusion within the scientific community on whether both of the listed criteria must be met before someone is excluded. Reid stresses that both criteria will be used and argues that as written, “the online petition is wrong."

Reid also rejects the charge of “blacklisting” and says the purpose of the policy change is to weed out the small number of scientists who submit multiple, poor applications. “They’re operating a scatter-gun approach” and placing a “huge burden” on the peer-reviews process, he says. Reid adds that EPSRC consulted with university officials who expressed a desire to know which of their researchers need mentoring or other training that could help them apply for research funding.

Still, no other U.K. research council has taken such a stance before. “We don’t think there’s any precedent elsewhere,” says Joseph Sweeney of the University of Reading, an organic chemist who signed the online petition and submitted a letter protesting the EPSRC policy to a London newspaper.

Several scientists who spoke with ScienceInsider expressed concern about excluding those below a 25% success rate, worrying that far more than 5% of applicants would eventually be excluded. Page also argues that for the large majority of proposals—those not universally recognized as excellent or poor—whether they end up in the "bottom half" of the rankings is “rather a lottery.” A more defensible criterion, he says, would have been to use the quality cutoff line many panels already employ to indicate proposals that do not merit funding.

Peter Lawrence, a Royal Society Fellow at the University of Cambridge, is another person who signed the online petition, in part to express his ire about another recent grant proposal requirement: a statement of economic impact. He makes the point that excluding people’s ideas based on past funding proposal failures “doesn’t support the unpredictability of original research.”

For the moment, the EPSRC is vigorously defending its new policy, suggesting that those protesting don't really understand the new system. But when told that some scientists were planning to withdraw their peer-review services, Reid paused and agreed that that could counter the benefits the EPSRC hoped to bring about with the new rules. “If more people start to complain, we will reconsider things,” he says. This is part of an “evolving program to reduce the [peer-review] burden.”

Posted In: