WASHINGTON, D.C.--Simply trusting that scientists are telling the truth is not enough when it comes to vetting blockbuster research. That appears to be the bottom line of an independent committee that reported today that Science could have more aggressively examined two papers on stem cells that turned out to be fake. Although the committee found that editors followed established procedures in reviewing the papers, it concluded that additional procedures would have made the fraud easier to detect. The committee, appointed by Science's top editors to review the journal's handling of papers from Woo Suk Hwang and his colleagues in South Korea and the United States, combed through reviewers' comments, editors' notes, and other documents to assess how Science handled the high-profile--and, as it turned out, largely fabricated--submissions.
In a telephone press conference held in conjunction with the report's release, Science Editor-in-Chief Donald Kennedy pledged to implement the report's recommendations, although he did not specify a timeline. "Part of me is relieved" by the report's conclusions that Science followed its procedures, Kennedy said. But he also called the report "challenging" and said it offers "tough advice" to the journal. That includes new guidance on preparing images, which were manipulated by Hwang's team and have been in other fraud cases, clarifying roles of individual authors, and performing "risk assessments" on high-impact papers to determine how likely they are to be faked.
These recommendations stem from one of the most audacious frauds in scientific publishing history. The Hwang team's papers, published in 2004 and 2005, posited spectacular achievements in stem cell biology; the second paper described the first-ever human embryonic stem cell lines that were genetically matched to patients, a key step required before embryonic stem cells can be used in medical treatment. Then last December, after an anonymous allegation in South Korea, doubts surfaced about the veracity of the publications. It turned out that images purported to represent cloned stem cells from patients were actually cells from a fertility clinic's fertilized embryos (ScienceNOW, 30 October).
About 6 months after the fraud came to light early this year, Science asked six individuals to collectively examine how the faked data made it past peer review, whether the journal had followed its own procedures for vetting papers, and whether it might have done anything differently. The committee included three members of Science's external Senior Editorial Board, two prominent stem cell biologists, and a top editor at Nature who used to work at Science. It was chaired by John Brauman, a chemist at Stanford University and one of the board members.
In a succinct four pages, the committee agreed that Science had followed its procedures for reviewing papers. "The editors made a serious effort--substantially greater than for most papers published in Science--to ensure that the science was sound," the report states. This included an unusual number of questions for the papers' authors from editors and reviewers and extensive revisions of the 2004 paper. Still, "the existing procedures led to an unfortunate outcome, and have done so on several previous occasions," the report reads. Those include eight papers by Jan Hendrik Schön, a physicist formerly at Bell Labs who was found to have committed widespread scientific fraud. And although the committee acknowledges that no antifraud strategy is airtight, it notes that Science could have taken additional steps to vet the Hwang papers.
For example, reviewers of the 2004 paper were concerned about a central claim of that paper, that a human stem cell line had been generated by cloning a blastocyst, a very early embryo. Although the reviewers initially asked Hwang's group for raw data to back up the claim, they later accepted the written explanations the South Koreans offered. Science's editors were, at the time, aware of this back-and-forth. Kennedy disputes this point, arguing that the data were provided as the reviewers requested.
The committee makes four recommendations. First, it suggests requiring a "risk assessment" for certain papers accepted for publication, by asking questions about the likelihood that the work may be faked or simply incorrect, and how that would affect the journal, academic credit, intellectual property, and other issues. Second, the committee urges Science to clarify the responsibilities of authors. (In the Hwang case, an American senior author was found not to have actively contributed to the experiments.) Third, it recommends requiring additional data, particularly primary data, in the published supporting material accompanying a paper. And fourth, it calls for high-profile journals including Science and Nature to come together and establish common standards.
"We at Nature welcome the external review conducted by Science and are considering its recommendations," wrote Nature's Editor-in-Chief Philip Campbell in an e-mail. His journal, he says, is also planning to publish a response to the report.
The review committee concludes that "Science must institutionalize a healthy level of concern in dealing with papers that it considers for publication." Kennedy agreed that "we have to abandon the hope that collective trust will keep on working." Still, one issue Science will weigh as it examines how to implement the recommendations is "whether this loss of trust might not be costing the system more than the occasional retraction."