Most of the controversy over probiotic therapies, in which live "beneficial" bacteria or other microbes are administered to treat or prevent disease, has centered on their effectiveness, not their safety. That's why it was such a shock early last year when during a Dutch probiotic study for acute pancreatitis, significantly more patients died in the treatment arm than in the placebo arm. Yesterday, the Dutch Health Care Inspectorate (IGZ) published a critical review of the so-called Propatria study.
The report (in Dutch, but with a short summary in English at the end) presents a laundry list of shortcomings and errors attributed to the study's investigators and sponsoring institutions, including disregard for scientific rules and regulations. “The study should never have been done this way,” says IGZ inspector-general Gerrit van der Wal.
In a statement responding to the report (in Dutch), the University Medical Center Utrecht (UMC), which sponsored the study, notes that the IGZ, the Dutch Central Committee on Research involving Human Subjects (CCMO), and the Food and Goods Authority (VWA) did not offer any explanation for the unexpected deaths. UMC also said that most of the shortcomings highlighted concerned data management and other procedural issues. Indeed, the IGZ admits in the report that it could not determine whether the problems it identified contributed to the extra deaths in the probiotics arm.
Among the issues the IGZ highlighted was that the informed-consent papers for the study stated there were no known complications to the probiotic formulation used, while the trial's official protocol acknowledged several. The report also contends that the privacy of patients was not sufficiently respected, and that some placebo batches were contaminated with probiotic bacteria. "The researchers were too ambitious and too optimistic," the IGZ concludes. It remains to be seen whether the government report offers enough fodder for malpractice lawsuits; the inspectorate stresses that most mistakes were “avoidable” but not necessarily due to negligence. At a meeting the IGZ arranged for surviving participants of the study and next of kin, nobody turned up.
The probiotic study caused a furor in the Netherlands in 2008, when the adverse outcome—24 deaths in the probiotics group (16%) and 9 deaths (6%) in the control group—was announced at a press conference 23 January, 3 weeks prior to publication in The Lancet. Immediately, concerns were raised about the design of the study and an interim analysis conducted halfway through the trial. Biostatistician Richard Gill of the University of Leiden, for example, has questioned whether the safety data had been appropriately analyzed and whether the use of triple-blinding, in which the safety data monitoring group didn’t know which group had received the treatment, was appropriate.
There was also a debate over whether the producer of the probiotics, Winclove, properly handled the product as a medicine to be tested in patients. It had presented the probiotic product to authorities as a “diet food for medicinal use”; in the Netherlands, as in many other countries, foods, unlike drugs, don’t require initial safety testing in animals and healthy people before being used for diseases.
Given all the questions, the Dutch minister of Health, Ab Klink, asked the IGZ to investigate both the causes of the extra deaths and the quality of the research. The IGZ concluded that UMC, which sponsored the study, the center's independent medical-ethical committee, and the university's data safety and monitoring committee all failed in one or more aspects. The IGZ notes, for example, that the decision to continue the study after the interim analysis was made in a meeting with three of the five members of the data safety committee absent, that the protocol to stop the trial in light of adverse events was not adhered to, and that study leaders merely crafted an ad hoc hypothesis to explain the differences in deaths in the two groups. “More could have been done to protect the safety of the subjects involved if a more critical approach to the assessment and reporting of the adverse events had been applied,” the report says.
The medical-ethical committee should also have insisted on prior studies of the probiotic in animals and healthy volunteers, according to the IGZ. The IGZ inspectors conclude that contrary to the claim of the authors in their Lancet article, the study was not done according to Good Clinical Practice (GCP) standards, an international series of guidelines for trials with human subjects.
Propatria's main investigator, Hein Gooszen, echoes UMC’s stance that serious mistakes were not made in the design or conduct of the study and that no errors have been identified that explain the extra deaths. According to the IGZ, Gooszen agreed to send a correction to the Lancet related to the study’s adherence to GCP guidelines, but the agency has not received any proof such a correction was sent. Gooszen, who recently left UMC for another post in Nijmegen, tells ScienceInsider that he feels that such a correction is not necessary as the study said it would follow the principles of GCP, not the exact rules.
The IGZ ends its report with several recommendations to tighten the rules for non-industry-sponsored studies by university scientists. And, it says, probiotics used in a clinical setting should be considered as drugs with the same legal requirement to report adverse events during a trial. “We cannot rule out the possibility that other similar investigations leave things to be desired. We will randomly check the 300 or so nonindustry trials with food supplements now running in the Netherlands,” says van der Wal. According to UMC, many of the recommendations from the IGZ report have already been implemented.