What will the Cochrane brand stand for?
John AbramsonPublished online: June 22, 2019 DOI: https://doi.org/10.20529/IJME.2019.032
Gotzsche PC, Death of a whistleblower and Cochrane’s moral collapse. People’s Press, 2019. 263 pp. Kindle edition, Rs 1330, ASIN: B07N927GXC.
The title of Peter Gøtzsche’s recent book, Death of a whistleblower and Cochrane’s moral collapse, forewarns readers they are about to embark on the telling of one side of an argument. Nonetheless, the book provides an important perspective on an episode that may stand as a landmark setback in healthcare professionals’ and the public’s practical access to unbiased and thorough evaluation of scientific evidence. The book describes the inherent tension—that erupted into an acute clash—between Gøtzsche’s relentless search for accuracy and completeness in the clinical assessment of drug efficacy and harms (describing himself as willing to be controversial) on the one hand, and Cochrane’s demand for institutional loyalty and protection of the “brand” on the other. As you have probably surmised, Gøtzsche came out on the short end of this conflict.
Back in 1993, Gøtzsche was one of the initial members of the Cochrane Collaboration and also established and became Director of the Nordic Cochrane Centre in Copenhagen. Twenty-five years later, in September 2018, he was summarily removed from both positions after six of the 13 members of the Cochrane Governing Board voted to expel him from the board. (One member abstained and Gøtzsche was not allowed to vote, thus six votes constituted a majority). This was followed by the resignation of four other directors who supported Gøtzsche. He was also dismissed from his university hospital position where he served as a clinical professor.
Although Cochrane’s legal Counsel framed the controversy as related to Gøtzsche’s “whole behavior” rather than any specific offense(s), there were three primary issues: First, Gøtzsche had persistently and repeatedly demonstrated “bad behaviour;” second, he had violated the Cochrane “spokesperson policy” by using Cochrane stationery to present non-Cochrane related professional opinions and otherwise failed to differentiate his personal opinions from those of the Cochrane organisation; and third, he co-authored an article published online by BMJ Evidence-Based Medicine in July 2018 that was critical of the Cochrane HPV vaccine review published two months earlier (1).
Gøtzsche’s telling of events is raw and the details are fine-grained, but necessary. He describes being caught in a power struggle between the “two wings” of Cochrane. The one led by Cochrane CEO Mark Wilson:
…advocated that everyone in Cochrane should speak with the same voice; opposed open scientific debates about the quality and reliability of concrete Cochrane reviews; emphasized “brand”, “our product” and “business” more than getting the science right; and allowed economic conflicts of interest in relation to the pharmaceutical industry.(Kindle location 4125)
The other, with which Gøtzsche identified:
… wanted to bring Cochrane back to its original values: Free scientific debate; no financial conflicts of interest for authors of Cochrane reviews in relation to the companies whose products they evaluate; and openness, transparency, democracy and cooperation.(location 4125)
Cochrane’s stated “mission is to promote evidence-informed health decision-making by producing high-quality, relevant, accessible evidence.”(2) This is accomplished by publication of readily accessible and practical Cochrane reviews, which—when trusted—provide clinicians and the public with the most complete and accessible summaries of the existing clinical trial evidence regarding efficacy and risk of harm associated with specific therapies. Maintaining this trust is, however, no easy task. Constant vigilance is required because most published randomised trials are commercially funded and such funding is associated with significantly more favourable outcomes and conclusions (3); and systematic reviews of clinical trials, guidelines, and setting of policy—according to John Ioannidis—”remain among the least evidence-based activities, impregnable strongholds of expert-based insolence and eminence-based innumeracy.”(4)
According to Gøtzsche, Cochrane CEO Mark Wilson engaged in dispiriting micromanagement, which included establishing targets for the number of clinical guidelines that were influenced by Cochrane reviews. In this context it is not hard to imagine the Cochrane management’s consternation over Gøtzsche and colleagues’ published critique of Cochrane’s HPV vaccine review. Among the criticisms offered by Gøtzsche and colleagues were: not including almost half of the eligible trials; failing to report that no trials used a true placebo; use of surrogate and composite endpoints; incompletely assessing adverse events and safety signals; and failing to inform readers that all trials included in the review had been industry funded, precluding determination of the influence of industry funding on outcomes. In addition, Gøtzsche and colleagues reported the announcement of Cochrane’s HPV vaccine review on Cochrane.org quoted favorable and uncritical comments from six experts, two of whom had financial ties to HPV vaccine makers.
These issues are not new. A classic example of the need for constant vigilance against commercial influence, not discussed in Gøtzsche’s book, is the 2006 Cochrane review of neuraminidase inhibitors for the prevention and treatment of influenza in healthy adults. Based on the available data, the review concluded that although the effectiveness of Tamiflu and similar drugs was too low to recommend their use for control of routine influenza outbreaks, “In a serious epidemic or pandemic, [Tamiflu and similar drugs] should be used with other public health measures.” (5) The review had relied heavily on a meta-analysis of 10 studies published in 2003 in the Archives of Internal Medicine, which showed that Tamiflu significantly reduced the risk of bronchitis and pneumonia, as well as the need for hospitalization when given to adults with acute flu-like symptoms (6, 7). By 2009, based at least in part on the 2006 Cochrane review, 96 countries had stockpiled enough Tamiflu to treat 350 million people in the event of a global pandemic (8). In 2009 the World Health Organization (WHO) added Tamiflu to its list of core essential medications (9).
During the 2009 swine-flu pandemic, a Japanese pediatrician, Dr. Keiji Hayashi, wrote to the Cochrane Collaboration after noticing that 8 of the 10 studies included in the 2003 meta-analysis (all sponsored by the drug’s manufacturer, Roche) had not been published in peer-reviewed journals and the results could not be verified (10). Dr. Tom Jefferson, lead author of the 2006 Cochrane review of neuraminidase inhibitors (and a co-author of the critique of the Cochrane HPV vaccine review in BMJ Evidence-Based Medicine), wrote to Roche requesting the full clinical study reports for all 10 of the trials included in the 2003 meta-analysis. Four years later, in 2013, Roche provided the full clinical study reports (CSRs) for all 77 of its trials of Tamiflu (11).
An updated Cochrane review published in 2014 (12), based on all CSRs sent by Roche reported the benefit of Tamiflu was minimal reduction of the duration of flu symptoms in adults from 7 to 6.3 days, with no evidence of reduction of the risk of serious complications, hospitalisation, or death. On the downside, adults treated with Tamiflu developed nausea (1 out of 28 people treated), vomiting (1 out of 22 people treated), headaches (1 out of 32 people treated), and psychiatric symptoms (1 out 94 people treated). The Cochrane review authors recommended that the “trade-off between benefits and harms should be borne in mind when making decisions to use [Tamiflu] for treatment, prophylaxis, or stockpiling.” In 2017, based at least in part on the evidence showing no reduction in hospitalisation or flu-related complications gleaned from the CSRs that Roche provided to the Cochrane reviewers, the World Health Organisation downgraded Tamiflu from its list of core essential medicines to a less-essential category (13).
Lesson learned? Evidently not. In 2013, the Cochrane statin review group recommended statin therapy for people at low risk of cardiovascular disease (14). This recommendation was based in large part on the Cholesterol Treatment Trialists (CTT) 2012 meta-analysis (15), even though the data upon which the CTT meta-analyses are based are submitted voluntarily (not all trials participate) and the data are held in “strict confidence,” meaning external verification of CTT analyses is not possible. In the denouement of the (unanimously rejected) demand for retraction of a paper I co-authored showing, based on the same Cholesterol Treatment Trialists (CTT) data, that statins do not significantly reduce mortality in low risk people (<20% 10-year risk) (16), BMJ editors wrote to the Cochrane statins review group inquiring about their willingness to analyse patient-level data from statins trials, should such data ever become available. The editors reported: “…disappointingly the [Cochrane statin review] group has shown no appetite to seek out the patient level data for its 2015 update.” (17)
These two examples show that Gøtzsche and colleagues’ concerns about alleged lapses in the Cochrane HPV vaccine review were not without context. Was Peter Gøtzsche too close to his rebuke by Cochrane to tell his story dispassionately? Investigative journalist and BMJ associate editor Jeanne Lenzer opines: “There was a catch-22 for Peter—wait, and the issues will be forgotten and/or moot, or write now with all the fire and fury of the moment. But hopefully out of this will come an effort to preserve what is important in medicine and research.” (18)
David Hammerstein, former Cochrane Governing Board member who voted against Gøtzsche’s expulsion and resigned afterwards, summarised the net effect of Gøtzsche’s banishment:
The big winner in this conflict has been the pharmaceutical industry, having succeeded in weakening the voice of one of its greatest critics and having consolidated a Cochrane leadership closer to industrial interests with fewer audible critical voices.(location 3746)
If Cochrane reviews are published without access to all data from relevant clinical trials or potentially compromised by commercial conflicts of interest, why should they be trusted any more than all the other publications known to be subject to industry bias? Is that a brand worth preserving? Peter Gøtzsche, by his words and actions, has taken a strong stand in support of improving the integrity, trustworthiness, and clinical value of Cochrane reviews. Will his efforts come to naught?
Disclosure: I serve as an expert adviser to plaintiffs’ attorneys in pharmaceutical litigation and am currently working on a book about the quality of information available to physicians..
- Jørgensen L, Gøtzsche PC, Jefferson T, The Cochrane HPV vaccine review was incomplete and ignored important evidence of bias. BMJ Evid-Based Med. 2018 Oct;23:165–8
- Cochrane. About us. Date unknown [cited 2019 May 28]. Available from: https://www.cochrane.org/about-us
- Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research outcome. Cochrane Database Syst Rev. 2017 Feb 16: 2. MR000033. DOI: 10.1002/14651858.MR000033.pub3
- Ioannidis JPA. Cochrane crisis: Secrecy, intolerance and evidence based values. Eur J Clin Invest. 2018;e13058.doi:10.1111/eci.13058
- Jefferson TO, Demicheli V, Di Pietrantoni C, Jones M, Rivetti D. Neuraminidase inhibitors for preventing and treating influenza in healthy adults. Cochrane Database Syst Rev. 2006 Jul 19;3:CD001265
- Doshi P. Neuraminidase inhibitors: the story behind the Cochrane review. BMJ. 2009 Dec 8;339:1348-51
- Kaiser L, Wat C, Mills T, Mahoney P, Ward P, Hayden F. Impact of oseltamivir treatment on influenza-related lower respiratory tract complications and hospitalizations. Arch Intern Med. 2003 Jul 28;163(4):1667-72
- Abbasi K. The missing data that cost $20bn. BMJ. 2014 Apr 10;348:g2695
- McCarthy M. What makes an essential medicine? WHO’s new list focuses on antibiotic esistance, adds expensive drugs, and downgrades Tamiflu. BMJ. 2017 Jul 7; 358:j3044. doi:10.1136/bmj.j3044.
- Jones M, Jefferson T, Doshi P, Del Mar C, Heneghan C, Onakpoya I. Commentary on Cochrane review of neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Clin Microbiol Infect. 2015 Mar; 21(3):217–21..
- Jefferson T, Doshi P. Multi system failure: the story of anti-influenza drugs. BMJ. 2014 Apr 9;348:g2263. doi:10.1136/bmj.g2263
- Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R, Thompson MJ, Spencer EA, Onakpoya IJ, Mahtani KR, Nunan D, Howick J, Heneghan CJ. Neuraminidase inhibitors for preventing and treating influenza in adults and children. Cochrane Database Syst Rev. 2014 Apr 10;4:CD008965. DOI: 10.1002/14651858.CD008965.pub4
- Kmietowicz Z. WHO downgrades oseltamivir on drugs list after reviewing evidence. BMJ. 2017 Jun 12;357:j2841 doi: 10.1136/bmj.j2841
- Taylor F, Huffman MD, Macedo AF, Moore TH, Burke M, Davey Smith G, Ward K, Ebrahim S. Statins for the primary prevention of cardiovascular disease. Cochrane Database Syst Rev. 2013 Jan 31;1:CD004816
- Cholesterol Treatment Trialists’ (CTT) Collaborators, Mihaylova B, Emberson J, Blackwell L, Keech A, Simes J, Barnes EH, Voysey M, Gray A, Collins R, Baigent C. The effects of lowering LDL cholesterol with statin therapy in people at low risk of vascular disease: meta-analysis of individual data from 27 randomised trials. Lancet. 2012 Aug 11;380(9841):581- 90
- Abramson JD, Rosenberg HG, Jewell N, Wright JM. Should people at low risk of cardiovascular disease take a statin? BMJ. 2013 Oct 22;347: f6123
- Parish E, Bloom T, Godlee F. Statins for people at low risk: Independent review of the trial data is an essential next step. BMJ. 2015;351:h3908
- Personal communication. May 20, 2019