The crisis in Cochrane: Evidence Debased Medicine
Published online: November 5, 2018
The mission of the Cochrane Collaboration, established in 1993, was to systematically review medical evidence with a view to producing the best quality and trustworthy evidence. Twenty-five years later, it is in a crisis that centres on the dismissal one of its founders and the question of access to clinical trial data. The original mission aimed at improving health. In the face of stalling life expectancies, the stakes in the current crisis could not be higher. This essay looks at the crisis in the context of the disastrous effects of medication for paediatric depression on children as a consequence of the suppression of adverse findings from clinical trials.
The first article by Iain Chalmers announcing the Cochrane Collaboration appeared in 1992 (1), with its mission being to systematically review medical evidence with a view to producing the best quality and trustworthy evidence (2). Writing The Antidepressant Era in 1995, I characterised systematic reviews as a logical, and necessary medical development (3). Although the founders came from Canada (Sackett and Enkin), the United States (Dickersin), Denmark (Gøtzsche) and elsewhere, from the mid-1990s, the United Kingdom (UK) became the home of the Collaboration. From the very start, there was a tension between a renegade disruptive element in Cochrane and an establishment function (2).
The idea of embodying Evidence Based Medicine (EBM) in Guidelines also took shape at this time. In Britain, in 1997, a Labour government created a National Institute for Clinical Excellence (NICE) which began issuing Guidelines underpinned by Cochrane methods and in some instances with Cochrane collaboration. The NICE process was and still is highly regarded, sufficiently so for the Labour government to issue a new plan for Britain’s National Health Service (NHS) which, on the basis of newly-minted standards of care, set about standardising the health service in a manner that embraces continuity of data with an interchangeability of personnel, rather than continuity of care (4).
In 2004, a world no-one anticipated came into view. As part of an FDA review of paediatric antidepressant trials at this point, it became clear that all trials in paediatric depression were negative, that all published studies were ghost or company written, in all cases the data were inaccessible and in the case of the published studies, the publications were at odds with the data regulators revealed. The data on both benefits and harms was systematically distorted in publications even in the leading medical journals (5). This came to a head over the issue of suicide in 2004, when New York State filed a fraud action against GlaxoSmithKline (GSK), primarily on the basis that a ghost-written publication of Study 3291 claimed paroxetine worked for and was safe for children who were depressed, when in an internal review it had recognised it didn’t work and had opted to pick out the good bits of this study and publish them (6).
This led reviewers within NICE, then compiling Guidelines for the treatment of paediatric depression, to publish an editorial “Depressing research” which raised a question as to whether it was possible in the circumstances revealed by these trials to undertake systematic reviews or write guidelines (7).
The issue of lack of access to the data and ghost writing of publications was therefore “known” within the Cochrane Collaboration and guideline apparatus as of 2004. This is not a feature of paediatric antidepressant trials alone, as what had been revealed appears to be standard industry operating mode (8).
Cochrane, NICE, and other guideline bodies, however, suppressed this awareness. Peter Gøtzsche, and later Tom Jefferson, have been the exceptions to this rule. Beginning in 2009, Gøtzsche began to lobby the European ombudsman for access to clinical trial data, and put the issue of access to data on the map. Jefferson, with others, chased missing studies on Tamiflu and as studies came to light, he and colleagues progressively revealed a picture of vanishing efficacy2 for this drug (9).
This process has led both Gøtzsche and Jefferson to encourage Cochrane reviewers to work from internal company Clinical Study Reports (CSRs) in addition to publications, and, latterly, as the issue of treatment-related harms has become more salient, to question whether reviews are possible without the data. Their efforts have received support from many, but not all, their colleagues.
Faced with stonewalling by regulators, the guideline apparatus, mainstream medicine, journals, and very little support, it has taken distinct personal qualities on the part of both Gøtzsche and Jefferson to pursue this course. Both men have called things as they are when others have been unwilling to do so. In the case of Gøtzsche, these personal qualities, shared with others among the renegade element present from the start, appear to have provided a basis for Cochrane to expel him in September 2018.
In 2012, the Cochrane Collaboration dropped the word Collaboration and became a more managed entity concerned with its brand – Cochrane™. As a director of a Cochrane centre and a Cochrane council member, Gøtzsche came into regular contact with the new management. His forthright manner alienated some of the organisation’s management.
Allied to this, from 2012 onwards, Gøtzsche had increasingly called attention to the hazards of antidepressants (10) and several weeks before the board meeting that led to his expulsion, Gøtzsche and Jefferson had publicly branded a Cochrane review of HPV vaccines as untrustworthy and as a betrayal of Cochrane’s core mission (11).
This provoked a crisis. Cochrane board members split over Gøtzsche’s expulsion. Almost half the board resigned. A large number of Cochrane centres around the world wrote expressing their support for Gøtzsche.
Cochrane centres are not funded by Cochrane™. They generate their own funds from national or provincial governments or other sources. This gives the directors of these centres a certain independence. For directors, however, the calculations as to what to do are not simple in all cases. Centre directors have “mouths to feed”. While supporting Gøtzsche might not initially lead to difficulties, some directors appear to believe that it opens them up to being pushed aside if another group sets up in their area and attracts the funding on the basis of an affiliation with the central organisation.
Cochrane and its directors face a crisis. Every decision has consequences.
In 2016, Jeremy Hunt, Britain’s then Minister of Health, stated that children’s mental health was the greatest point of failure of the NHS (12). As of 2016, senior personnel in NICE had a de facto policy of not sharing a platform with anyone who might state that their Guidelines were based on ghost written articles and were prepared without access to the data that outside observers in general assume underpin them.
In 2018, children and their apparently deteriorating mental health was a regular and prominent feature in North American and European news features. BBC ran a primetime flagship television programme (13) and a radio programme (14) on the issue of children’s mental health and the use of antidepressant medicines. Both programmes were briefed on the contents of an article then in press (15), which outlined that as of 2018, it appears that every single one of the 30 RCTs of antidepressants undertaken in childhood depression, involving over 10,000 children, have been negative on their primary outcome measures, and all appear to show an excess of suicidal events on active treatment compared to placebo. Both programmes were made aware of data from the Centers for Disease Control (CDC) that despite the results of these studies, antidepressants now appear to be the most commonly used drugs by teenage girls except for oral contraceptives (16).
Both programmes were told that Prozac (fluoxetine) had been licensed for use in paediatric depression by American and European regulators in 2001 on the back of two negative trials. The licensing took place before concerns about pediatric antidepressants became widely known in 2004. From 2004 onwards, regulators and guideline bodies have continued to state that the pediatric fluoxetine trials are positive, when in fact, on their primary outcome measures they are negative and as with other treatments there were more suicidal acts on fluoxetine compared to placebo – in one trial 34 suicidal acts on fluoxetine compared to three on placebo but these data are effectively hidden (17).
Both programmes balked at airing these issues. One (13) of the two made it clear that they had made enquiries of NICE in respect of the Prozac data and that NICE had refused to comment.
As outlined above, there is nothing unusual about paediatric depression. The evidence in this domain is produced in the same way as in any other medical domain.
As of 2018, the BMJ and other journals have carried several articles on falling or stalling life expectancies in several developed countries (18, 19). There is no generally accepted explanation for this. A possible contributing factor lies in the fact that more than 50% of people over the age of 45 in the USA are now on three or more medicines and more than 45% of over 65s are on five or more medicines (20). These data, allied to evidence that reducing medication burden to five medicines or less per day has the potential to reduce hospitalisation rates and extend life span, in addition to improving quality of life (21), suggest that poly-prescribing is having a detrimental effect on our overall health.
The current figures for medication consumption are almost certainly driven by a hyping of the benefits of medicines and hiding of their harms in ghost-written articles accompanied by a lack of access to the data from studies undertaken. If this has a comparable effect on the health of the population in general as it appears to be having on children’s mental health, then the current crisis in Cochrane represents a defining moment in modern medical history.
While every director of a Cochrane centre has a responsibility to the mouths they have to feed, how can the Cochrane organisation justify tolerating 15 years’ worth of reviews based on ghost-written articles and no scrutiny of trial data due to lack of access? Surely, this has been as deep a betrayal of the core Cochrane mission as it is possible to imagine.
1 See full details at: https://study329.org/
2 See also: https://www.bmj.com/tamiflu
- Chalmers I, Dickerson K, Chalmers TC. Getting to grips with Archie Cochrane’s Agenda. BMJ. 1992 Oct 3; 305(6857); 786-8.
- Cassels A. The Cochrane Collaboration. Medicine’s Best-kept Secret. Agio Publishing House, Vancouver, 2015.
- Healy D. The Antidepressant Era. Cambridge, Ma; Harvard University Press, 1997.
- Department of Health, UK. The NHS Plan. 2000 Jul 1 [cited 2018 Oct 30]. Available from; https://navigator.health.org.uk/content/nhs-plan-plan-investment-plan-reform-2000
- Keller MB, Ryan ND, Strober M, Klein RG, Kutcher SP, Birmaher B. et al. Efficacy of Paroxetine in the treatment of adolescent major depression: A randomized, controlled trial. J Am Acad Child Adolesc Psychiatry, 2001 Jul:40: 7. Available from: https://study329.org/wp-content/uploads/2015/02/Keller-et-al-329.pdf
- Sibbald B. Legal action against GSK over SSRI data. CMAJ. 2004 Jul 6 [cited 2018 Oct 30]. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC437671/
- Lancet Editorial. Depressing research. Lancet. 2004 Apr 24 [cited 2018 Oct 27]; 363(9418):1335. Available from: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(04)16080-7/fulltext
- Healy D. Pharmageddon. Berkeley, California: California University Press; 2012.
- Godlee F. We want raw data now. BMJ. 2009 Dec 10;339; b5405.
- Sharma T, Guski LS, Freund N, Gøtzsche PC. Suicidality and aggression during antidepressant treatment: systematic review and meta-analyses based on clinical study reports. BMJ. 2016 Jan 27;352:i65.
- Jørgensen L, Gøtzsche PC, Jefferson T. The Cochrane HPV vaccine review was incomplete and ignored important evidence of bias. BMJ Evid Based Med. 2018 Oct;23(5):165-8. doi: 10.1136/bmjebm-2018-111012. Epub 2018 Jul 27.
- Campbell D. Jeremy Hunt says children’s mental health services are the NHS’ biggest failing. theguardian.com. 2018 Oct 20 [cited 2018 Oct 28]. Available from: https://www.theguardian.com/society/2016/oct/20/jeremy-hunt-promises-better-mental-health-services-children-adolescents
- BBC One. The doctor who gave up drugs. Series 2. Available from: https://www.bbc.co.uk/programmes/b0b4nykn
- BBC Radio 4. Counting the cost: Antidepressant use in children. Available from: https://www.bbc.co.uk/sounds/play/b0bbq1gr
- Healy D, Le Noury J, Jureidini J. Paediatric antidepressants: Benefits and risks.. Int J Risk Saf Med. 2019; 30(1): 1-7.
- Pratt LA, Brody DJ, Gu Q. Antidepressant use among persons aged 12 and over: United States, 2011–2014. NCHS Data Brief. 2017 Aug; 283:1-8.
- Högberg G, Antonuccio DO, Healy D. Suicidal risk from TADS study was higher than it first appeared. Int J Risk Saf Med. 2015;27(2):85-91. doi: 10.3233/JRS-150645.’
- Hiam L, Harrison D, McKee M, Dorling. Why is life expectancy in England and Wales ‘stalling’? J Epidemiol Community Health 2018 May; 72(5):404-8. doi: 10.1136/jech-2017-210401.
- Ho JY, Hendi AS. Recent trends in life expectancy across high income countries. BMJ. 2018 Aug 15;362, k2562. doi: 10.1136/bmj.k2562.
- Health, United States, 2016. US Department of Health and Human Services. Centers for Disease Control and Prevention. 2017 May [cited 2018 Oct 29]. Available from: https://www.cdc.gov/nchs/data/hus/hus16.pdf
- Garfinkel, D, Mangin, D. Feasibility study of a systematic approach for discontinuation of multiple medications in older adults. Arch Intern Med. 2010 Oct 11;170 (18):1648-54. doi:10.1001/archinternmed.2010.355.