Abstract

This paper examines the issues related to conflict of interest (COI) in generation and dissemination of evidence from systematic reviews and its influence on evidence in developing public health policy. Several examples exist on COI in the health and nutrition field due to the influence of private corporations and funding institutions. COI is an important factor contributing to publication bias in primary studies because of dynamics such as delayed publication, suppression of negative findings, and falsifying of data, thus influencing systematic review findings. Systematic review findings have also been found to be biased because of financial and/or non-financial COI. A set of recommendations, such as increased government funding towards research, explicit COI policies in journals, clinical trial data transparency, and methodological guidelines, including COI compliance while conducting and reporting systematic reviews, is proposed. The government has a larger role in regulating COI in production and reporting of evidence and its use in public policy decision-making.

Keywords: Conflict of interest, Cochrane, systematic reviews, health technology assessment, public health policy

Introduction

Any policy process seeks to discover “what works”, “for whom”, and “at what cost” through scientific appraisal of all studies conducted on the issue and aims to be unbiased and free from conflicts of interest (COIs) of authors and funding organisations. However, recent controversies raised around the use of vaccines, statins, drugs for psychiatric disorders, therapeutic foods in under-nutrition, and introduction of soda-tax to combat obesity have cast doubts on the neutrality of evidence (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12). The recent Cochrane controversy has raised questions about the impartiality of evidence in policy-making. This paper attempts to examine COIs in scientific publishing and their potential to impact public health policy.

In May 2018, Cochrane published its review of the vaccines being used to prevent human papilloma virus (HPV) infections that could lead to cervical cancer. Based on 26 clinical trials, the review concluded that “there is high-certainty evidence that HPV vaccines protect against cervical precancer in adolescent girls and young women aged 15 to 26…. we did not find an increased risk of serious adverse effects. The deaths reported in the studies have been judged not to be related to the vaccine”. (13). On May 9, the British Medical Journal (BMJ) published a news item about the review titled “HPV vaccines are effective and safe and work best in young women, review finds” (14). Several responses in the form of blogs and letters followed challenging the conclusion of the review (15, 16, 17, 18).Questions were also raised regarding the connections of the lead author with the pharmaceutical industry, which Cochrane denied (19, 20). In July, Jorgensen et al identified 206 studies, of which 145 were industry-funded, and concluded that the conclusions of the earlier review were unreliable as it did not take all the available studies into consideration (1). In the study published by BMJ Evidence-Based Medicine, the authors raised, among others, issues related to exclusion of eligible trials, use of placebos and comparators, incomplete assessment of adverse events, industry funding, and COI (1). Following this, the Cochrane Board dismissed Peter Gotzsche, one of the founding members of the Cochrane Collaboration and co-author with Jorgensen, which attracted wide-spread criticism and resulted in the resignation of several board members of Cochrane (21). The Cochrane controversy has raised issues related to reliability of evidence – about who conducts the research or reviews, who sponsors or funds it, whether it is published in peer reviewed journals, and what values and judgements inform their work.

Research and conflicts of interest

COI is defined as, “a financial or intellectual relationship that may impact an individual’s ability to approach a scientific question with an open mind” (22, 23). ). Researchers can be pressurised to change the design of the study, the methodology, or even the results (24, 25). In the health and nutrition sectors, research is often conducted or sponsored by pharmacological and food and beverage companies, resulting in potential COI, which may influence the results and/or conclusions of the research (26, 27, 28, 29, 30). Often, professors of reputed universities and institutions are funded by the food and beverages industry to conduct research, which then carries the weight of academic authority and gets included in scientific literature. (31, 32, 33). Companies such as Nestle Nutrition, BASF, PepsiCo, Hinsdale Farms, American Vineyard Foundation, Iowa Soybean Association, United Soybean Board, American Cattlemen’s Association, and National Pork Board are also among those who fund the food science departments and chairs of various universities (34); Pfizer is an important funder of biosciences in several universities (34). In 2018, Fabbri et al claimed that the results of such research are often distorted to meet the requirements of the funder (35). Similarly, the results and conclusions of research studies funded by the pharmaceutical and medical devices industry have also been reported to be tilted in favour of the products (36, 37). In a recent blog, Dr Howard White, Chief Executive Officer of The Campbell Collaboration, stated that “the problem is that drug companies, and the researchers in their pay, don’t play fair. They don’t register their trials, they suppress studies showing the ‘wrong result’, they selectively report outcomes in the studies they do publish, and they support ghost written articles – that is studies written by an industry consultant but published under the name of academics who are paid to allow their name on the paper to give the impression of independence.” (38).

Government research institutes are also increasingly being funded by the food and pharmaceutical industries (39, 40). In India, Pfizer has recently entered into a collaboration with the Indian Council of Medical Research (ICMR), the country’s premier government-funded research organisation on health and nutrition, to conduct surveillance on antimicrobial resistance to “collect national data, guide treatment practices and rationalize antibiotic use in the country” (41). The Bill and Melinda Gates Foundation (BMGF) is the largest funder for GAVI, the Vaccine Alliance, which facilitates the increased use of vaccines in national health programmes in developing countries. According to information on its website, BMGF also invests in vaccine makers (42) and is also the largest non-government funder of the World Health Organization (WHO) (43). Such funding raises serious questions on the neutrality of research by public-sector organisations that accept funds from the industry.

COIs in systematic reviews

Systematic reviews summarise and integrate large numbers of research studies, which may often be contradictory, to provide “credible” evidence for both clinical practice and public health policy-making because of COIs . As these reviews are based on published papers, bias on the part of researchers, editors, and reviewers can impact their conclusions.

Martinson and colleagues listed 10 behaviours of researchers that distort scientific knowledge; these include, among others, falsifying data, subjective selection of data, selective reporting of data, non-disclosure of involvement with firms whose products are based on one’s own research, relationships that may be questionable in terms of COI, and changing the design, methodology, or results because of pressure from the funder (24). A meta-analysis of 39 surveys on scientific misconduct, as admitted by scientists or informed by their colleagues, revealed that scientists often indulge in distorting scientific knowledge by fabricating or falsifying data (25); the latter is particularly hard to detect. Assessing the quality of research published in orthopaedic journals, Parsons et al noted that over a third of the studies were of low quality (44).

Editors and reviewers also contribute to publication bias, which can be said to refer to those situations that can affect the decision to publish a study or not (45). The term includes publishing more studies where the results are positive or statistically significant, delaying reviews and publication of studies with negative results, which affects their dissemination, being influenced by factors such as the country of origin of the study, academic institutions related to the research, or participation in corporate groups (46, 47, 48, 49, 50, 51, 52, 53).

Studies sponsored by commercial entities have a greater chance of being published. Firstly, more clinical trials are sponsored by industry than otherwise. Secondly, such sponsored studies more often present positive outcomes (54, 55, 56). A 2013 study of 17 systematic reviews investigated the association between consumption of sugar-sweetened beverages (SSB) and weight gain or obesity; the results revealed that reviews in which a potential COI was disclosed were five times more likely to present a conclusion of no positive association between SSB consumption and weight gain than reviews that reported having no financial COI (28). Ebrahim et al found that in nearly two-thirds of the meta-analyses related to antidepressants that they examined for COIs, the authors had industry links (57).

The International Committee of Medical Journal Editors (ICMJE) announced in 2004 that it would require registration of clinical trials as a condition for publication. However, this is not being consistently followed. Viergever and Li note that, while registration has increased in many countries, many trials, especially in lower middle-income countries (LMICs) and low-income countries are not registered (58). A more recent study of registered clinical trials in India noted that of the studies registered with the Clinical Trials Registry – India (CTRI) from July 2009 to June 2015, all were incomplete when measured against WHO criteria (59).

Non-publication reduces the effectiveness of search when conducting systematic reviews (60). The Helsinki Declaration of the World Medical Association states: “Researchers, authors, sponsors, editors and publishers all have ethical obligations with regard to the publication of the results of research. Researchers have a duty to make publicly available the results of their research on human subjects and are accountable for the completeness and accuracy of their reports…. Negative and inconclusive as well as positive results should be published or otherwise made publicly available” (61: Sec 36).

Editors of journals, as gatekeepers in publication, are trustees of public good. However, reducing the risk of bias poses several challenges, including publication bias, which covers reporting bias and delayed publication of studies. Viergever and Li also note that in a significant number of cases, the results are not published (58). Further, of the 2938 studies listed in the CTRI, publication details of only 78 studies were mentioned (59).

In spite of the use of rigorous methods and standards, systematic reviews themselves can involve COIs, both financial and non-financial. A recent Cochrane review on financial conflicts of interest in systematic reviews (such as funding by drug or device companies or author’s collaboration with such companies) reported favourable conclusions towards the drug/device in focus and also had lower methodological quality than systematic reviews without financial COI (62). However, the study also pointed out the uncertainty of these financial COIs being associated with the results of these systematic reviews.

While financial COIs are easier to identify, non-financial COIs, such as personal relationships (including those that may be adversarial), institutional relationships, personal or professional beliefs, and desire for academic recognition or advancement are more difficult to identify and manage, as meta-analyses and systematic reviews usually only require the disclosure of financial COIs.

Cochrane defines COI as “a set of conditions in which professional judgement concerning a primary interest (such as patients’ welfare or the validity of research) may be unduly influenced by a secondary interest (such as financial gain) or may be perceived to be influenced by a secondary interest.” It further requires authors and reviewers to submit a Declaration of Interests at every stage of the review and its publication (63). The Declaration form for potential COIs also has a section on relationships (64, 65). Given the presence of COIs in health and nutrition research, the manipulation of data, the preponderance of positive results in published studies, and the non-availability of raw data from clinical trials, the “evidence” generated by systematic reviews can be lop-sided. Saini et al (2014) found that 86% of 92 Cochrane reviews did not include data of harm outcomes (66). Jefferson and Jorgensen term the results of such studies and reviews based on them as “garbage in, garbage out” (67).

In addition to COIs, decisions such as those regarding framing of research questions, selection of studies, methods of data extraction and analysis, selection of assessment criteria, and interpretation of results are subjective, based on the reviewer’s judgement, and can compromise the conclusions. A team of reviewers, after discussions amongst themselves, can reduce such compromise.

Evidence from systematic reviews forms part of the evidence for public health policy decision-making from health technology assessments (HTA) – which is “a systematic evaluation of properties, effects, and/or impacts of healthcare technology” (68). A review of the UK HTA programme systematic reviews published in 2004 and 2014 was found to be poor in addressing or acknowledging publication bias when compared with Cochrane and non-Cochrane reviews (69).

While generating evidence using systematic review in an HTA, a COI could cause an individual to be biased in interpreting evidence or formulating findings and recommendations. However, in recent times, agencies such as the National Institute of Clinical and Healthcare Excellence (NICE) and the Agency for Healthcare Research and Quality (AHRQ) of the US Department of Health and Human Services have issued specific guidelines for declaring and managing COIs for HTA researchers and HTA committees (70, 71, 72). While acknowledging that non-financial COIs cannot be eliminated altogether, the AHRQ’s fairly detailed questionnaire attempts to identify such conflicts (71).

How important are COI systematic reviews for public health policy decision-making?

Over the last decade, systematic reviews have been used to inform public health policy-making. However, they reflect the lack of importance given to diseases that afflict LMICs, which account for most of the world’s population and global disease burden. A primary reason for this is lack of funds to either conduct research or build research capacity in these countries. Health budgets in LMICs form a fraction of the national budget. LMICs are mostly dependent on official development assistance (ODA) and funding organisations such as BMGF and GAVI for conducting research on health. WHO’s Global Observatory on Health Research and Development data from 2018 reveals the inequalities that exist in investment for research and development (R&D) globally (73). For instance, only four high income countries in Africa received funding for R&D. Of these, Seychelles received almost eight times the average amount received by other African countries, most of which are LMICs (74). There should be space and encouragement for private agencies for funding health research; however, there should also be legislation/regulation of such financing in ways that preclude/avoid COI.

COI in systematic reviews in the context of LMICs do not necessarily indicate financial COI; it can refer to the dissonance that may exist between a national health priority and the priority of the funding organisation. For instance, research on vaccines received US$ 11.67 billion in 2016, while R&D on medicine received US$ 5.93 billion; within this, R&D on HIV/AIDS received 11.743 million, followed by tuberculosis (TB) and malaria, which received approximately half this amount respectively (73). Diseases which have the potential to cause epidemics, such as Ebola and pneumonic plague do not find mention in the list (72). Given the paucity of research conducted in diseases that plague LMICs, there are few systematic reviews related to them. Furthermore, research funded by non-governmental sources is mostly directed towards efficacy of new drugs. As these are chiefly tested against placebos rather than existing efficacious drugs (74), reviews of these studies may conclude that the new drug should be included in the treatment protocol. As the new drugs are generally costlier than the older drugs, out-of-pocket expenses of patients in LMICs may increase. Again, many small but scalable successful projects/implementation practices exist in LMICs, as elsewhere; however, health departments do not consider them viable as little research is done on them, nor are there systematic reviews.

Conclusion and recommendations

The Cochrane crisis highlights the values and judgements that underlie research as well as systematic reviewers and their potential for causing disagreements and contradictions among the results of meta-analysis on the same issue, carried out by different research teams (75, 76). Science and evidence are vital to public policy-making. While it may not be possible to eliminate all forms of COI, it is critical that research and guidelines and policy development have high standards of probity.

While it is clearly necessary that public health policy responds to the latest evidence regarding cures or prevention, it is equally essential to take into account both evidence of efficacy and ethical principles (do good, do no harm, respect, empower, sustain, be socially responsible, invite people’s participation) (77). The State, as the mandated provider of public good, must accept the principle of accountability and consider not just the potential benefits of each intervention but also the predictors of COIs and the potential adverse events that may be prompted by the use of such evidence for health policy decision-making.

Competing interests and funding support

The first author is not employed by any organisation. The second author is Convenor, Campbell and Cochrane Economic Methods Group (CCEMG); Member, Cochrane Advocacy Advisory Group; and currently working on 3 Cochrane reviews. He is full-time staff of Campbell South Asia, a regional and collaborative organisation of Campbell Collaboration. Campbell Collaboration is a partner organisation of Cochrane. Both authors have not received any financial support for developing this paper.