Conflict of interest in systematic reviews and its implications for public health policy
Radha Holla Bhar, Denny JohnPublished online on October 1, 2019
This paper examines the issues related to conflict of interest (COI) in generation and dissemination of evidence from systematic reviews and its influence on evidence in developing public health policy. Several examples exist on COI in the health and nutrition field due to the influence of private corporations and funding institutions. COI is an important factor contributing to publication bias in primary studies because of dynamics such as delayed publication, suppression of negative findings, and falsifying of data, thus influencing systematic review findings. Systematic review findings have also been found to be biased because of financial and/or non-financial COI. A set of recommendations, such as increased government funding towards research, explicit COI policies in journals, clinical trial data transparency, and methodological guidelines, including COI compliance while conducting and reporting systematic reviews, is proposed. The government has a larger role in regulating COI in production and reporting of evidence and its use in public policy decision-making.
Keywords: Conflict of interest, Cochrane, systematic reviews, health technology assessment, public health policy
Any policy process seeks to discover “what works”, “for whom”, and “at what cost” through scientific appraisal of all studies conducted on the issue and aims to be unbiased and free from conflicts of interest (COIs) of authors and funding organisations. However, recent controversies raised around the use of vaccines, statins, drugs for psychiatric disorders, therapeutic foods in under-nutrition, and introduction of soda-tax to combat obesity have cast doubts on the neutrality of evidence (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12). The recent Cochrane controversy has raised questions about the impartiality of evidence in policy-making. This paper attempts to examine COIs in scientific publishing and their potential to impact public health policy.
In May 2018, Cochrane published its review of the vaccines being used to prevent human papilloma virus (HPV) infections that could lead to cervical cancer. Based on 26 clinical trials, the review concluded that “there is high-certainty evidence that HPV vaccines protect against cervical precancer in adolescent girls and young women aged 15 to 26…. we did not find an increased risk of serious adverse effects. The deaths reported in the studies have been judged not to be related to the vaccine”. (13). On May 9, the British Medical Journal (BMJ) published a news item about the review titled “HPV vaccines are effective and safe and work best in young women, review finds” (14). Several responses in the form of blogs and letters followed challenging the conclusion of the review (15, 16, 17, 18).Questions were also raised regarding the connections of the lead author with the pharmaceutical industry, which Cochrane denied (19, 20). In July, Jorgensen et al identified 206 studies, of which 145 were industry-funded, and concluded that the conclusions of the earlier review were unreliable as it did not take all the available studies into consideration (1). In the study published by BMJ Evidence-Based Medicine, the authors raised, among others, issues related to exclusion of eligible trials, use of placebos and comparators, incomplete assessment of adverse events, industry funding, and COI (1). Following this, the Cochrane Board dismissed Peter Gotzsche, one of the founding members of the Cochrane Collaboration and co-author with Jorgensen, which attracted wide-spread criticism and resulted in the resignation of several board members of Cochrane (21). The Cochrane controversy has raised issues related to reliability of evidence – about who conducts the research or reviews, who sponsors or funds it, whether it is published in peer reviewed journals, and what values and judgements inform their work.
Research and conflicts of interest
COI is defined as, “a financial or intellectual relationship that may impact an individual’s ability to approach a scientific question with an open mind” (22, 23). ). Researchers can be pressurised to change the design of the study, the methodology, or even the results (24, 25). In the health and nutrition sectors, research is often conducted or sponsored by pharmacological and food and beverage companies, resulting in potential COI, which may influence the results and/or conclusions of the research (26, 27, 28, 29, 30). Often, professors of reputed universities and institutions are funded by the food and beverages industry to conduct research, which then carries the weight of academic authority and gets included in scientific literature. (31, 32, 33). Companies such as Nestle Nutrition, BASF, PepsiCo, Hinsdale Farms, American Vineyard Foundation, Iowa Soybean Association, United Soybean Board, American Cattlemen’s Association, and National Pork Board are also among those who fund the food science departments and chairs of various universities (34); Pfizer is an important funder of biosciences in several universities (34). In 2018, Fabbri et al claimed that the results of such research are often distorted to meet the requirements of the funder (35). Similarly, the results and conclusions of research studies funded by the pharmaceutical and medical devices industry have also been reported to be tilted in favour of the products (36, 37). In a recent blog, Dr Howard White, Chief Executive Officer of The Campbell Collaboration, stated that “the problem is that drug companies, and the researchers in their pay, don’t play fair. They don’t register their trials, they suppress studies showing the ‘wrong result’, they selectively report outcomes in the studies they do publish, and they support ghost written articles – that is studies written by an industry consultant but published under the name of academics who are paid to allow their name on the paper to give the impression of independence.” (38).
Government research institutes are also increasingly being funded by the food and pharmaceutical industries (39, 40). In India, Pfizer has recently entered into a collaboration with the Indian Council of Medical Research (ICMR), the country’s premier government-funded research organisation on health and nutrition, to conduct surveillance on antimicrobial resistance to “collect national data, guide treatment practices and rationalize antibiotic use in the country” (41). The Bill and Melinda Gates Foundation (BMGF) is the largest funder for GAVI, the Vaccine Alliance, which facilitates the increased use of vaccines in national health programmes in developing countries. According to information on its website, BMGF also invests in vaccine makers (42) and is also the largest non-government funder of the World Health Organization (WHO) (43). Such funding raises serious questions on the neutrality of research by public-sector organisations that accept funds from the industry.
COIs in systematic reviews
Systematic reviews summarise and integrate large numbers of research studies, which may often be contradictory, to provide “credible” evidence for both clinical practice and public health policy-making because of COIs . As these reviews are based on published papers, bias on the part of researchers, editors, and reviewers can impact their conclusions.
Martinson and colleagues listed 10 behaviours of researchers that distort scientific knowledge; these include, among others, falsifying data, subjective selection of data, selective reporting of data, non-disclosure of involvement with firms whose products are based on one’s own research, relationships that may be questionable in terms of COI, and changing the design, methodology, or results because of pressure from the funder (24). A meta-analysis of 39 surveys on scientific misconduct, as admitted by scientists or informed by their colleagues, revealed that scientists often indulge in distorting scientific knowledge by fabricating or falsifying data (25); the latter is particularly hard to detect. Assessing the quality of research published in orthopaedic journals, Parsons et al noted that over a third of the studies were of low quality (44).
Editors and reviewers also contribute to publication bias, which can be said to refer to those situations that can affect the decision to publish a study or not (45). The term includes publishing more studies where the results are positive or statistically significant, delaying reviews and publication of studies with negative results, which affects their dissemination, being influenced by factors such as the country of origin of the study, academic institutions related to the research, or participation in corporate groups (46, 47, 48, 49, 50, 51, 52, 53).
Studies sponsored by commercial entities have a greater chance of being published. Firstly, more clinical trials are sponsored by industry than otherwise. Secondly, such sponsored studies more often present positive outcomes (54, 55, 56). A 2013 study of 17 systematic reviews investigated the association between consumption of sugar-sweetened beverages (SSB) and weight gain or obesity; the results revealed that reviews in which a potential COI was disclosed were five times more likely to present a conclusion of no positive association between SSB consumption and weight gain than reviews that reported having no financial COI (28). Ebrahim et al found that in nearly two-thirds of the meta-analyses related to antidepressants that they examined for COIs, the authors had industry links (57).
The International Committee of Medical Journal Editors (ICMJE) announced in 2004 that it would require registration of clinical trials as a condition for publication. However, this is not being consistently followed. Viergever and Li note that, while registration has increased in many countries, many trials, especially in lower middle-income countries (LMICs) and low-income countries are not registered (58). A more recent study of registered clinical trials in India noted that of the studies registered with the Clinical Trials Registry – India (CTRI) from July 2009 to June 2015, all were incomplete when measured against WHO criteria (59).
Non-publication reduces the effectiveness of search when conducting systematic reviews (60). The Helsinki Declaration of the World Medical Association states: “Researchers, authors, sponsors, editors and publishers all have ethical obligations with regard to the publication of the results of research. Researchers have a duty to make publicly available the results of their research on human subjects and are accountable for the completeness and accuracy of their reports…. Negative and inconclusive as well as positive results should be published or otherwise made publicly available” (61: Sec 36).
Editors of journals, as gatekeepers in publication, are trustees of public good. However, reducing the risk of bias poses several challenges, including publication bias, which covers reporting bias and delayed publication of studies. Viergever and Li also note that in a significant number of cases, the results are not published (58). Further, of the 2938 studies listed in the CTRI, publication details of only 78 studies were mentioned (59).
In spite of the use of rigorous methods and standards, systematic reviews themselves can involve COIs, both financial and non-financial. A recent Cochrane review on financial conflicts of interest in systematic reviews (such as funding by drug or device companies or author’s collaboration with such companies) reported favourable conclusions towards the drug/device in focus and also had lower methodological quality than systematic reviews without financial COI (62). However, the study also pointed out the uncertainty of these financial COIs being associated with the results of these systematic reviews.
While financial COIs are easier to identify, non-financial COIs, such as personal relationships (including those that may be adversarial), institutional relationships, personal or professional beliefs, and desire for academic recognition or advancement are more difficult to identify and manage, as meta-analyses and systematic reviews usually only require the disclosure of financial COIs.
Cochrane defines COI as “a set of conditions in which professional judgement concerning a primary interest (such as patients’ welfare or the validity of research) may be unduly influenced by a secondary interest (such as financial gain) or may be perceived to be influenced by a secondary interest.” It further requires authors and reviewers to submit a Declaration of Interests at every stage of the review and its publication (63). The Declaration form for potential COIs also has a section on relationships (64, 65). Given the presence of COIs in health and nutrition research, the manipulation of data, the preponderance of positive results in published studies, and the non-availability of raw data from clinical trials, the “evidence” generated by systematic reviews can be lop-sided. Saini et al (2014) found that 86% of 92 Cochrane reviews did not include data of harm outcomes (66). Jefferson and Jorgensen term the results of such studies and reviews based on them as “garbage in, garbage out” (67).
In addition to COIs, decisions such as those regarding framing of research questions, selection of studies, methods of data extraction and analysis, selection of assessment criteria, and interpretation of results are subjective, based on the reviewer’s judgement, and can compromise the conclusions. A team of reviewers, after discussions amongst themselves, can reduce such compromise.
Evidence from systematic reviews forms part of the evidence for public health policy decision-making from health technology assessments (HTA) – which is “a systematic evaluation of properties, effects, and/or impacts of healthcare technology” (68). A review of the UK HTA programme systematic reviews published in 2004 and 2014 was found to be poor in addressing or acknowledging publication bias when compared with Cochrane and non-Cochrane reviews (69).
While generating evidence using systematic review in an HTA, a COI could cause an individual to be biased in interpreting evidence or formulating findings and recommendations. However, in recent times, agencies such as the National Institute of Clinical and Healthcare Excellence (NICE) and the Agency for Healthcare Research and Quality (AHRQ) of the US Department of Health and Human Services have issued specific guidelines for declaring and managing COIs for HTA researchers and HTA committees (70, 71, 72). While acknowledging that non-financial COIs cannot be eliminated altogether, the AHRQ’s fairly detailed questionnaire attempts to identify such conflicts (71).
How important are COI systematic reviews for public health policy decision-making?
Over the last decade, systematic reviews have been used to inform public health policy-making. However, they reflect the lack of importance given to diseases that afflict LMICs, which account for most of the world’s population and global disease burden. A primary reason for this is lack of funds to either conduct research or build research capacity in these countries. Health budgets in LMICs form a fraction of the national budget. LMICs are mostly dependent on official development assistance (ODA) and funding organisations such as BMGF and GAVI for conducting research on health. WHO’s Global Observatory on Health Research and Development data from 2018 reveals the inequalities that exist in investment for research and development (R&D) globally (73). For instance, only four high income countries in Africa received funding for R&D. Of these, Seychelles received almost eight times the average amount received by other African countries, most of which are LMICs (74). There should be space and encouragement for private agencies for funding health research; however, there should also be legislation/regulation of such financing in ways that preclude/avoid COI.
COI in systematic reviews in the context of LMICs do not necessarily indicate financial COI; it can refer to the dissonance that may exist between a national health priority and the priority of the funding organisation. For instance, research on vaccines received US$ 11.67 billion in 2016, while R&D on medicine received US$ 5.93 billion; within this, R&D on HIV/AIDS received 11.743 million, followed by tuberculosis (TB) and malaria, which received approximately half this amount respectively (73). Diseases which have the potential to cause epidemics, such as Ebola and pneumonic plague do not find mention in the list (72). Given the paucity of research conducted in diseases that plague LMICs, there are few systematic reviews related to them. Furthermore, research funded by non-governmental sources is mostly directed towards efficacy of new drugs. As these are chiefly tested against placebos rather than existing efficacious drugs (74), reviews of these studies may conclude that the new drug should be included in the treatment protocol. As the new drugs are generally costlier than the older drugs, out-of-pocket expenses of patients in LMICs may increase. Again, many small but scalable successful projects/implementation practices exist in LMICs, as elsewhere; however, health departments do not consider them viable as little research is done on them, nor are there systematic reviews.
Conclusion and recommendations
The Cochrane crisis highlights the values and judgements that underlie research as well as systematic reviewers and their potential for causing disagreements and contradictions among the results of meta-analysis on the same issue, carried out by different research teams (75, 76). Science and evidence are vital to public policy-making. While it may not be possible to eliminate all forms of COI, it is critical that research and guidelines and policy development have high standards of probity.
While it is clearly necessary that public health policy responds to the latest evidence regarding cures or prevention, it is equally essential to take into account both evidence of efficacy and ethical principles (do good, do no harm, respect, empower, sustain, be socially responsible, invite people’s participation) (77). The State, as the mandated provider of public good, must accept the principle of accountability and consider not just the potential benefits of each intervention but also the predictors of COIs and the potential adverse events that may be prompted by the use of such evidence for health policy decision-making.
Competing interests and funding support
The first author is not employed by any organisation. The second author is Convenor, Campbell and Cochrane Economic Methods Group (CCEMG); Member, Cochrane Advocacy Advisory Group; and currently working on 3 Cochrane reviews. He is full-time staff of Campbell South Asia, a regional and collaborative organisation of Campbell Collaboration. Campbell Collaboration is a partner organisation of Cochrane. Both authors have not received any financial support for developing this paper.
- Jorgensen L, Gotzsche PC, Jefferson T. The Cochrane HPV vaccine review was incomplete and ignored important evidence of bias. BMJ Evid Based Med. 2018 Oct; 23(5):163-8. doi: http://dx.doi.org/10.1136/bmjebm-2018-111012
- Puliyel J, Phadke A. Deaths following pentavalent vaccine and the revised AEFI classification. Indian J Med Ethics. 2017 Oct-Dec; 2(4): 300-2.
- DuBroff R, de Lorgeril M. Cholesterol confusion and statin controversy. World J Cardiol. 2015 Jul 26; 7(7): 404–9.
- de Lorgeril M, Salen P, Abramson J, Dodin S, Hamazaki T, Kostucki W, Okuyama H, Pavy B, Rabaeus M. Cholesterol lowering, cardiovascular diseases, and the rosuvastatin-JUPITER controversy: a critical reappraisal. Arch Intern Med. 2010 Jun 28;170(12):1032-6.doi: 10.1001/archinternmed.2010.184.
- Thibaut F. Controversies in psychiatry. Dialogues Clin Neurosci. 2018 Sep; 20(3): 151–2.
- Gotzsche PC, Young AH, Crace J. Does long term use of psychiatric drugs cause more harm than good? BMJ. 2015 May 12; 350:h2435. doi: 10.1136/bmj.h2435.
- Wise J. Cochrane distances itself from controversial views on psychiatric drugs. BMJ. 2015 Sep 23;351:h5073. doi: 10.1136/bmj.h5073.
- Stuckler D, Nestle M. Big food, food systems, and global health. PLoS Med. 2012 Jun; 9(6): e1001242. doi: 10.1371/journal.pmed.1001242.
- Erickson J, Slavin J. Are restrictive guidelines for added sugars science based? Nutr J. 2015 Dec 12; 14:124.
- Bhandari N, Mohan SB, Bose A, Iyengar SD, Taneja S. Mazumder S, Pricilla RA, Iyengar K, Sachdev HS, Mohan VR, Suhalka V, Yoshida S, Martines J, Bahl R. Efficacy of three feeding regimens for home-based management of children with uncomplicated severe acute malnutrition: a randomised trial in India. BMJ Glob Health. 2016 Dec 30; 1(4):e000144. doi:10.1136/bmjgh-2016-000144.
- Prasad V. Reading between the lines of the RUTF trial, India. BMJ Glob Health. 2017 Jan 31[cited 2018 Sep 30]. Available from: https://gh.bmj.com/content/1/4/e000144.responses#re-reading-between-the-lines-of-the-rutf-trial-india
- Garg CC, Mazumder S, Taneja S, Shekhar M, Mohan SB, Bose A, Iyengar SD, Bahl R, Martines J, Bhandari N. Costing of three feeding regimens for home-based management of children with uncomplicated severe acute malnutrition from a randomised trial in India BMJ Global Health. 2018;3:e000702.
- Arbyn M, Xu L, Simoens C, Martin-Hirsch PP. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Database Syst Rev. 2018 May 9; 5: CD009069. doi:10.1002/14651858.cd009069.pub3.
- Hawkes N. HPV vaccines are effective and safe and work best in young women, review finds. BMJ, 2018 May 9;36:k2059. doi:10.1136/bmj.k2059.
- Demasi, M. Cochrane – a sinking ship? Blog/BMJ EBM Spotlight; BMJ Evid Based Med. 2018 Sep 16 [cited 2019 Mar 23]. Available from: https://blogs.bmj.com/bmjebmspotlight/2018/09/16/cochrane-a-sinking-ship/
- Newman M. Has Cochrane lost its way? BMJ 2019 Jan 3; 364. doi: 10.1136/bmj.k5302 Available from: https://www.bmj.com/content/364/bmj.k5302.full
- Nass M, Noble JH. Whither Cochrane? Indian J Med Ethics. 2019 Jan-Mar; 4(1)NS: 54-8. doi: 10.20529/IJME.2018.092. Epub November 19, 2018. Available from: https://ijme.in/articles/whither-cochrane/?galley=html
- Healy D. Cochrane cock-up. Davidhealy.org. 2018 Sep 17 [cited 2018 Sep 30]. Available from: https://davidhealy.org/cochrane-cock-up/
- Gervas J. Twitter. 26 Sept 2018. Available from: https://twitter.com/JRBtrip/status/1044860118972665856.
- Hawkes N. Lead author of Cochrane HPV review did not breach conflicts policy, find arbiters. BMJ. 2018 Oct 16;363: k4352. doi:10.1136/bmj.k4352.
- Vesper I. Mass resignation guts board of prestigious Cochrane Collaboration. Nature. 2018 Sep 17 [cited 2019 Mar 23]. Available from: https://www.nature.com/articles/d41586-018-06727-0
- Lo B, Field MJ. Conflict of interest in medical research, education, and practice. Washington (DC): National Academies Press (US); 2009.
- Schünemann HJ, Osborne M, Moss J, Manthous C, Wagner G, Sicilian L, Ohar J, McDermott S, Lucas L, Jaeschke R. An official American Thoracic Society policy statement: managing conflict of interest in professional societies. Am J Respir Crit Care Med. 2009 Sep 15;180(6):564–580. doi: https://doi.org/10.1164/rccm.200901-0126ST.
- Martinson BC, Anderson MS, de Vries R. Scientists behaving badly. Nature. 2005 Jun 9; 435(7043):737–8.
- Fanelli D. How many scientists fabricate and falsify research? a systematic review and meta-analysis of survey data. PLoS ONE. 2009 May 29; 4(5): e5738. DOI 10.1371/journal.pone.0005738.
- Ehrhardt S, Appel LJ, Meinert CL. Trends in National Institutes of Health funding for clinical trials registered in ClinicalTrials.gov. JAMA. 2015 Dec 15; 314 (23): 2566 doi: 10.1001/jama.2015.12206
- Rasmussen K, Bero L, Redberg R, Gøtzsche PC, Lundh A. Collaboration between academics and industry in clinical trials: cross sectional study of publications and survey of lead academic authors. BMJ. 2018 Oct 3; 363. doi: 10.1136/bmj.k3654.
- Bes-Rastrollo, M, Schulze MB, Ruiz-Canela M, Martinez-Gonzalez MA. Financial conflicts of interest and reporting bias regarding the association between sugar-sweetened beverages and weight gain: A systematic review of systematic reviews. PLOS Med. 2013 Dec;10(12):e1001578. doi: 10.1371/journal.pmed.1001578
- Katzmarzyk PT, Broyles ST, Champagne CM, Chaput JP, Fogelholm M, Hu G, Kuriyan R, Kurpad A, Lambert EV, Maia J, Matsudo V, Olds T, Onywera V, Sarmiento OL, Standage M, Tremblay MS, Tudor-Locke C, Zhao P. Relationship between soft drink consumption and obesity in 9-11 years old children in a multi-national study. Nutrients. 2016 Dec; 8(12), 770. doi:10.3390/nu8120770
- Siri-Tarino PW, Qi Sun, Hu FB, Krauss RM. Meta-analysis of prospective cohort studies evaluating the association of saturated fat with cardiovascular disease. Am J Clin Nutr. 2010 Mar 1; 91 (3):535-46.
- Mars Centre for Cocoa Health Science. Partnerships. 2017[cited 2019 Mar 23]. Available from: https://www.marscocoascience.com/research/partnership
- O’Connor A. Coca-Cola Funds Scientists Who Shift Blame for Obesity Away From Bad Diets. New York Times, August 9, 2015 [cited 2019 Mar 23]. Available from: https://well.blogs.nytimes.com/2015/08/09/coca-cola-funds-scientists-who-shift-blame-for-obesity-away-from-bad-diets/?_r=0
- Rebello CJ, Johnson WD, Martin CK, Han H, Yi-Fang C, Bordenave N, van Klinken JW, O’Shea M, Greenway Fl. Instant Oatmeal Increases Satiety and Reduces Energy Intake Compared to a Ready-to-Eat Oat-Based Breakfast Cereal: A Randomized Crossover Trial. J Am Coll Nutr. 2016 Jan 2; 35(1): 41–49. doi: 10.1080/07315724.2015.1032442. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4674378/
- Food & Water Watch. 2012. Public research, private gain: Corporate influence over university agricultural research. Available from: https://www.foodandwaterwatch.org/sites/default/files/Public%20Research%20Private%20Gain%20Report%20April%202012.pdf
- Fabbri A, Holland TJ, Bero L. Food industry sponsorship of academic research: investigating commercial bias in the research agenda. Public Health Nutr. 2018 Dec; 21(18):3422-30. Available from: https://doi.org/10.1017/S1368980018002100/
- Norris P, Herxheimer A, Lexchin J, Mansfield P. Drug promotion: what we know, what we have yet to learn – Reviews of materials in the WHO/HAI database on drug promotion. World Health Organization and Health Action International. 2005[cited 2019 Mar 24]. Available from: http://apps.who.int/medicinedocs/pdf/s8109e/s8109e.pdf
- Jimbo M, Granberg CF, Osumah TS, Bandari J, Cannon GM, Routh JC, Gargollo PC. Discrepancies in Self-Reported and Actual Conflicts of Interest for Robotic Pediatric Urological Surgery. J Urol. 2019 Feb 1; 201(2): 393-9. doi: https://doi.org/10.1016/j.juro.2018.07.043. Available from: https://www.auajournals.org/doi/abs/10.1016/j.juro.2018.07.043
- White H. Conflict of Interest in Campbell and Cochrane Reviews: Threat or Opportunity? Campbellcollaboration.org. 2018 Nov 13[cited 2019 Mar 23]. Available from: https://campbellcollaboration.org/blog/conflict-of-interest-in-campbell-and-cochrane-reviews-threat-or-opportunity.html
- Coca Cola website. Our commitment to transparency. 2019 Jul 26[cited 2019 Mar 23]. Available from: https://www.coca-colacompany.com/transparency/our-commitment-transparency.
- CDC Foundation. Fiscal year 2018 – Report to contributors. Date unknown [cited 2019 Mar 23]. Available from: https://www.cdcfoundation.org/FY2018/organizations.
- Hindu Business Line Bureau. Pfizer-ICMR collaborate to tackle anti-microbial resistance in India. thehindubusinessline.org. 2018 Jan 31[cited 2019 Mar 23]. Available from: https://www.thehindubusinessline.com/companies/pfizer-icmr-collaborate-to-tackle-anti-microbial-resistance-in-india/article22607938.ece
- Bill and Melinda Gates Foundation. Portfolio. Strategic Investment Fund. 2019 [cited 2019 Mar 23]. Available from: https://sif.gatesfoundation.org/portfolio/?fwp_investment-type=direct-equity-investment
- Reddy SK, Mazhar S, Lencucha R. The financial sustainability of the World Health Organization and the political economy of global health governance: a review of funding proposals. Global Health. 2018 Nov 29;14(1):119. Available from: https://globalizationandhealth.biomedcentral.com/articles/10.1186/s12992-018-0436-8
- Parsons NR, Hiskens R, Price CL, Achten J, Costa ML. A systematic survey of the quality of research reporting in general orthopaedic journals. J Bone Joint Surg Br. 2011 Sep; 93(9):1154-9.
- Dickersin K, Olson CM, Rennie D, Cook D, Flanagin A, Zhu Q, Reiling J , Pace B. Association between time interval to publication and statistical significance. JAMA. 2002 Jun 5; 287(21):2829-31.
- Fanelli D. Negative results are disappearing from most disciplines and countries. Scientometrics. 2012 Mar; 90(3):891–904. https://doi.org/10.1007/s11192-011-0494-7.
- Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev. 2009 Jan 21;(1):MR000006. doi: 10.1002/14651858.MR000006.pub3.pmid:19160345.
- Matías-Guiu J, Moral E, García-Ramos R, Martínez-Vila E. The profile of evaluators of a medical publication in relation to the response. Neurología. 2010; 25:530-5.
- Yousefi -Nooraie R, Shakiba B, Mortaz-Hejri S. Country development and manuscript selection bias: a review of published studies. BMC Med Res Methodol. 2006; 6:37.
- Patel V, Sumathipala A. International representation in psychiatric literature: survey of six leading journals. Br J Psychiatry. 200;178:409.
- Okike K, Kocher MS, Mehlman CT, Heckman JD, Bhandari M. (2008). Nonscientific factors associated with acceptance for publication in The Journal of Bone and Joint Surgery. J Bone Joint Surg Am. 2008 Mar;90(3):2432-7.
- Ross JS, Gross CP, Desai MM, Hong Y, Grant AO, Daniels SR, Hachinski VC, Gibbons RJ, Gardner TJ, Krumholz HM. Effect of blinded peer review on abstract acceptance. JAMA. 2006; 295:1675-80.
- Figg WD, Dunn L, Liewehr DJ, Steinberg SM, Thurman PW, Barrett JC, Birkinshaw J. Scientific collaboration results in higher citation rates of published articles. Pharmacotherapy. 2006; 26:759-67.
- Lynch JR, Cunningham MR, Warme WJ, Schaad DC, Wolf FM, Leopold SS. Commercially funded and United States-based research is more likely to be published; good-quality studies with negative outcomes are not. J Bone Joint Surg Am.2007 May; 89(5):1010-8.
- Ahn R, Woodbridge A, Abraham A, Saba S, Korenstein D, Boscardin WJ, Keyhani S. Financial ties of principal investigators and randomized controlled trial outcomes: cross sectional study. BMJ. 2017 Jan 17; 356:i6770. doi: 10.1136/bmj.i6770.
- Mandrioli D, Kearns CE, Bero LA. Relationship between Research Outcomes and Risk of Bias, Study Sponsorship, and Author Financial Conflicts of Interest in Reviews of the Effects of Artificially Sweetened Beverages on Weight Outcomes: A Systematic Review of Reviews. PLoS One. 2016; doi: 10.1371/journal.pone.0162198.
- Ebrahim S, Bance S, Athalke A, Malachowski C, Ioannidis JPA. Meta-analyses with industry involvement are massively published and report no caveats for antidepressants. J Clin Epidemiol. 2016 Feb;70:155-63. doi: 10.1016/j.clinepi.2015.08.021.
- Viergever RF, Li K. Trends in global clinical trial registration: an analysis of numbers of registered clinical trials in different parts of the world from 2004 to 2013. BMJ Open. 2015 Sep 25;5(9):e008932. doi: 10.1136/bmjopen-2015-008932.
- John D, Stanley A, Gudi N, Rao MV. Trial publication after registration in Clinical Trials Registry of India (CTRI): A cross-sectional analysis of randomized controlled trials. 2018 Sep17 [cited 2019 Mar 23]. Cochrane Colloquium, Edinburgh. Available from:https://colloquium2018.cochrane.org/abstracts/trial-publication-after-registration-clinical-trials-registry-india-ctri-cross-sectional
- Malicki M, Marusic A, OPEN Consortium O. Is there a solution to publication bias? Researchers call for changes in dissemination of clinical research results. J Clin Epidemiol. 2014 Oct;67(10):1103–10. doi: https://doi.org/10.1016/j.jclinepi.2014.06.002.
- Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Subjects. 2008, World Medical Association; 2008 [cited 2019 Mar 23]. Available from: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/
- Hansen C, Lundh A, Rasmussen K, Hrobjartsson A. Financial conflicts of interest in systematic reviews: associations with results, conclusions, and methodological quality. Cochrane Database Syst Rev. 2019 Aug 5[cited 2019 Sep 19];8: MR000047. Available from: 10.1002/14651858.MR000047.pub2
- Cochrane. Commercial sponsorship policy. Cochrane policy on commercial sponsorship of Cochrane Reviews and Cochrane Groups. 2019 [cited 2019 Mar 23]. Available from: https://community.cochrane.org/organizational-info/resources/policies/commercial-sponsorship-policy.
- Lasserson T, Churchill R, Chandler J, Tovey D, Higgins JPT. Standards for reporting of protocols of new Cochrane intervention reviews (PR1-44) (undated). In: Higgins JPT, Lasserson T, Chandler J, Tovey D, Churchill R. Methodological Expectations of Cochrane Intervention Reviews. Cochrane: London; July 2019 [cited 2019 Mar 23]. Available from: https://community.cochrane.org/mecir-manual/standards-reporting-protocols-new-cochrane-intervention-reviews-pr1-44/reporting-review-plan-pr1-44/declarations-interest-pr43.
- Cochrane Community. Conflict of interest policy for Cochrane Groups. Updated 2014 [cited 2019 Mar 23]. Available from: https://community.cochrane.org/organizational-info/resources/policies/conflict-interest-policy-cochrane-groups.
- Saini P, Loke YK, Gamble C, Altman DG, Williamson PR, Kirkham JJ. Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. BMJ. 2014 Nov 21; 349:g6501. doi:10.1136/bmj.g6501 pmid: 25416499.
- Jefferson T, Jorgensen L. Redefining the ‘E’ in EBM. BMJ Evid Based Med. 2018; Apr 23(2):46-7.
- Banta D. What is technology assessment? Int J Technol Asses Health Care. 2009 Jul; 25:7-9.
- Fava GA. Meta-analyses and conflict of interest. CNS Drugs. 2012 Feb; 26: 93–6.
- National Institute for Health and Care Excellence. Policy on declaring and managing interests for NICE advisory committees. Available from: https://www.nice.org.uk/Media/Default/About/Who-we-are/Policies-and-procedures/declaration-of-interests-policy.pdf
- Vishwanathan, M, Carey TS, Belinson SE, Berliner E, Chang S, Graham E, Guise J-M, Ip SS, Maglione MA, McCrory D, McPheeters M, Newberry SJ, Sista P, White MC. Identifying and Managing Nonfinancial Conflicts of Interest for Systematic Reviews. Rockville, MD; Agency for Healthcare Research and Quality; 2013 May [cited 2019 Mar 23]. Available from https://www.ncbi.nlm.nih.gov/books/NBK148586/
- Carroll C, Kaltenthaler, E. Nature and reporting characteristics of UK health technology assessment systematic reviews. BMC Med Res Methodol. 2018 May 8; 18(1):35 10.1186/s12874-018-0498-6.
- World Health Organization. One year on, Global Observatory on Health R&D identifies striking gaps and inequalities. Geneva: WHO; 2019 Feb 18[cited 2019 Mar 23]. Available from:https://www.who.int/features/2018/health-research-and-development/en/
- Dietz H P. Bias in research and conflict of interest: why should we care? Int Urogynecol J. 2007 Mar; 18(3):241–3 DOI https://doi.org/10.1007/s00192-006-0236-1.
- Greenhalgh T. The Cochrane Collaboration – what crisis? The BMJ Opinion. Sept. 17, 2018 [cited 2019 Mar 23]. Available from: https://blogs.bmj.com/bmj/2018/09/17/trish-greenhalgh-the-cochrane-collaboration-what-crisis/
- Novoa A. Greenhalgh is wrong about Cochrane crisis: it’s about science and democracy, not damage control. Nogracias.eu. 2018 Sep 17 [cited 2018 Sep 30]. Available from: http://www.nogracias.eu/2018/09/17/greenhalgh-is-wrong-about-cochrane-crisis-its-about-science-and-democracy-not-damage-control/
- Tannahill A. Beyond evidence—to ethics: a decision-making framework for health promotion, public health and health improvement. Health Promot Int. 2008 Dec 1; 23(4):380–90. doi: 10.1093/heapro/dan032.