Introduction

Healthcare providers had a rude awakening at the beginning of the 21st century with increasing reports of medical errors and of variations in the quality and content of healthcare delivery according to the geographical location and specialty (1, 2, 3). Such reports prompted regulatory bodies, government agencies, and the public to demand more transparency and accountability regarding the quality of these services. One outcome has been that the United States’ healthcare agenda includes facilitating the adoption of evidence-based healthcare, i.e., the more rapid integration of new knowledge into clinical practice and policy. However, this must go hand in hand with improvements in the quality and clarity of the evidence. This will depend, at least in part, on policy makers, regulatory agencies, quality improvement agencies, and perhaps even institutional review boards (IRBs) establishing policies that make it mandatory for principal investigators to explain study results to clinicians and their patients in ways they can understand, and to apply the findings rapidly with less variation.

Three factors are responsible for existing variations in the understanding and integration of recent research findings into clinical care. They are: 1) the overwhelming and exponential growth of published research data; 2) conflicting data from multiple studies that may not be comparable; and 3) complicated and complex descriptions of research results. The consequence of this is that healthcare professionals, especially those in the community, have difficulty arriving at a clear understanding of how findings reported in the published literature should be applied in the clinical arena, which only increases the lag time between the acquisition of new knowledge and its application to clinical practice (4, 5).

Controlled clinical trials are the foundation for innovations and improvements in healthcare delivery, and they should retain this paramount position because they provide the maximum opportunity for demonstrating the benefits of new treatments. Indeed, academic institutions and pharmaceutical companies should be applauded for the amount of clinical trial research they do. However, as with other clinical research findings, often, the findings from these trials do not make their way quickly into clinical practice for the reasons given above- it is difficult for those in clinical practice to know how to apply the findings.

Therefore, just as researchers must comply with ethical guidelines in the conduct of research, clinical researchers (clinical trial experts, clinicians, and other researchers) and drug marketers also have a responsibility to report data and their interpretation in a manner that is useful and transparent so that clinicians can integrate these findings promptly into the care that they deliver. Journals that publish these results have an equal obligation to require the authors of clinical research articles to report their findings in a clear and transparent fashion by explicitly stating the level of evidence.

An important step in improving the reporting of clinical trial results has been the development of “The Consolidated Standard of Reporting Trials” (CONSORT) statement guidelines. This statement has now become a well-established tool for the reporting of randomised controlled trials (6). In fact, the CONSORT guidelines grew out of the recognised need to provide enough valid and meaningful information concerning the design, conduct, findings, and generalisability of the results from randomised clinical trials so that users could make use of the findings. These guidelines have been adopted by a variety of journals and are continuously being improved. Now there is good evidence to show that the quality of reports in journals that follow these guidelines has improved (7). Efforts are also being made to add requirements to the CONSORT guidelines stipulating that results be reported in simple terms, so that community physicians, other healthcare professionals, and the public can make use of the new knowledge rapidly.

Another important reason for transparency in research reporting is to enhance public knowledge and awareness. The lay public is, after all, the primary consumer of healthcare services. Indeed, patient participation in the decision-making process and in patient-physician communication is becoming an increasingly important component of healthcare delivery. Accurate information regarding the risks and benefits of available treatment options is now an integral and established part of a patient’s decision-making process. Thus, as with informed consent documents, study results should be described in language that can be understood by all. This becomes an even more important consideration when one realises that only about half of US adults read at or above an eighth-grade reading level (8), though this varies widely worldwide. One way to address this is for researchers, pharmaceutical companies, and reporters in the lay press to explain the benefits of treatment in simple frequency terms (e.g., the number of patients responding to a specific treatment out of 100 or 1,000 or 100,000 compared with the benefits and costs) instead of in complex statistical terms, such as relative risk (RR). This will increase the public’s understanding and knowledge of the treatment options available for various diseases. Such an approach more faithfully represents the true significance of the findings to the public.

Drawbacks of statistics, clinical trials, and the drug approval process

Because of the way that the results of clinical trials are reported, physicians are often left struggling to understand the relevance of the results to their patient population and the actual clinical significance of a trial result. Determining this requires a knowledge of disease prevalence, baseline risk estimates, the likelihood of a disease, and effect size. Adding to this is the fact that clinical trials use different statistical measures that, although complex, are meant to yield important details regarding the outcome of a study. However, these measures are often difficult for healthcare professionals, patients, and the public to distinguish, understand, and use. For example, clinical trialists use different measures to gauge the effectiveness of trial drugs, including relative risk reduction (RRR), absolute risk reduction (ARR), and number needed to treat (NNT). (See Appendix A for definitions.) In reports to the public and community physicians, most clinical trialists and drug companies report the results of a clinical trial in terms of RRR.

The following example illustrates the different conclusions that can be arrived at, depending on the reported measure. In a hypothetical clinical trial of an investigational new drug to treat hypertension and thereby prevent or reduce stroke events, the agent is being tested against a drug already on the market. If the stroke event rate in the group taking the investigational drug is 2 out of 100 patients, whereas it is 3 out of 100 patients in the group taking the approved drug during the study period, then, in reality, only one more patient benefits from using this new drug. However, such results from trials are usually reported as relative risk (RR), which in this case would be 0.67 (i.e., 2% divided by 3% = 67%). This is then construed as indicating that there is a 33% benefit, or RRR, from using the investigational drug instead of the approved drug. This would artificially inflate the significance of the finding.

In a more accurate appraisal, since the absolute difference is 1% and the NNT is 100/(3-2) = 100, one would need to treat 100 patients to see a benefit in a single patient. Further, although the RRR is largely constant across a range of absolute risks, among high-risk people, the ARR is higher and the NNT is lower. For example, if a patient’s baseline risk of stroke is 20% and the therapy is expected to reduce the risk by 33%, treatment with the new drug may reduce this risk to 13.4%, which would translate into a 6.6% ARR for that patient. However, in a patient with a 2% baseline risk, the relative risk remains the same but the absolute risk declines to 1.34%, which would translate into an ARR of less than 1% (0.66%). One would therefore need to treat 15 patients to prevent one stroke in the patients in the high-risk group and 152 patients to prevent one stroke in the low-risk group. (See Table 1 for calculation details.)

Another consideration is that ARR and NNT can be significantly different in two different trials even though the RRR is the same (0.67) (Table 2). This is illustrated in Table 2, in which the ARR in Trial A is about 10% and the NNT is one for every 10 patients treated with the new drug; this contrasts with the small ARR of less than 1% in Trial B. In addition, one needs to treat 155 patients with the drug tested in Trial B for one patient to benefit from the treatment.

Despite criticisms and concerns about NNT (9), it has many advantages – most important, it is easy to understand. The NNT is easily calculable as 100 divided by the ARR. The NNT represents the absolute difference between two proportions (derived from the respective benefit-risk of two different treatments). Finally, the NNT is one of the best-researched measures in statistics. If one evaluates the respective ability of absolute and relative numbers to gauge the benefits and risks of treatment, from the standpoint of research ethics, absolute numbers will always be the more reliable measure. Therefore, for community physicians and the public to be able to understand and use the results of clinical trials, researchers (10) should report trial results in terms of ARR and NNT and not just RRR.

The same variation in results can occur when evaluating alternatives in the context of health policy. In Table 3, we show the results of simple hypothetical scenarios of cost-effectiveness analyses involving sets of comparisons of two pharmaceutical agents. In all comparisons, Drug A is the standard of care, and Drug B is a new pharmaceutical agent. In all five scenarios, Drug B shows a 33% improvement in failure probability compared to Drug A. In the example, the pharmaceuticals are taken daily for one year. The daily cost of the old drug is $0.50 and the cost of the new drug is $2.00. The cost of a fatal event is $20,000, and if patients can avoid the fatal event in the first year, then they are assumed to live 20 more years. We used a discount rate of 3% in this calculation.

An additional consideration in the interpretation of the statistical data from clinical trials is that the incremental cost-effectiveness ratio (i.e., the difference in cost divided by the difference in effectiveness) varies depending on the baseline failure probability. As shown in the example in Table 3, if the new drug reduces the failure probability from 30% to 20%, the use of the new drug is a dominating strategy – the new drug is not only more effective, it is also cheaper to use it to avoid the high cost of death. However, if a drug reduces the failure probability from 3% to 2%, it is very cost-effective to use the drug, because the incremental cost-effectiveness analysis shows that it will cost only $2,268 to gain a life year. However, if there is initially a low failure probability, and the new drug reduces the failure probability from 3/10,000 to 2/10,000, the incremental cost-effectiveness ratio shows that it would then cost $356,000 per life year gained, which would clearly not be considered a cost-effective use of healthcare resources. Thus, despite the same RRR, there can be substantial differences in ARR leading to very different conclusions regarding cost-effectiveness.

These examples therefore show why the results of clinical trials should be presented in absolute terms (11). This will not only help healthcare providers and patients understand and make better use of the results, it will also advance evidence-based healthcare and overall healthcare quality.

Efficacy and effectiveness

Crucial to an understanding of the results of clinical studies, and especially clinical trials, is understanding the difference between two closely related concepts: efficacy and effectiveness. Efficacy refers to the chances that a drug or treatment will be beneficial for a certain condition based on findings from controlled clinical trials in specific clinical populations. Effectiveness, a less precise but essential measure, refers to how beneficial a drug or treatment is, based on observations in the general population and how well a treatment will perform in a practice setting where factors such as convenience, co-morbidities, resources, and tolerability influence healthcare decisions. Effectiveness is the extent to which specific clinical interventions do what they are intended to do, ie maintain and improve the health of patients securing the greatest possible health gain from the available resources (12). Evidence-based healthcare is about doing the right thing to the right person at the right time and is concerned with demonstrating improvements in quality and performance by optimising resources for healthcare and cost effectiveness. Effectiveness is about improving patients’ total experience of their healthcare. Therefore, only effectiveness reflects the generalised scope of an outcome by not restricting the use of treatment on the basis of exclusionary criteria. More details regarding the differences in these two concepts are presented in Table 4.

Irrespective of the differences between efficacy and effectiveness (13), both are essential considerations in patient care and are, in essence, two sides of the “quality” coin. However, efficacy is the important consideration in a controlled trial where the selection and randomisation of subjects into interventions are tightly controlled. Effectiveness is important when implementing clinical trial findings regarding efficacy into clinical practice, where the benefits of a new treatment plus the mediating and moderating factors in real-world situations must also be taken into consideration. Further, focus on effectiveness is particularly important to shortening the lag time between the acquisition of research findings and their application to clinical practice.

Clinical ethics and dissemination of research results

The increasing emphasis on clinical efficacy, effectiveness, and quality has brought clinical ethics into the forefront of discussions about evidence-based medicine and public policy (14). Clinical ethics, in essence, is concerned with improving the quality of patient care by addressing ethics-related issues and dilemmas that arise in clinical practice (15). One ongoing challenge in clinical practice is providing patients with the tools and information they need to make informed healthcare decisions. It is important to this process to have healthcare information that is accessible, understandable, accurate (especially regarding risks and benefits), and relevant to the healthcare decision at hand. Patients now access such healthcare information in a variety of ways. They may get it from healthcare professionals as part of an informed consent process (e.g., during outpatient visits or during hospitalisations), or they may get it from the print or electronic media (e.g., newspapers, magazines, journals, internet, radio, television). Regardless, how research results are communicated and disseminated to patients is not only clinically but also ethically relevant.

In addition to respecting and empowering patients to make their own healthcare decisions, minimising harm (duty of nonmaleficence) and maximising benefits (duty of beneficence) are integral components of both clinical practice and research. Consonant with these principles, those engaged in research have an ethical obligation to disseminate the results of their research in simple and transparent ways. Harmful or even fatal medical errors can be serious consequences of any misapplication of research results stemming from a lack of clear understanding of the results. The appropriate application of the knowledge gained from research to patients is the ultimate aim of clinical research. It is what endows it with the social value that, in the end, justifies clinical research (16).

Finally, participants in research studies have a right to expect that the aggregated data resulting from their participation are disseminated in clinically useful and understandable ways. That is, many research subjects volunteering to participate in studies of new therapies do so not only because they may benefit from the therapy, especially in the case of Phase III studies, but also because other patients with the same or similar illnesses may benefit from the knowledge gained. This takes on an even greater meaning when one considers the time patients spend and the risks they accept as volunteer research subjects.

Strategies for change

As was noted above, target audiences for research results that are both transparent and understandable should include community healthcare professionals and the lay public. However, the public dissemination of research results has traditionally been accomplished via peer-reviewed medical and scientific journals, and more recently via electronic publications. However, research results published in specialised journals are typically written at a level appropriate to the knowledge level of scientists and experts in the field – not at the level of the community physician or lay public (17). To overcome this problem, medical journals should require that the authors of research articles provide statistical presentations that can be comprehended easily by community healthcare professionals, while continuing to demand that research be conducted with scientific rigour and that scientific data undergo robust statistical analyses.

Academic institutions can also play a role in shortening the lag time between the acquisition and dissemination of research results. Because academic institutions usually consider research to be an important part of their mission and values, and because publications are a major consideration in determining faculty promotions, promotion committees could also require that publications reporting the efficacy of new treatments include a description of how the findings should be incorporated into clinical practice. Such a requirement could further help to fine-tune research results to suit care delivery by validating findings, by considering important patient characteristics that are unique to each setting.

Recently, healthcare journalists have joined the dialogue about advancing the public’s understanding of healthcare issues and information. Many such journalists are aware of the special challenges they face in covering complex healthcare topics and reporting on the clinical relevance of published research data (18). In this regard, the Association of Health Care Journalists (AHCJ) has paid a lot of attention to quantifying the magnitude of the benefits and risks in their stories-but only when the original published journal article included information on RR, absolute risk, and NNT. The AHCJ could play an even more active role, indeed a leading role, by additionally advocating for the more transparent dissemination of research results in original medical publications.

In 2004, the UK House of Commons Health Committee and the American Medical Association both recognised that selective reporting and publication bias are major problems that can threaten evidence-based healthcare and the clinical quality of care. The registration of clinical trials represents an attempt to eliminate such problems by providing a means of informing reviewers, physicians, and other stakeholders of trials that have been started. However, unless a comprehensive system is developed to track, organise, and disseminate information about all ongoing clinical trials (19), positive study results will continue to dominate the literature. The mandatory registration, dissemination, and follow up of harmful side effects could be considered as a priority when allocating resources for clinical research. The developments regarding the selective serotonin reuptake inhibitor paroxetine (Paxil) represent a case for the importance of early trial registration and transparency in the conduct and findings of clinical trials (20).

Conclusions and recommendations

The tremendous benefits that derive from medical advances in terms of diagnosis, treatment, and survival can be further improved by shortening the lag time between research and practice. Simplifying research reports such that they can be easily understood by clinicians and the lay public is an important part of this effort. After all, human subjects who participate in clinical trials often do so in the belief that they are helping improve the welfare of all. They already trust their healthcare providers and the research process, and this can only be further enhanced by increasing the transparency and simplicity with which research results are reported.

Internationally, improvements in healthcare delivery and the implementation of evidence-based healthcare also depend on innovative and transparent programs in related clinical fields such as patient care, quality, ethics, and research. Professionals in these areas have a responsibility at individual, group, and organisational levels to provide accurate, complete, meaningful, and timely information important to healthcare research results, processes, and outcomes.

a. Relative risk (RR) = Event rate (Drug) / Event rate (Placebo).b. Relative risk reduction (RRR) = 1- Relative risk x 100.c. % Absolute risk reduction (ARR) = % Event rate (Placebo) – % Event rate (Drug).d. Number needed to treat (NNT) = 100 / % absolute reduction.

Appendix: Review of clinical epidemiology terms

Relative risk reduction (RRR) is simply the ratio of events in the treatment group and control group. Relative risk measures have the advantage of being stable across populations with different baseline risks and are, for instance, useful when combining the results of different trials in a meta-analysis. However, they have the major disadvantage of not reflecting the baseline risk of the individual with regard to the outcome being measured. Therefore, they do not give a true reflection of how much benefit the individual would derive from the intervention, as they cannot discriminate between small and large treatment effects.

Absolute risk reduction (ARR) or the risk difference is the difference between the risk of an event in the control group and the risk of an event in the treatment group. The advantages of ARR are that it is easy to compute; the confidence interval obtained is easy to interpret; it reflects both the underlying risk without treatment and risk reduction associated with treatment; and it has a clear meaning that makes it appealing to the practitioner. Absolute risk measures overcome the drawbacks of RRR because they reflect the baseline risk and are better at discriminating between small and large treatment effects. Despite the obvious advantages of absolute risk measures, because they are dependent on baseline risk, they are of limited generalisability. It would, for example, be inappropriate to extrapolate published absolute risk measures from one population to another population with a different baseline risk.

The number needed to treat (NNT)is the reciprocal of ARR. The meaning of this measure is the number of patients that need to be treated to obtain the desired outcome in one patient who would not have benefited otherwise. NNT takes into account the absolute benefit and is meaningful because it addresses both statistical and clinical significance. It is also worth noting that the numerical value of NNT is a function of the disease, the intervention, and the outcome.

An intention to treat (ITT) analysis is generally interpreted as an analysis including all patients, regardless of whether they actually satisfied the criteria of assignment, the treatment was actually received, or they subsequently withdrew or deviated from the protocol. ITT helps retain the benefit of randomisation in that it helps in making comparisons between groups. ITT may benefit effectiveness regardless of clinical efficacy. ITT also minimises bias with respect to dropouts related to outcome and simplifies the task of dealing with suspicious outcomes, all of which can protect against attempts to drive the results in a desirable direction. ITT reflects the way treatments will be performed in the population by ignoring adherence when the data are analysed.