Clinical trials: the pitfalls of interpretatio
Yash Lokhandwalla, Bharat Dalvi
Statistical information can be misleadingly presented
Advances are constantly being made in medical science. There has been a marked increase in the number of medical journals. The corporate sector has also entered this field in a big way. Many newer medical periodicals are being published by corporate houses. These are often distributed by post or by personal delivery to doctors. Pharmaceutical companies also increasingly mail “convenient” medical literature to the doctors. Medical representatives, often with half-baked information, try and enlighten doctors about newer drug formulations.
Every week there are conferences where new medicines are presented. While some of these meetings are organised by professional medical societies, an increasing number are promoted by the pharmaceutical industry. At times the speakers have extensive research and clinical experience with these drugs and have published their findings in reputed journals. Moreover, often the speakers are selected for the business they give to that company, and not for their scientific expertise. With the profusion of new drugs, drug delivery systems and combinations entering the market every day, it becomes impossible for a doctor to be aware of all of them.
Thus, sponsored conferences and corporate literature become the means whereby many doctors acquire knowledge about the latest preparations. In such an atmosphere, there is a strong likelihood of developing erroneous impressions about a new medicine.
During medical training there is in-depth teaching, discussion and peer review about the usage of drugs, especially the more potent, more hazardous and more expensive drugs. But once doctors enter private practice, such modes of honing one’s knowledge are no longer available.
Moreover, doctors’ access to reputed journals is limited, since these are very expensive. Doctors who can afford these journals are by and large very busy and do not have the time to read them, leave alone reviewing them in depth. They are often able only to glance through the summaries of the articles and make some impressions, which can be imprecise.
Just as statistics often hide more than they reveal, so also the titles and conclusions of some articles, even in reputed journals, are at variance with the data. However, it is tedious and time-consuming to go through the methods, results and statistical analysis described in articles, leave aside going through the references.
This, of course, is assuming that the data is honestly presented. There have been instances of frauds being discovered in medical research and publications. One imagines that many more falsehoods go undetected.
Thus in many instances one can find contradictory results of a particular drug in different trials. In such a scenario, if one wishes to promote that drug, one need quote only the convenient literature. As Shakespeare wrote, “The devil may cite scripture for his purpose.”
Clinical practice is greatly influenced by the recommendations of clinical trials, aided by incentives to doctors from the industry. If a new drug does not produce the expected effect, this fact is rarely noticed early. It may take months before the medical fraternity realises tat things are not as they are made to seem.
This is because, especially in India, there are very few regular prescription and efficacy audits, leave alone multicentric registries. By the time realisation dawns that a drug is not all that it is made out to be, the company already ahs a new drug in the market, so it does not mind withdrawing or dropping the old one, on which it has already made millions of rupees.
In fact, as Alvin Toffler wrote in his book Future Shock, we live in the age of transience. When a company launches a new product, its bigwigs know that very often it will last for only a few months. But the launch is mad eon the basis of calculations showing that enough profits will have been made in those few months, before the product is eventually eased off the market.
To put this whole issue in clearer perspective, we would like to review the use of certain drugs for heart disease. Myocardial infarction (heart attack) is caused by the clotting of blood in the arteries carrying blood, and thereby oxygen, to the heart muscle. If this clot does not dissolve within six hours, the part of the heart muscle being supplied by that artery will undergo irreversible damage.
Frauds are being discovered constantly, but many more go undetected
Thus medical treatment in these patients revolves around efforts to dissolve the clot as early as possible and restore the blood flow. IN Rambo terminology: “Time is muscle”.
Until just two decades ago, there were no potent medicines to dissolve the blood clot. Then a major breakthrough came along in the form of streptokinase, a drug that could dissolve the clot in at least 50 per cent of patients who received it. The use of streptokinase reduced the death rate from mhocardial infarction from 13 per cent to 10.7 per cent (1).
Streptokinase worked best if it could be given within three hours of the onset of chest pain. However, it was expensive and could produce bleeding in the brain and stomach in a few patients. Over the next decade, streptokinase became more easily available and with widespread use, its cost reduced in relative terms (considering inflation). This drug is now widely used in India.
Still, millions of dollars were put in by the pharmaceutical industry to develop a better drug that would dissolve the clot within the heart but not produce bleeding elsewhere.
Research in this direction yielded a medication called t-PA (tissue-type plasminogen activator). This was much more expensive than the by-now economical Streptokinase.
One trial found t-PA more effective, but the benefit was marginal
Several large-scale clinical trials were undertaken to study the relative efficacies of t-PA over streptokinase. One of the earliest reports (2) found that there was no difference between t-PA and streptokinase with respect to in-hospital mortality. Thus finding implied that both drugs seemed to be equally effective. Then why use t-PA, which was at least eight times more expensive than streptokinase? The relevant dosage of streptokinase would cost Rs.1,500 while t-PA would cost around Rs.15,000 in the early ’80s.
Over the next two years, two more major clinical trials were reported. These had enrolled more patients that the earlier trial. One of these trials (3) studied over 40,000 patients. Surprisingly it found that t-PA and streptokinase produced eqivalent results. Moreover, t-PA, not streptokinase, was more likely to cause bleeding.
Thus the conclusions reached by the two trials seemed to suggest that it was better and cheaper to use “good old streptokinase”.
This finding would have been a major blow to the economics of companies which had spent millions of dollars developing t-PA. Interestingly, many leading doctors had invested in shares of the companies manufacturing t-PA, a fact that came to light only later. Some of these doctors conducted trials which showed some advantage for t-PA over streptokinase.
Within a year, the results of another large trial (4) were pubished. This trial’s findings indicated that t-PA was more effective than streptokinase. However, the benefit was modest. A complex statistical analysis indicated that only one more patient was saved for every 100 treated with t-PA as compared to streptokinase. Also, there was a higher chance of producing bleeding in the brain when t-PA was used.
These conflicting results raised a major debate and controversy. Why did two trials show no added benefit of t-PA while the third showed otherwise? The answer: it all depended on the trial design, the specific dosages used, the population studied, the outcomes measure, the statistical methods used – all of which allowed the investigators or proponents of a particular medication to present results in a light favourable to ‘their’ drug. For example, proponents of t-PA argued that the suboptimal effects were due to the subcutaneous administration of another drug (Heparin), given with t-PA in the first two trials (GISSI 2, ISIS 3). Heparin given intravenously with t-PA, they argued, gave better results.
The debate became dramatic. Professor Peter Sleight from Oxford launched a campaign on behalf of streptokinase. At one major meeting in the USA to discuss this issue, he even went on stage barefoot in rolled-up trousers to show that he was the “Poor but scientific” counterpart of the wealthy (but industry-driven) American proponents of t-PA.
It was partly in response to this controversy that guidelines were laid down requiring researchers publishing trial results to unequivocally disclose all sources of funding.
One can now understand how necessary it is to critically examine data, bias, funding source, market factors, credibility of the investigators and statistical methods used before accepting the findings of any clinical trial.
Equally important, we must take our socio-economic realities and health priorities into account when embracing new therapies, especially if they are expensive.
In most hospitals in India streptokinase is widely used. At the expense of sounding harsh, we feel that any new therapy must be judged vis-a-vis its cost-effectiveness for society. One must ask oneself whether it makes sense to spend millions of rupees to save one life instead of utilising that money for larger primary care programmes, which eventually go to save many more lives. Even wealthy nations do such cost-benefit analyses to decide, for instance, whether heart transplantation, though feasible, should be encouraged. Therefore, especially where public funds are concerned, such analyses are very necessary in India.
- GISSI: Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Lancet I:397-402, 1986.
- GISSI-A factorial randomised trial of alteplase versus streptokinase and heparin versus no heparin among 12, 490 patients with acute myocardial infarction. Lancet 336:65-71, 1990.
- ISIS-A randomised comparison of streptokinase versus t-PA among 41,229 cases of suspected acute myocardial infarction. Lancet 339:753-70, 1992.
- The GUSTO Investigators: An international randomised trial comparing four thromobolytic strategies for acute myocardial infarction. N Engl J Med 329:673-82, 1773.