Quality and completeness of data documentation in an investigator-initiated trial versus an industry-sponsored trial
Soumil Patwardhan , Nithya Gogtay, Urmila Thatte, C S Pramesh
Literature on the quality and completeness of data and documentation in investigator-initiated research studies is scarce. We carried out a study to compare the quality of data and documentation in an investigator-initiated trial (IIT) with those in an industry-sponsored study. We retrospectively studied the archived data pertaining to 42 patients enrolled in two trials, 14 patients in an industry-sponsored study and 28 randomly selected patients from an IIT. Trial-related documents were examined and scored for the completeness of the acquisition of data and for storage as per a pre-formulated checklist. Weighted scores were given for each point on the checklist proportional to its relative importance in the data documentation process. A global score and sub-scores for specific modules were given for each subject. The scores in the two studies were compared using the Mann Whitney U test. The total score for general documents was similar in the IIT (14/14, 100%) and the sponsored study (24/25, 96%). The mean summary global score obtained for study-specific documents (maximum possible score, 32) in the IIT (27.1; 95% CI 26.4-27.8) was also not significantly different from that in the sponsored study (27.9; 95% CI 26.7-29.1; p=0.1291). Thus, investigatorinitiated studies carried out by independent researchers in highvolume academic centres, even without active data monitoring and formal audits, appear to adhere to the high standards laid out in the International Conference on Harmonisation-Good Clinical Practices guidelines, ensuring accuracy and completeness in data documentation and archival.
The basic tenets of the International Conference on Harmonisation-Good Clinical Practices (ICH-GCP) guidelines include the accurate capture, storage and reporting of data (1). The importance of these guidelines in the context of providing vital data from clinical trials for statistical analysis has been stressed by other studies as well (2, 3). However, despite these guidelines, research has shown that intensive data monitoring and adherence to data documentation are not yet a part of the conduct of clinical trials (4). The documentation of data and accuracy of reporting are subject to regular monitoring, audits and inspections in an industry-funded study. Investigatorinitiated research, on the other hand, is not subject to such rigorous monitoring and may never undergo a formal audit or an inspection. Several publications have highlighted the importance of, and raised concerns over, adherence to methodological quality and the quality of reporting of research, both in industry-funded and investigator-initiated trials (5, 6).
Despite an extensive search of the literature, we were unable to find any study which compared the quality of documentation in investigator-initiated research with that in industry-funded research. Hence, we conducted a study to compare the quality and completeness of documentation in an investigatorinitiated randomised trial of two types of lymph node dissections for oesophageal cancer (with one of the authors, C S Pramesh, as the principal investigator) with an open-label industry-funded study of the use of granulocyte colonystimulating factor in the treatment of febrile neutropenia induced by cancer chemotherapy.
We initiated the study after obtaining the approval of the institutional ethics committees of the institutions concerned. Both the sponsored and investigator-initiated studies were conducted at the Tata Memorial Hospital, which is a highvolume tertiary referral centre for the treatment of cancer in India. The principal investigators in both studies were experienced in clinical trials and had already participated in more than 10 research studies (both IITs and industrysponsored trials). A request was made for the waiver of consent and this was granted on the grounds that the study was retrospective and no confidential information about the patients would be made public. Unique identifiers, which were pre-coded, were used for all patients. Prior to the initiation of the study, permission was taken from the principal investigators of both studies, as well as the sponsor of the funded study.
In the industry-funded study, a total of 14 patients were recruited from the Tata Memorial Hospital. In the investigatorinitiated study, 28 patients were randomly selected from a total of 200 patients with the help of a random number table. The ratio for comparison between the two studies was thus 1:2. We drew up a checklist consisting of various criteria that are important in trial documentation. We classified these as general documents (Table 1) and patient-related documents (Tables 2-4). All were examined and scored for completeness of data acquisition and archiving. Each point on the checklist was given a weighted score, which was summated into a global score for each patient. We also calculated sub-scores for specific modules in the documentation (Table 4). As the investigator-initiated study was a surgical trial, some parameters were not applicable and, therefore, were not considered for scoring. The data were checked for normality using the Kolmogorov Smirnov test and analysed using the Mann Whitney U test. Graph Pad InStat, Version 3.10 was used for all analyses which were done at 5% significance.
The scores for general documents are listed in Table 1. The maximum score possible for the investigator-initiated study was 14 and that for the sponsored study, 25. The IIT scored 14/14 (100%). and the sponsored study, 24/25 (96%) Thus, there was barely any difference with respect to the general documents. Table 2 provides the details of the patientrelated documents and is subdivided into (i) admission criteria, (ii) informed consent form, (iii) case record form, (iv) source documents, and (v) protocol and safety reporting. The maximum score was 28 for the IIT and 14 for the sponsored study for all the subdivisions, except the subdivision of the reporting of serious adverse events (SAEs) to the ethics committee, sponsor, regulator and follow-up reporting, for which the maximum score was 56 in the case of the IIT and 28 in the case of the sponsored study. Thus the maximum possible score for the investigator-initiated study was 896 and that for the sponsored study was 448. The scores obtained were 760/896 (84.8%) and 390/448 (87%), respectively. Table 3 gives the global score for patient-related documents. It may be seen that the score for the investigator-initiated study ranges from 23.5 to 29.5 and that for the sponsored study from 23.5 to 30. The mean summary global score for patient-related documents (Table 3) obtained in the investigator-initiated study (27.1; 95% CI 26.4-27.8) was also not significantly different from that in the sponsored study (27.9; 95% CI 26.7-29.1); (p=0.1291, Mann Whitney U test). The points assigned for individual parameters, the sub-scores for specific modules and the global score obtained for each trial are shown in Table 4. While the global scores were not significantly different, a difference was seen in the sub-score of informed consent, with the investigator-initiated study scoring 78.3% and the sponsored study scoring 84.8%.
The ICH-GCP has provided guidelines for data documentation, archival and reporting (1); however, adherence to these is not as frequently emphasised as adherence to guidelines relating to ethics and the actual conduct of the study. This might be a bigger problem in investigator-initiated research, as monitoring is not as strict as in sponsored studies. A large investment, in terms of finances and resources, has to be made by investigators in order for trials to strictly comply with the GCP guidelines (7). Sponsored studies employ qualified and trained staff, as well as monitors, to carry out intensive reviews and arduous monitoring of the proceedings of a trial. In investigator-initiated and non-commercial studies, the existing staff members are usually responsible for monitoring and review, which they carry out together with their regular duties, as cost constraints do not permit the use of additional manpower. Despite the hurdles that the investigator has to face, there is no alternative to accurate and diligent data collection, storage and reporting, the absence of which can significantly alter statistical analysis and interpretation, thus affecting the results of the trial (8).
General trial-related documents: checklist, points for sub-scores and scores of both studies
|S. no.||Criterion||Investigatorinitiated research||Pharmaceuticalsponsored research|
|1b.||Final approved version of protocol||1||1|
|1c.||IEC approval of above||1||1|
|1d.||Protocol signed by PI||1||1|
|1e.||Protocol amendments, if any||1||1|
|1f.||IEC approval of protocol amendments||1||1|
|Total score for protocol parameters = 6||6/6||6/6|
|2||Final version of case report form||1||1|
|3||Final approved version of informed consent form||1||1|
|4||Final approved versions of Marathi and Hindi translations of the informed consent form||1||1|
|5||Investigator’s brochure – presence of approved versions||NA||1|
|6||IEC approval of investigator’s brochure||NA||1|
|7||Clinical trial agreement – signed and dated||NA||1|
|8||Investigator’s undertaking – signed and dated||NA||1|
|12||Work distribution log||1||0|
|13||Address details of laboratory||NA||1|
|14||Laboratory – normal values||NA||1|
|15||Documentation of laboratory procedure||NA||1|
|16||Approved subject diary||NA||1|
|17||Translation of subject diary in Hindi and Marathi||NA||1|
|18||Other relevant documents||1||1|
|19||CVs of study personnel||1||1|
|20||Investigational product – information accountability||1||1|
|(Maximum score, 14)||(Maximum score, 25)|
Patient-related documents: checklist and scores for individual items
|S. no.||Parameter||Score (for each patient)||Maximum possible score||Calculated score|
|Investigator-initiated trial (with 28 patients)||Sponsored trial (with 14 patients)||Investigatorinitiated trial||Sponsored trial|
|2||Informed consent form|
|3||Case report form|
|5||Protocol and safety|
|5a||Documentation of deviations/ violations||1||28||14||28||14|
|5b||EC reporting of deviations/ violations||1||28||14||28||14|
|5c||AEs documented in source notes and CRFs||1||28||14||26.5||13|
|5d||SAEs documented in source notes and CRFs||1||28||14||27||13|
|5e||SAE reporting to EC, sponsor, regulator and follow-up reporting&||2||56||28||52||25.5|
Our study found that there was no significant difference between an investigator-initiated and an industry-sponsored trial in terms of the quality and completeness of data documentation (Tables 1-4). All areas of data documentation and archival were found to be quite comparable in the two trials. This shows that in trials in institutes where a high volume of studies is conducted, data documentation and archival can be done with similar rigour to that in industrysponsored studies.
The informed consent form was one area in which a difference was observed. As can be seen in Table 4, the sponsored study fared better (78.3% for the investigator-initiated study versus 84.8% for the sponsored study). The informed consent process is the cornerstone of sound ethical and scientific research and has been the subject of numerous studies, both in developed and developing countries. The signature of the investigator, along with the date of the consent process, ensures the adequacy and completeness of the informed consent process. A previous study has shown that issues related to the informed consent process are the third most common reason for the United States Food and Drugs Administration (US FDA) issuing warning letters to investigators (9). In both studies, there was no documentation of the photocopies of the informed consent form having been given to the participants (Tables 3 and 4). While this may simply have been a lapse in documentation, it still indicates that the informed consent process is an area that needs to be addressed and strengthened by all the stakeholders.
Our study had certain limitations. A comparison was made of only one study from each category, the investigatorinitiated and sponsored type. This was because in the case of industry-funded studies, the sponsor’s approval was required for analysing the data and only one sponsor from the several that we contacted permitted analysis of his study. This limited the generalisability of our results. Further, the investigators in the two studies were different. A more accurate interpretation would have been possible if trials conducted by the same investigator had been analysed. Our study focused only on the quality and completion of documentation in the two studies; it did not assess the actual conduct of the trial. We compared a surgical trial (investigator-initiated) with a medical one (sponsored study). The weight of responsibility on the investigator handling the surgical trial may have been relatively greater due to increased concern about the patients’ safety, resulting in the use of higher standards in the conduct of the study. There is also a possibility of the introduction of a bias due to the fact that one of the authors was also the principal investigator of the investigator-initiated study. Finally, the results could have been influenced by the extent of training received by the principal investigator in clinical research and his experience in conducting clinical trials. Notwithstanding these limitations, we do feel that the results of our study are important as very few data exist on the subject.
This study has shown that researchers in academic settings adhere to the high standards of the GCP, since hardly any differences were observed between the IIT and the trial sponsored by a pharmaceutical industry. This is important, considering that investigator-initiated studies are associated with lower running costs than sponsored studies (10). We, however, cannot deny the importance of sponsored studies as investigators who have already conducted one are experienced and hence, are more likely to adhere to GCP guidelines while conducting their own trial. Be that as it may, investigators could improve the quality and documentation of their research by employing internal monitors and applying for additional funding for their studies, either within their own institutes or to governmental agencies. In this way, external, independent monitors could be appointed and data checked periodically. Institutional review boards, too, should regularly monitor investigator-initiated research as a part of their activities. This would help ensure that issues related to the consent process and the general conduct of the trial are picked up early on in the course of the study, and training and retraining of the staff can be carried out. Finally, researchers should realise that the GCP are essentially and will remain an attitude and approach towards the conduct of research, and must be the same regardless of whether the research is sponsored (and, therefore, heavily monitored) or investigatorinitiated.
Conflict of interest. One of the authors (CS Pramesh) was the principal investigator for the investigator-initiated study and is from the same hospital where the study was conducted.
Patient-related documents: global scores (calculated for each patient, out of a maximum score of 32)
|Investigator-initiated study||Sponsored study|
|Patient No.||Score (maximum = 32)||Patient No.||Score (maximum = 32)|
|Mean summary global score with 95% CI||27.1 [26.4, 27.8]||27.9 [26.7, 29.1]|
Analysis of individual criteria giving sub-scores within the patient-related documents
|S. no.||Parameter||Investigator-initiated study||Pharmaceutical industry-sponsored study|
|1c||Correlation with source documents||24.5||28||87.5||10||14||71.4|
|2||Informed consent form|
|2b||Approved version used||27||28||96.4||14||14||100|
|2c||Source documentation of informed consent Process||28||28||100||11||14||78.5|
|2d||Documentation of photocopy being given||0||28||0||0||14||0|
|2f||Participant – signature||13.5||14||96.4||7||7||100|
|2g||Participant – date||11.5||14||82.1||7||7||100|
|2h||Doctor – signature||8||14||57.1||7||7||100|
|2i||Doctor – date||6.5||14||46.4||7||7||100|
|2j||Witness – signature||14||14||100||5.5||7||78.5|
|2k||Witness – date||13||14||92.9||5.5||7||78.6|
|3||Case report form|
|5||Protocol and safety reporting|
|Grand total||760||896||760/896 (84.8%)||390||448||390/448
We thank Dr Kumar Prabhash for providing valuable logistic assistance in the conduct of the study and the Directors of the Tata Memorial Centre and King Edward Memorial Hospital for permission to carry out the study. We also thank Dr S Kannan from the Department of Clinical Pharmacology for assisting us with the statistical analysis and the editing of the manuscript.
- ICH Expert Working Group. ICH Harmonized Tripartite Guideline: Guideline for Good Clinical Practice E6 (R1) [Internet]. International Conference on Harmonisation (ICH); 1996 Jun 10 [cited 2013 Nov 2]. Available from: http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6_R1/Step4/E6_R1__Guideline.pdf
- Bargaje C. Good documentation practice in clinical research. Perspect Clin Res. 2011;2(2):59-63.
- Vijayananthan A, Nawawi O. The importance of Good Clinical Practice guidelines and its role in clinical trials. Biomed Imaging Interv J. 2008 Jan-Mar; 4(1):e5.
- Bortolussi R, Nicholson D. Auditing of clinical research ethics in a children’s and women’s academic hospital. Clin Invest Med. 2002;25:83-8.
- Taghinia AH, Liao EC, May JW Jr. Randomized controlled trials in plastic surgery: a 20-year review of reporting standards, methodologic quality, and impact. Plast Reconstr Surg. 2008 Oct;122(4):1253-63.
- Siegfried N, Clarke M, Volmink J, Van der Merve L. African HIV/AIDS trials are more likely to report adequate allocation concealment and random generation than North American trials. PLoS ONE. 2008;3(10):e3491. doi:10.1371/journal.pone.0003491
- Sweatman J. Good clinical practice: a nuisance, a help or a necessity for clinical pharmacology? Br J Clin Pharmacol. 2003;55:1-5.
- Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005 Apr 2;330(7494): 753. Epub 2005 Jan 28.
- Gogtay NJ, Doshi BM, Kannan S, Thatte U. A study of warning letters issued to clinical investigators and institutional review boards by the United States Food and Drug Administration. Indian J Med Ethics. 2011 Oct-Dec;8(4):211-14.
- Bergmann L, Berns B, Dalgleish AG, von Euler M, Hecht TT, Lappin GL, Reed N, Palmeri S, Smyth J, Embacher-Aichorn S, Zwierzina H, Biotherapy Development Association. Investigator-initiated trials of targeted oncology agents: why independent research is at risk? Ann Oncol. 2010 Aug;21(8):1573-8. doi: 10.1093/annonc/mdq018. Epub 2010 Feb 4.