ARTICLES

Quality and completeness of data documentation in an investigator-initiated trial versus an industry-sponsored trial

Soumil Patwardhan , Nithya Gogtay, Urmila Thatte, C S Pramesh

DOI: https://doi.org/10.20529/IJME.2014.006


Abstract

Literature on the quality and completeness of data and documentation in investigator-initiated research studies is scarce. We carried out a study to compare the quality of data and documentation in an investigator-initiated trial (IIT) with those in an industry-sponsored study. We retrospectively studied the archived data pertaining to 42 patients enrolled in two trials, 14 patients in an industry-sponsored study and 28 randomly selected patients from an IIT. Trial-related documents were examined and scored for the completeness of the acquisition of data and for storage as per a pre-formulated checklist. Weighted scores were given for each point on the checklist proportional to its relative importance in the data documentation process. A global score and sub-scores for specific modules were given for each subject. The scores in the two studies were compared using the Mann Whitney U test. The total score for general documents was similar in the IIT (14/14, 100%) and the sponsored study (24/25, 96%). The mean summary global score obtained for study-specific documents (maximum possible score, 32) in the IIT (27.1; 95% CI 26.4-27.8) was also not significantly different from that in the sponsored study (27.9; 95% CI 26.7-29.1; p=0.1291). Thus, investigatorinitiated studies carried out by independent researchers in highvolume academic centres, even without active data monitoring and formal audits, appear to adhere to the high standards laid out in the International Conference on Harmonisation-Good Clinical Practices guidelines, ensuring accuracy and completeness in data documentation and archival.

Introduction

The basic tenets of the International Conference on Harmonisation-Good Clinical Practices (ICH-GCP) guidelines include the accurate capture, storage and reporting of data (1). The importance of these guidelines in the context of providing vital data from clinical trials for statistical analysis has been stressed by other studies as well (2, 3). However, despite these guidelines, research has shown that intensive data monitoring and adherence to data documentation are not yet a part of the conduct of clinical trials (4). The documentation of data and accuracy of reporting are subject to regular monitoring, audits and inspections in an industry-funded study. Investigatorinitiated research, on the other hand, is not subject to such rigorous monitoring and may never undergo a formal audit or an inspection. Several publications have highlighted the importance of, and raised concerns over, adherence to methodological quality and the quality of reporting of research, both in industry-funded and investigator-initiated trials (5, 6).

Despite an extensive search of the literature, we were unable to find any study which compared the quality of documentation in investigator-initiated research with that in industry-funded research. Hence, we conducted a study to compare the quality and completeness of documentation in an investigatorinitiated randomised trial of two types of lymph node dissections for oesophageal cancer (with one of the authors, C S Pramesh, as the principal investigator) with an open-label industry-funded study of the use of granulocyte colonystimulating factor in the treatment of febrile neutropenia induced by cancer chemotherapy.

Methods

  1. Ethics
  2. We initiated the study after obtaining the approval of the institutional ethics committees of the institutions concerned. Both the sponsored and investigator-initiated studies were conducted at the Tata Memorial Hospital, which is a highvolume tertiary referral centre for the treatment of cancer in India. The principal investigators in both studies were experienced in clinical trials and had already participated in more than 10 research studies (both IITs and industrysponsored trials). A request was made for the waiver of consent and this was granted on the grounds that the study was retrospective and no confidential information about the patients would be made public. Unique identifiers, which were pre-coded, were used for all patients. Prior to the initiation of the study, permission was taken from the principal investigators of both studies, as well as the sponsor of the funded study.

  3. Methodology
  4. In the industry-funded study, a total of 14 patients were recruited from the Tata Memorial Hospital. In the investigatorinitiated study, 28 patients were randomly selected from a total of 200 patients with the help of a random number table. The ratio for comparison between the two studies was thus 1:2. We drew up a checklist consisting of various criteria that are important in trial documentation. We classified these as general documents (Table 1) and patient-related documents (Tables 2-4). All were examined and scored for completeness of data acquisition and archiving. Each point on the checklist was given a weighted score, which was summated into a global score for each patient. We also calculated sub-scores for specific modules in the documentation (Table 4). As the investigator-initiated study was a surgical trial, some parameters were not applicable and, therefore, were not considered for scoring. The data were checked for normality using the Kolmogorov Smirnov test and analysed using the Mann Whitney U test. Graph Pad InStat, Version 3.10 was used for all analyses which were done at 5% significance.

Results

The scores for general documents are listed in Table 1. The maximum score possible for the investigator-initiated study was 14 and that for the sponsored study, 25. The IIT scored 14/14 (100%). and the sponsored study, 24/25 (96%) Thus, there was barely any difference with respect to the general documents. Table 2 provides the details of the patientrelated documents and is subdivided into (i) admission criteria, (ii) informed consent form, (iii) case record form, (iv) source documents, and (v) protocol and safety reporting. The maximum score was 28 for the IIT and 14 for the sponsored study for all the subdivisions, except the subdivision of the reporting of serious adverse events (SAEs) to the ethics committee, sponsor, regulator and follow-up reporting, for which the maximum score was 56 in the case of the IIT and 28 in the case of the sponsored study. Thus the maximum possible score for the investigator-initiated study was 896 and that for the sponsored study was 448. The scores obtained were 760/896 (84.8%) and 390/448 (87%), respectively. Table 3 gives the global score for patient-related documents. It may be seen that the score for the investigator-initiated study ranges from 23.5 to 29.5 and that for the sponsored study from 23.5 to 30. The mean summary global score for patient-related documents (Table 3) obtained in the investigator-initiated study (27.1; 95% CI 26.4-27.8) was also not significantly different from that in the sponsored study (27.9; 95% CI 26.7-29.1); (p=0.1291, Mann Whitney U test). The points assigned for individual parameters, the sub-scores for specific modules and the global score obtained for each trial are shown in Table 4. While the global scores were not significantly different, a difference was seen in the sub-score of informed consent, with the investigator-initiated study scoring 78.3% and the sponsored study scoring 84.8%.

Discussion

The ICH-GCP has provided guidelines for data documentation, archival and reporting (1); however, adherence to these is not as frequently emphasised as adherence to guidelines relating to ethics and the actual conduct of the study. This might be a bigger problem in investigator-initiated research, as monitoring is not as strict as in sponsored studies. A large investment, in terms of finances and resources, has to be made by investigators in order for trials to strictly comply with the GCP guidelines (7). Sponsored studies employ qualified and trained staff, as well as monitors, to carry out intensive reviews and arduous monitoring of the proceedings of a trial. In investigator-initiated and non-commercial studies, the existing staff members are usually responsible for monitoring and review, which they carry out together with their regular duties, as cost constraints do not permit the use of additional manpower. Despite the hurdles that the investigator has to face, there is no alternative to accurate and diligent data collection, storage and reporting, the absence of which can significantly alter statistical analysis and interpretation, thus affecting the results of the trial (8).

Table 1

General trial-related documents: checklist, points for sub-scores and scores of both studies

S. no. Criterion Investigatorinitiated research Pharmaceuticalsponsored research
1 Protocol parameters
1a. Protocol 1 1
1b. Final approved version of protocol 1 1
1c. IEC approval of above 1 1
1d. Protocol signed by PI 1 1
1e. Protocol amendments, if any 1 1
1f. IEC approval of protocol amendments 1 1
Total score for protocol parameters = 6 6/6 6/6
2 Final version of case report form 1 1
3 Final approved version of informed consent form 1 1
4 Final approved versions of Marathi and Hindi translations of the informed consent form 1 1
5 Investigator’s brochure – presence of approved versions NA 1
6 IEC approval of investigator’s brochure NA 1
7 Clinical trial agreement – signed and dated NA 1
8 Investigator’s undertaking – signed and dated NA 1
9 Insurance NA 1
10 Indemnity NA 1
11 Progress reports 1 1
12 Work distribution log 1 0
13 Address details of laboratory NA 1
14 Laboratory – normal values NA 1
15 Documentation of laboratory procedure NA 1
16 Approved subject diary NA 1
17 Translation of subject diary in Hindi and Marathi NA 1
18 Other relevant documents 1 1
19 CVs of study personnel 1 1
20 Investigational product – information accountability 1 1
Total 14 24
(Maximum score, 14) (Maximum score, 25)
Percentage 100 96

Table 2

Patient-related documents: checklist and scores for individual items

S. no. Parameter Score (for each patient) Maximum possible score Calculated score
Investigator-initiated trial (with 28 patients) Sponsored trial (with 14 patients) Investigatorinitiated trial Sponsored trial
1 Admission Criteria
1a 1 28 14 28 14
1b 1 28 14 27.5 14
1c 1 28 14 24.5 10
2 Informed consent form
2a 1 28 14 27 14
2b 1 28 14 27 14
2c 1 28 14 28 11
2d 1 28 14 0 0
2e 1 28 14 26 14
2f 0..5 14 7 13.5 7
2g 0.5 14 7 11.5 7
2h 0.5 14 7 8 7
2i 0.5 14 7 6.5 7
2j 0.5 14 7 14 5.5
2k 0.5 14 7 13 5.5
3 Case report form
3a 1 28 14 28 14
3b 1 28 14 0 0
3c 1 28 14 26.5 14
4 Source document/s
4a 1 28 14 28 14
4b 9 252 126 208 110.5
4c 2 56 28 53.5 28
5 Protocol and safety
5a Documentation of deviations/ violations 1 28 14 28 14
5b EC reporting of deviations/ violations 1 28 14 28 14
5c AEs documented in source notes and CRFs 1 28 14 26.5 13
5d SAEs documented in source notes and CRFs 1 28 14 27 13
5e SAE reporting to EC, sponsor, regulator and follow-up reporting& 2 56 28 52 25.5
Total 32 896 448 760/896
(84.8%)
390/448 (87%)
Mann-Whitney U test, p*>0.05

Our study found that there was no significant difference between an investigator-initiated and an industry-sponsored trial in terms of the quality and completeness of data documentation (Tables 1-4). All areas of data documentation and archival were found to be quite comparable in the two trials. This shows that in trials in institutes where a high volume of studies is conducted, data documentation and archival can be done with similar rigour to that in industrysponsored studies.

The informed consent form was one area in which a difference was observed. As can be seen in Table 4, the sponsored study fared better (78.3% for the investigator-initiated study versus 84.8% for the sponsored study). The informed consent process is the cornerstone of sound ethical and scientific research and has been the subject of numerous studies, both in developed and developing countries. The signature of the investigator, along with the date of the consent process, ensures the adequacy and completeness of the informed consent process. A previous study has shown that issues related to the informed consent process are the third most common reason for the United States Food and Drugs Administration (US FDA) issuing warning letters to investigators (9). In both studies, there was no documentation of the photocopies of the informed consent form having been given to the participants (Tables 3 and 4). While this may simply have been a lapse in documentation, it still indicates that the informed consent process is an area that needs to be addressed and strengthened by all the stakeholders.

Our study had certain limitations. A comparison was made of only one study from each category, the investigatorinitiated and sponsored type. This was because in the case of industry-funded studies, the sponsor’s approval was required for analysing the data and only one sponsor from the several that we contacted permitted analysis of his study. This limited the generalisability of our results. Further, the investigators in the two studies were different. A more accurate interpretation would have been possible if trials conducted by the same investigator had been analysed. Our study focused only on the quality and completion of documentation in the two studies; it did not assess the actual conduct of the trial. We compared a surgical trial (investigator-initiated) with a medical one (sponsored study). The weight of responsibility on the investigator handling the surgical trial may have been relatively greater due to increased concern about the patients’ safety, resulting in the use of higher standards in the conduct of the study. There is also a possibility of the introduction of a bias due to the fact that one of the authors was also the principal investigator of the investigator-initiated study. Finally, the results could have been influenced by the extent of training received by the principal investigator in clinical research and his experience in conducting clinical trials. Notwithstanding these limitations, we do feel that the results of our study are important as very few data exist on the subject.

This study has shown that researchers in academic settings adhere to the high standards of the GCP, since hardly any differences were observed between the IIT and the trial sponsored by a pharmaceutical industry. This is important, considering that investigator-initiated studies are associated with lower running costs than sponsored studies (10). We, however, cannot deny the importance of sponsored studies as investigators who have already conducted one are experienced and hence, are more likely to adhere to GCP guidelines while conducting their own trial. Be that as it may, investigators could improve the quality and documentation of their research by employing internal monitors and applying for additional funding for their studies, either within their own institutes or to governmental agencies. In this way, external, independent monitors could be appointed and data checked periodically. Institutional review boards, too, should regularly monitor investigator-initiated research as a part of their activities. This would help ensure that issues related to the consent process and the general conduct of the trial are picked up early on in the course of the study, and training and retraining of the staff can be carried out. Finally, researchers should realise that the GCP are essentially and will remain an attitude and approach towards the conduct of research, and must be the same regardless of whether the research is sponsored (and, therefore, heavily monitored) or investigatorinitiated.

Conflict of interest. One of the authors (CS Pramesh) was the principal investigator for the investigator-initiated study and is from the same hospital where the study was conducted.

Table 3

Patient-related documents: global scores (calculated for each patient, out of a maximum score of 32)

Investigator-initiated study Sponsored study
Patient No. Score (maximum = 32) Patient No. Score (maximum = 32)
1 23.5 1 29
2 28.5 2 29
3 26 3 23.5
4 26.5 4 27.5
5 27 5 29.5
6 27 6 28
7 27.5 7 30
8 29.5 8 23.5
9 29 9 29.5
10 27.5 10 29
11 28 11 29
12 29.5 12 26.5
13 28.5 13 28
14 28 14 28
15 28.5
16 27
17 25.5
18 29.5
19 28
20 28
21 28
22 27.5
23 24.5
24 23.5
25 24
26 24.5
27 29
28 26.5
Range 23.5-29.5 23.5-30
Mean summary global score with 95% CI 27.1 [26.4, 27.8] 27.9 [26.7, 29.1]
*=0.1291, Mann-Whitney U test
Table 4

Analysis of individual criteria giving sub-scores within the patient-related documents

S. no. Parameter Investigator-initiated study Pharmaceutical industry-sponsored study
Score Maximum Percentage Score Maximum Percentage
1 Admission criteria
1a Present 28 28 100 14 14 100
1b Completeness 27.5 28 98.2 14 14 100
1c Correlation with source documents 24.5 28 87.5 10 14 71.4
Total 80 84 95.2 38 42 90.5
2 Informed consent form
2a Present 27 28 96.4 14 14 100
2b Approved version used 27 28 96.4 14 14 100
2c Source documentation of informed consent Process 28 28 100 11 14 78.5
2d Documentation of photocopy being given 0 28 0 0 14 0
2e Language match 26 28 92.8 14 14 100
2f Participant – signature 13.5 14 96.4 7 7 100
2g Participant – date 11.5 14 82.1 7 7 100
2h Doctor – signature 8 14 57.1 7 7 100
2i Doctor – date 6.5 14 46.4 7 7 100
2j Witness – signature 14 14 100 5.5 7 78.5
2k Witness – date 13 14 92.9 5.5 7 78.6
Total 174.5 224 78.3 92 112 84.8
3 Case report form
3a 28 28 100 14 14 100
3b 0 28 0 0 14 0
3c 26.5 28 94.6 14 14 100
Total 54.5 84 64.9 28 42 66.7
4 Source document/s
4a 28 28 100 14 14 100
4b 208 252 82.5 110.5 126 87.6
4c 53.5 56 95.5 28 28 100
Total 289.5 336 86.2 152.5 168 90.8
5 Protocol and safety reporting
5a 28 28 100 14 14 100
5b 28 28 100 14 14 100
5c 26.5 28 94.6 13 14 92.8
5d 27 28 96.4 13 14 92.8
5e 52 56 92.8 25.5 28 91.0
Total 161.5 168 96.1 79.5 84 94.6
Grand total 760 896 760/896 (84.8%) 390 448 390/448
(87.1%)

Acknowledgements

We thank Dr Kumar Prabhash for providing valuable logistic assistance in the conduct of the study and the Directors of the Tata Memorial Centre and King Edward Memorial Hospital for permission to carry out the study. We also thank Dr S Kannan from the Department of Clinical Pharmacology for assisting us with the statistical analysis and the editing of the manuscript.

References

  1. ICH Expert Working Group. ICH Harmonized Tripartite Guideline: Guideline for Good Clinical Practice E6 (R1) [Internet]. International Conference on Harmonisation (ICH); 1996 Jun 10 [cited 2013 Nov 2]. Available from: http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6_R1/Step4/E6_R1__Guideline.pdf
  2. Bargaje C. Good documentation practice in clinical research. Perspect Clin Res. 2011;2(2):59-63.
  3. Vijayananthan A, Nawawi O. The importance of Good Clinical Practice guidelines and its role in clinical trials. Biomed Imaging Interv J. 2008 Jan-Mar; 4(1):e5.
  4. Bortolussi R, Nicholson D. Auditing of clinical research ethics in a children’s and women’s academic hospital. Clin Invest Med. 2002;25:83-8.
  5. Taghinia AH, Liao EC, May JW Jr. Randomized controlled trials in plastic surgery: a 20-year review of reporting standards, methodologic quality, and impact. Plast Reconstr Surg. 2008 Oct;122(4):1253-63.
  6. Siegfried N, Clarke M, Volmink J, Van der Merve L. African HIV/AIDS trials are more likely to report adequate allocation concealment and random generation than North American trials. PLoS ONE. 2008;3(10):e3491. doi:10.1371/journal.pone.0003491
  7. Sweatman J. Good clinical practice: a nuisance, a help or a necessity for clinical pharmacology? Br J Clin Pharmacol. 2003;55:1-5.
  8. Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005 Apr 2;330(7494): 753. Epub 2005 Jan 28.
  9. Gogtay NJ, Doshi BM, Kannan S, Thatte U. A study of warning letters issued to clinical investigators and institutional review boards by the United States Food and Drug Administration. Indian J Med Ethics. 2011 Oct-Dec;8(4):211-14.
  10. Bergmann L, Berns B, Dalgleish AG, von Euler M, Hecht TT, Lappin GL, Reed N, Palmeri S, Smyth J, Embacher-Aichorn S, Zwierzina H, Biotherapy Development Association. Investigator-initiated trials of targeted oncology agents: why independent research is at risk? Ann Oncol. 2010 Aug;21(8):1573-8. doi: 10.1093/annonc/mdq018. Epub 2010 Feb 4.
About the Authors

Soumil Patwardhan (cspramesh@gmail.com)

Department of Clinical Pharmacology

Seth GS Medical College and KEM Hospital, Parel, Mumbai 400 012

Nithya Gogtay (cspramesh@gmail.com)

Department of Clinical Pharmacology

Seth GS Medical College and KEM Hospital, Parel, Mumbai 400 012

Urmila Thatte (cspramesh@gmail.com)

Department of Clinical Pharmacology

Seth GS Medical College and KEM Hospital, Parel, Mumbai 400 012

C S Pramesh (cspramesh@gmail.com)

Department of Surgical Oncology

Tata Memorial Centre, Parel, Mumbai 400 012

Keywords

N/A

Refbacks

There are currently no refbacks.