DOI: https://doi.org/10.20529/IJME.2011.082
Background: Warning letters (WLs) issued by the US FDA (United States Food and Drug Administration) mention the nature of violations by clinical investigators and institutional review boards (IRBS) and can help as training tools.
Methods: WLs issued by the US FDA between January 2005 and December 2010 to clinical investigators and IRBs were reviewed for various violation themes.
Results: A total of 129 WLs were issued to investigators and 40 to IRBs. Among the WLs issued to investigators, 67 (51.95%) were issued for drug-related research and 62 (48.06%) were for device-related research. For investigators, deviation from the investigational plan was the most common violation (81% ) followed by failure to maintain accurate and adequate case histories (58.1% ) and then informed consent issues (48.06%). Among WLs issued to IRBs, failure to have and follow standard operating procedures (SOPs) was seen in 93.8% followed by issues pertaining to membership (59.4%). When compared to a similar study published in 2004, for clinical investigators, no improvement was seen with respect to deviation from the investigational plan and study supervision. However, a significant improvement was seen in reporting of adverse events to IRBs, and some improvement was seen in the area of informed consent. For IRBs, no improvement was seen in most areas which included maintaining and following SOPs, membership, quorum requirements, misuse of expedited review and informed consent.
Conclusion: WLs serve as indicators of an active regulatory agency which should translate into greater safety for participants in clinical trials. For developing countries with weak regulatory systems, these can serve as useful learning tools to help improve systems and put in patient safeguards.
The United States Food and Drug Administration (US FDA) oversees clinical research studies and ensures study integrity as well as safety and welfare of human research subjects (1). The FDA expects clinical investigators and Institutional Review Boards (IRBs) to follow standards of good clinical practice. The FDA has also laid down standards for IRBs in the Code of Federal Regulations (2, 4).
Through their bioresearch monitoring programme, the US FDA conducts site visits of clinical investigators, sponsors, monitors, contract research organisations, IRBs, nonclinical (animal) laboratories, and bioequivalence analytical laboratories (5). The findings of site visits are noted in the form FDA 483, “Notice of inspectional observations” (3) which is submitted by the field inspector to his/her superiors (6). The Clinical Investigator Inspection List deficiency codes (7) are made by the US FDA which assigns a code for each deficiency found during inspection of a clinical investigator, such as case closed with Memo To File, No Action Indicated, Voluntary Action Indicated and Official Action Indicated. In the last case, i.e., when issues are found serious enough to jeopardise safety of study participants, a Warning Letter (WL) is issued. The US FDA is the only regulatory authority that has imposed serious sanctions to date by issuing a ‘blacklist’, which lists all investigators who have been found to be non-compliant and have been barred from clinical research for FDA submissions (8). Against this backdrop, a study by Bramstedt (1, 2) of WLs issued to clinical investigators from 2004 to 2005 and to IRBs from 1997 to 2004 showed several areas of violation. The authors stated that these WLs acted as useful instructional tools for clinical investigators and IRBs. We conducted the present study to assess the nature of WLs issued by the US FDA to clinical investigators and IRBs from January 2005 to December 2010, and to see if there was any change from the earlier findings reported by Bramstedt (1, 2).
The study protocol was submitted to the IRB and was deemed to be exempt from review. The online FDA Warning Letter Index (9) lists WLs by year. The years January 2005-Dec 2010 were looked at and WLs issued to clinical investigators and IRBs (sub-heading) were identified via a manual screening process.
The WLs to clinical investigators were evaluated for violations of seven themes: (i) deviation from investigational plan; (ii) maintaining adequate and accurate case records; (iii) informed consent; (iv) regulatory non-compliance; (v) violations related to the investigational product (drug or device); (vi) personal supervision of study conduct, and (vii) adverse event and IRB reporting. Similarly, the WLs to IRBs were evaluated for the violations in the following seven themes: (i) failure to follow SOPs and maintain documentation; (ii) informed consent issues; (iii) inappropriate membership, quorum issues, want of a lay person in meetings, misuse of expedited review; (iv) failure to follow regulatory requirements; (v) inadequacy or lack of systems for monitoring; (vi) failure to address conflict of interest; and (vii) failure to protect vulnerable participants and address risk minimisation.
The responses from clinical investigators and IRBs to these WLs were also reviewed. A comparison of current findings with those of Bramstedt et al was done for categorical data at 5% significance using Minitab, version 16.
A total of 129 WLs were issued to clinical investigators. Of these, 67 (51.95%) were issued for drug-related research and 62 (48.06%) were for device-related research. Of the 67 drug- related WLs, information on specialties was available for 39 WLs. These were oncology [9], psychiatry [8], pulmonary [9], endocrine [4], antimicrobials [4], gastroenterology [2], cardiology [1], ophthalmology [1] and dentistry. [1]. For device-related research, information on specialities was available for 29 WLs. These were orthopaedics [10], surgery [15], radiology [3], and ear, nose & throat [1].
The most common violation among WLs was deviation from the investigational plan (104/129; 80.6%). Of these 57/104 (54.8%) were issued for drug-related research. This included permitting use of prohibited study medications, enrolment of participants before the screening results became available, or enrolling subjects who didn’t fit inclusion criteria. Failure to maintain accurate and adequate case histories and/or inability to retain records for inspection or inability to produce records for inspection was the next most common violation (75/129; 58.1%). There were 62 (48.06%) WLs issued for informed consent related issues. These included failure to obtain consent before screening, backdating by the clinical investigator, and using the consent form of a different study. Other areas of violation included regulatory non-compliance (50/129; 38.8%), failure to maintain records of the investigational product (38/129; 29.5%), failure to protect subject safety and/or report adverse events (AEs) to the IRBs (30/129; 23.3%) and failure to personally supervise the study (27/129; 20.9%). Table 1 lists all violations by theme for clinical investigators. All WLs documented violations in at least 2 out of the 7 themes.
A total of 40 WLs were issued to IRBs. Of these, 25 were issued to hospital/medical centre IRBs, 9 to university IRBs and 6 to private IRBs. Nine WLs (22.5%) were issued for drug- related research and 23 (57.5%) for device-related research. For 8, it was not mentioned whether the WL was for drug- or device-related research.
The most common regulatory violation reported in the WLs to IRBs was the failure to have and follow standard operating procedures (SOPs), followed by failure to document IRB activities, seen in 38/40 (95%) WLs. Inappropriate membership, quorum issues, want of a lay person in meetings, and/or misuse of expedited review were seen in 26/40 (65%) WLs. Table 2 lists the number and themes of violations by IRBs.
Response letters from only 7 investigators and 1 IRB were identified. All investigators accepted the findings and assured the FDA that they would look into why the problems had arisen and that necessary corrective action would be taken. The IRB, however, asked the FDA to withdraw its WL, which they felt lacked factual foundation. The FDA in turn replied that it would look into the matter.
Clinical investigators: No significant difference was seen with respect to deviation from the investigational plan and supervision of study conduct while significant improvements were seen in the areas of study reporting AEs to IRBs and informed consent. (Table 1)
IRBs: No significant difference was found between the two studies in most areas. These included maintaining and following SOPs, membership, quorum requirements, misuse of expedited review and informed consent (Table 2).
In this study, 169 WLs issued by the US FDA over a 5-year period (2005-10) were studied. Clinical investigators were issued 129 and IRBs 40 letters. For the former, the most common reason for issue of a WL was deviation from the investigational plan while for the latter it was failure to have and/or adhere to SOPs. The rate of issue of WLs (except for a couple of areas) in our study was found to be similar to two studies by Bramstedt (1, 2). With respect to clinical investigators, the proportion of WLs was similar with respect to drug- and device-related research (52% and 48%). However, for IRBs, a much larger proportion of WLs were for device-related research (72%) indicating that this is an area that needs to be strengthened for IRBs.
Deviation from the investigational plan accounted for 81% of WLs to investigators and this had largely remained unchanged over the years. This was followed by inability of investigators to maintain accurate and adequate case histories and retaining records. The third most common finding was issues related to informed consent. Both of these can lead to study participants being exposed to unnecessary risks. Principal investigators often delegate responsibility of the study conduct and maintaining case records to coordinators and may not provide sufficient oversight. The informed consent process forms the backbone of clinical research and enrolling patients prior to signing the informed consent form or using consent forms of another study or a wrong version of the consent form violates the basic tenets of autonomy and non-maleficence. Training and periodic re-training of the study team in the protocol and related procedures and having SOPs for various study-related processes can help minimise this problem. One other way of addressing this is to have oversight through a dedicated on site quality assurance manager. Sponsors and investigators should work together to determine the best approach to quality assurance in a given study which would in turn depend upon the site’s experience in clinical research, the trial complexity and risks that patients are exposed to (10).
Our study showed that, 7 years after a similar study was published, most areas with respect to IRBs did not show any improvement. This indicates that the IRBs failed in their primary responsibility of protecting the rights, safety and well being of participants in research. A recent systematic review of empirical evidence from 43 published studies on IRBs in the United States showed both inefficiencies and inconsistencies in the evaluation process (11). Similar findings in Africa have been reported by Nyika et al (12). Capacity building of IRBs worldwide, as also research on how they actually accomplish their objectives, their quality of review and whether they are really effective at protecting the rights and safety of human participants, is also needed.
The present study is limited by its analysis of WLs from a solitary regulatory agency from a developed country. The analysis itself though could be useful for developing countries with weak regulatory systems where these can serve as useful learning tools. Lower operational costs, recent regulatory reforms and several logistic advantages make India today an attractive destination for conducting clinical trials (13, 14). However, it must be remembered that future clinical trials are likely to become more complex both in design and execution. Thus maintaining high ethical standards, continuous capacity building of both investigators and IRBs and stringent quality assurance will become exceedingly important to ensure that WLs are minimised and the rights, safety and well being of participants in research are protected.
| Violation themes | Current study(drug-related research) N = 67 (51.95%) | Current study(device-related research)N = 62 (48.06%) | Current study(drug + device) N = 129N (percentage) | Bramstedt study(drug- or device- related research) N = 36N (percentage) | P value |
| Deviation from investigational plan | 57(85) | 47(76) | 104 (80.6) | 32 (88.9) | Not significant |
| Maintaining adequate and accurate case records and histories + inability to retain records or produce records for inspection | 44 (66) | 31 (50) | 75(58.1) | not reported | — |
| Informed consent | 36 (54) | 26 (42) | 62(48) | 24 (66.7) | 0.05 |
| Regulatory non-compliance | 24 (36) | 26 (42) | 50(38.8) | not reported | — |
| Violations related to investigational product | 27 (40) | 11 (18) | 38(29.4) | not reported | — |
| Failure to personally supervise the study | 16 (24) | 11 (18) | 27(20.9) | 02(5.6) | 0.04 |
| Failure to protect subject safety, report AEs to IRBs | 20 (30) | 10 (16) | 30(23.2) | 17 (47.2) | 0.006 |
(Only n= 32 analysed as information for 8 WLs as to whether it was drug or device was not available).
| Violation themes | Current study(drug-related research) N = 9 (28%) | Current study(device-related research) N = 23 (72) | Current study(drug + device) N = 32N = 32 | Bramstedt study(drug- or device- related research) ( n = 52) N (percentage) | P value |
| Failure to follow SOPs and maintain documentation | 8 (89) | 22 (96) | 30 (93.8) | 50 (96.1) | Not significant |
| Inappropriate membership, quorum issues, misuse ofexpedited review, want ofa lay person in meetings | 3 (33) | 16 (70) | 19 (59.4) | 30 (58) | Not significant |
| Informed consent issues | 7 (78) | 8 (35) | 15 (46.9) | 19 (37) | Not significant |
| Failure to follow regulatory requirements | 1 (11) | 6 (26) | 7 (21.9) | not reported | — |
| Inadequate or lack of systems for monitoring | 1 (11) | 1 (4) | 2(6.7) | not reported | — |
| Failure to address conflict of interest | 1 (11) | 2 (8) | 3 (9.4) | not reported | — |
| Failure to address risk minimisation and protect vulnerable participants | 3 (33) | 1 (4) | 4 (12.5) | 11 (21.1) | Not significant |