Abstract

The present audit was carried out with the objective of evaluating warning letters (WLs) issued to trial sponsors, clinical investigators and institutional review boards (IRBs) by the United States Food and Drug Administration during a six-year period and compare it with two similar earlier audits. WLs were reviewed and classified as per stakeholders and further categorised as per predefined violation themes. The chi-square test was performed for trend analysis of WLs. A total of 62 WLs were issued to the three stakeholders. The maximum number of WLs were issued to the clinical investigators (36/62, 58.06%), followed by sponsors (19/62, 30.64%), and least to the IRBs (7/62, 11.29%). Among sponsors, lack of standard operating procedures for the monitoring, receipt, evaluation and reporting of post-marketing adverse drug events was the most common violation theme (8/19, 42.1%). Among clinical investigators, deviation from investigational plan was the most common violation theme (31/36, 86.11%.). For IRBs, inadequate documentation was the most common violation theme (6/7, 85.71%). We saw an overall reduction in the number of WLs issued to the stakeholders. Thus, we identified multiple areas on which each stakeholder should work for improvement.

Keywords: Investigational plan, informed consent, monitoring, documentation

Introduction

The United States Food and Drug Administration (US FDA or FDA) periodically conducts inspections to ascertain data integrity and participant safety and inspects alike the three key stakeholders — sponsors, institutional review boards (IRBs) and investigators. Subsequent to these inspections, if something is found objectionable, in the inspector’s opinion, Form 483 (also called Inspectional Observations) is issued. While it does not constitute a final determination by the FDA, the stakeholder is expected to respond in writing expeditiously, and also document corrective action. Lack of a response or an inadequate response from any stakeholder leads to the issuing of a Warning Letter (WL) which represents an escalation from the Form 483. These WLs are available under the United States Freedom of Information Act 1996 in the public domain (1).

Two previous audits (2, 3) evaluating WLs issued by the FDA have shown that among clinical investigators, deviation from the investigational plan was the most common violation; while among IRBs, failure to maintain adequate documentation and retain IRB records was the most common violation. Among sponsors, inadequate monitoring of the clinical investigations was the most common violation seen. The present study was envisaged as a follow up to the earlier two audits (2, 3) to assess whether the continued issuance of WLs has improved the functioning of the three stakeholders.

Methods

Ethics

The study was exempted from review by the Institutional Ethics Committee of the Seth GS Medical College and KEM Hospital, Mumbai, (EC/OA-27/2019) as it involved analysis of data available electronically in the public domain.

Study design, time frame, selection criteria and study sample

This audit was a retrospective analysis, which included all WLs issued by the FDA to the clinical investigators, sponsors and IRBs from January 2014 to December 2019 which formed the study sample. The WLs unrelated to clinical research, such as good manufacturing practices (GMP) deviations, adulterated animal food, labelling/false and misleading/new drug/misbranded, unapproved and misbranded new drugs, adulterated dietary supplement, family smoking prevention and tobacco control act/adulterated/misbranded, and illegal drug residue were excluded from the study.

Methodology

All WLs were hand searched and downloaded from the FDA database (1). These were reviewed and data was extracted by three authors (US, SS, and DB), independently. The data was further verified by the two senior authors (NG and UMT). We also collated the total number of FDA inspections for each year.

Classification of WLs and violation themes

Each WL was classified as per defined stakeholder. Various violation themes were predefined for individual stakeholders based on methodology from the two previous studies (2, 3). These included: a) Violation themes for clinical investigators: deviation from investigational plan, failure to maintain accurate, complete, and current records of each subject’s case history and exposure to the device, failure to personally conduct or supervise the clinical investigations, failure to obtain informed consent, violations related to investigational product, failure to comply with regulatory guidelines, failure to maintain adequate records of a drug and the disposition of the drug and failure to retain records; b) Violation themes for sponsors: inadequate monitoring, lack of standard operating procedures (SOPs) for the surveillance, receipt, evaluation, and reporting of post-marketing adverse drug experiences, failure to submit investigational new drug (IND) application, failure to maintain required records, non-adherence with FDA regulations, inadequate reporting of adverse events, failure to obtain IRB approval and failure to comply with IRB, and failure to include essential elements in the informed consent; c) Violation themes for IRBs: inadequate documentation, inadequate monitoring (overseeing studies conducted by the investigators) (4), non-declaration of conflict of interest (COI), failure to review proposed research at convened meetings, lack of SOPs, and failing to ensure that the essential elements were included in an informed consent document (ICD).

Outcome measures

Outcome measures include: a) The total number of WLs issued to sponsors, investigators, and IRBs; b) Nature (violation themes) of WLs issued to all stakeholders; c) Trend analysis of WLs with two previously conducted studies (2, 3); and d) Total number of FDA inspections over six years, and percentage of WLs issued to sponsors, investigators and IRBs as compared to FDA inspections.

Statistical analysis

Categorical variables like number of WLs issued to each of the stakeholders, and number of WLs issued under each of the violation themes were expressed as proportions. Chi-square test was used for trend analysis of WLs issued over a six-year period in comparison with two previous studies — Gogtay et al (2) and Shetty et al (3), respectively. Chi-square test was used for statistical significance for comparison of violations themes with two previous studies and post hoc analysis was done using the Bonferroni test. All analyses were performed at 5% significance level using Statistical Package for the Social Sciences (SPSS) version 24.

Results

Number of FDA inspections and characteristics of studies

The total number of FDA inspections for the years 2014, 2015, 2016, 2017, 2018, 2019 and 2020 were 4943, 4751, 4528, 5045, 4910, 4770 and 2778, respectively. A total of 62 WLs were issued to three stakeholders over the study period. The maximum WLs were issued to clinical investigators (36/62, 58.06%), followed by sponsors (19/62, 30.64%), while the smallest numbers of WLs were issued to IRBs (7/62, 11.29 %). Overall, there was a significant reduction in the issuance of WLs seen in this study relative to the two previous audits (p<0.001). This difference was significant between clinical investigators (129 in Gogtay et al, 20 in Shetty et al, and 36 in the current study) and sponsors (46 in Shetty et al, and 19 in the current study), but not with respect to IRBs (32 in Gogtay et al, 18 in Shetty et al, and 7 in the current study) [Table 1].

Analysis of WLs issued to individual stakeholders

Clinical Investigators

Of the 36 WLs issued to the clinical investigators, the common themes seen were: deviation from the investigational plan (31/36, 86.11%); followed by the failure to maintain adequate records of the drug and the disposition of the drug and failure to retain records (failure to maintain adequate records of the disposition of the drug, including dates, quantity, and use by participants) (11/36, 30.55%); failure to maintain accurate, complete, and current records of each subject’s case history and exposure to the device (10/36, 27.77%); failure to personally conduct or supervise the clinical investigations (7/36, 19.44%); and failure to obtain informed consent (4/36, 11.11%).

Sponsors

Of the 19 WLs issued to sponsors, the most common violation was: lack of SOPs for surveillance, receipt, evaluation and reporting of post-marketing adverse drug experiences (8/19, 42.1%); followed by inadequate monitoring (essentially, overseeing study) of the clinical investigations (6/19, 31.57%); failure to submit IND applications (5/19, 26.31%); inadequate reporting of adverse events (4/19, 21.05%); failure to maintain required records (4/19, 21.05%); non-adherence with FDA regulations (4/19, 21.05%); failure to obtain IRB approval [medical device studies (3/19, 15.78%)]; and failure to include essential elements in the informed consent document (1/19, 5.26%).

IRBs

Of the seven WLs issued to the IRBs, inadequate documentation of IRB meetings and discussions including voting and maintaining records for a defined time period (6/7; 85.71%) was the most common violation; followed by lack of SOPs (5/7, 71.42%); failure to review proposed research at convened meetings (4/7, 57.14%); inadequate monitoring (3/7, 42.85%), non-declaration of conflict of interest (1/7, 14.28%); and failing to ensure that the essential elements were included in the ICD required for adequate understanding by the participant and failing to complete ICD (1/7; 14.28%).

Details of WLs issued to all three stakeholders are depicted in Table 2.

Trend analysis

A comparison of WLs covered in the present study with WLs seen in our previous studies (2, 3) is presented in Table 2.

Clinical investigators

A significant reduction (p<0.05) was seen in the areas of record keeping, informed consent, investigational product (IP)-related violations and compliance with regulatory guidelines relative to the previous studies. On the other hand, areas such as deviation from investigational plan and supervision of the clinical investigations did not show any improvement (p>0.05).

Sponsors

There was a significant reduction in the area of inadequate monitoring of the clinical investigations indicating improved monitoring (p<0.05). Improvement was seen with regard to adherence to the FDA’s regulatory guidelines (p<0.05). Other domains such as failure to a) submit IND application, b) obtain IRB approval, c) maintain records, d) include essential elements in the informed consent, and e) report adverse events, did not show any improvement (p>0.05) over the previous studies.

IRBs

There was a significant difference (p<0.05) in inadequate documentation and records of projects submitted to IRBs for review, lack of SOPs and inadequate monitoring; while areas such as non-declaration of conflict of interest and deviation in informed consent document did not show any improvement (p>0.05).

Table 1: Total number of warning letters (WLs) issued to three stakeholders
Table 2: Violation themes among WLs issued to clinical investigators, IRBs and sponsors

Discussion

The present study found that 62 WLs were issued to three stakeholders (clinical investigators, sponsors, and IRBs) in clinical trials over a period of six-years, of which more than 58% were issued to clinical investigators. There was overall reduction in the number of WLs issued to all three stakeholders as compared to the previous two studies (2, 3). That difference was significant with regard to clinical investigators and sponsors but not with respect to IRBs.

Comparison of the findings from the current study with the two previous audits (2, 3) indicated that for investigators, there was a significant reduction in violations related to the informed consent process, errors in documentation of case history and non–adherence to regulatory guidelines (2, 3). This potentially could imply greater awareness and a significant spread of both the letter and spirit of good clinical practices (GCP) over the years. Arango et al did a literature survey of GCP training programmes over a ten-year period and found that many different training programmes addressing multiple stakeholders have been developed and disseminated (5). A questionnaire-based survey was done by Awatagiri et al, who concluded that GCP training programmes helped in improving the knowledge of investigators and research staff (6).

The lack of improvement in areas such as deviation from investigational plan, inadequate conduct and supervision of the clinical investigations is worrisome. A study by Romano et al overlaps with our study and has similar findings. Sixty WLs [2005-09 and 2010-14] from the same database issued after inspections of clinical investigation sites were studied. Deviation from the investigational plan turned out to be the most frequent violation theme. The reason this theme recurs and has not changed over time (7), is because the investigators who are more experienced could be shouldering a greater burden of studies leaving them with inadequate time for supervision. Additionally, pressure from the pharmaceutical industry to meet stringent timelines may lead investigators to take short cuts or buckle under pressure. We also observed that many clinical investigators or sites failed to maintain and retain adequate records of the trial drug and the disposition of the drug. This was the most common violation and difficulty seen during the audits of the sponsors as well as the regulatory authorities. The same reasons possibly apply to inadequate documentations, along with staff attrition and inadequate attention to training new recruits. In institutes like ours, almost all research staff and coordinators are contractual and not permanent employees so there could be inadequate training of the contractual staff that leads to inadequate documentation that affects the research activities. Conducting clinical research is a humungous task with myriad responsibilities resting with the principal investigator (8, 9). As a result of the increase in issuing of WLs, US FDA has issued a guidance document entitled “Guidance for Industry: Investigator Responsibilities – Protecting the Rights, Safety and Welfare of Study Subjects, October 2009” (10), to assist investigators and sponsors. The contents of this document will always remain relevant. Some aspects in this document include delegation of authority and the use of standard operating procedures, both of which can guide investigators on how to plan and conduct studies more effectively. Frequent internal reviews — online or offline, and early detection and addressing of errors will help address the deviations or violations related to conduct of clinical research. Also, good documentation practices by the investigators will ensure the credibility and validity of clinical research (11).

In comparison to earlier studies, among IRBs, there has been a significant improvement in adequate documentation, better monitoring of studies by the investigators, and adherence to the SOPs. This improvement could have been a result of the increase in GCP training programmes for IRBs (12). We observed that some WLs issued to IRBs were for the failure to review proposed research at convened meetings, which could be attributed to the fact that some IRBs that are more experienced — similar to experienced investigators — may be more burdened than others IRBs. However, there was no difference with regard to non-declaration of COI and deviation in informed consent documents. Appreciation and understanding of the importance of COI is a key aspect of IRB functioning (13) and needs attention. The Cancer Council New South Wales in Australia laid down guidelines in 2016 to address the COI among Ethics Committee members which states that all member must disclose their COI in writing when a potential or perception of conflict exists. The guidelines further state that the disclosure should be prompt and complete, and it is important that IRBs adhere to them (14). Accreditation by the national accreditation bodies is likely to play an important role in reducing violations and strengthening the functioning of IRBs. Specific to India, agencies such as the National Accreditation Board for Hospitals and Health Care (NABH) have begun accreditations of IRBs all over the country, and as on October 20, 2020, a total of 156 IRBs have been accredited by them (15, 16, 17). COI is an area that is being reviewed during the accreditation process to decrease the chances of violations.

Among sponsors, there was a significant improvement in the adequate monitoring of the clinical investigations. One of the reasons for this improvement could be the use of risk-based monitoring (RBM). RBM included various techniques and platforms to identify signals that indicated the potential violation related to conduct of the trial, safety and well-being of the research participants, and data integrity of the clinical research. This RBM has also been given importance in the recent “E6(R2) Good Clinical Practice: Integrated Addendum to ICH E6(R1) — Guidance for Industry, 2018” (4, 18). However, we did observe lack of SOPs for the surveillance, receipt, evaluation, and reporting of post-marketing adverse drug experiences. An overall approach to reduce the number of WLs is to use Quality-by-Design (QbD) for all stakeholders. The QbD approach improves the quality of clinical research by prospectively examining the research objectives and identifying the data, design, processes and strategy essential for meeting the research objective and eliminating nonessential or unnecessary activities (19). A public-private partnership co-funded by the US FDA and Duke University, “The Clinical Trials Transformation Initiative” (CTTI), held several workshops to encourage implementation of QbD in clinical research. The main objective of this initiative is to identify the challenges and lacunae in the field of clinical research and decrease the errors, thereby improving the quality and efficacy of clinical research (20).

The present study is limited by the fact that it included analysis of WLs from a solitary regulatory agency (though the audit may have been conducted in any country around the world) and individual therapeutic areas of the WLs like oncology were not analysed. Unlike the US FDA, many regulators do not place the findings of their inspections in the public domain, and hence, the extent of the problem or improvements (or lack thereof) over time in individual countries is not easy to define. The number and nature of trials registered with FDA would be variable and from different geographic locations. In addition, the two previous studies had evaluated different time period — five years and two years, respectively. Hence, the inferential statistics done by us must be viewed in perspective.

Conclusion

There has been an overall reduction in WLs issued by the US FDA to investigators, IRBs and sponsors during the study period. Key areas that need significant strengthening are: deviation from investigational plan and failure to supervise the clinical investigations for investigators; non-declaration of conflict of interest, and deviation in inclusion of essential element in informed consent for IRBs; and failure to submit IND application, obtain IRB approval, maintain records, include essential elements in informed consent and reporting of adverse events for sponsors.