FDA Drug-Review Deadlines and Safety Problems
NEJM March 27, 2008
Daniel Carpenter, Ph.D., Evan James Zucker, B.A., and Jerry Avorn, M.D.
From the Department of Government, Faculty of Arts and Sciences, Harvard University, Cambridge, MA (D.C.); and Harvard Medical School (E.J.Z., J.A.); and the Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital (J.A.) - both in Boston.
Background: The Prescription Drug User Fee Act (PDUFA) imposes deadlines for the completion of drug reviews by the Food and Drug Administration (FDA). Critics have suggested that these deadlines may result in rushed approvals and the emergence of unanticipated safety problems once a product is in clinical use.
Methods: We assessed the association between the PDUFA deadlines and the timing of FDA drug approval by constructing dynamic Cox proportional-hazards models of review times for all new molecular entities approved between 1950 and 2005. To determine whether the deadlines were associated with postmarketing safety problems, we focused on drugs submitted since January 1993, when the deadlines were first imposed. We used exact logistic regression to determine whether drugs approved immediately before the deadlines were associated with a higher rate of postmarketing safety problems (e.g., withdrawals and black-box warnings) than drugs approved at other times.
Initiation of the PDUFA requirements concentrated the number of approval decisions made in the weeks immediately preceding the deadlines. As compared with drugs approved at other times, drugs approved in the 2 months before their PDUFA deadlines were more likely to be withdrawn for safety reasons (odds ratio, 5.5; 95% confidence interval [CI], 1.3 to 27.8), more likely to carry a subsequent black-box warning (odds ratio, 4.4; 95% CI, 1.2 to 20.5), and more likely to have one or more dosage forms voluntarily discontinued by the manufacturer (odds ratio, 3.3; 95% CI, 1.5 to 7.5).
Conclusions: PDUFA deadlines have appreciably changed the approval decisions of the FDA. Once medications are in clinical use, the discovery of safety problems is more likely for drugs approved immediately before a deadline than for those approved at other times.
The Prescription Drug User Fee Act (PDUFA), originally enacted in 1992, transformed the organization of the Food and Drug Administration (FDA) in several important ways. It was initiated at a time when the FDA was perceived to lack the staff needed to review new drug applications rapidly. Working on the assumption that Congress was unlikely to appropriate additional funds to expand the FDA's staff, the federal government negotiated a plan with the pharmaceutical industry under which drug manufacturers would pay a user fee for each drug review to help cover the costs of the staff required to perform that work. The agreement required the FDA to make a decision on each application within a fixed period after submission.1,2 Deadlines differed for priority reviews and standard reviews.3,4,5,6
Critics have argued that the user-fee program makes the agency too dependent on the industry it regulates and has led the FDA to focus disproportionately on the needs of the manufacturers that now fund more than half of its drug-review budget and staff.7,8,9,10 Such criticisms have been heightened by the safety-based withdrawals of rofecoxib (Vioxx, Merck) and valdecoxib (Bextra, Pfizer),11,12 the delayed recognition of suicidality in children and young adults taking selective serotonin-reuptake inhibitors,11 and the addition of a black-box warning about congestive heart failure and controversy over the risk of myocardial infarction caused by rosiglitazone (Avandia, GlaxoSmithKline) 8 years after it was approved.13 A survey14 and anecdotal reports9 suggest that FDA scientists perceive the PDUFA as having reduced the agency's focus on drug risks, but there has been no rigorous analysis linking any feature of the user-fee legislation to subsequent safety problems. Some have argued that the PDUFA was successful in speeding the approval of new drugs and found that the overall rate of safety recalls did not increase after its implementation.6,15
Many other aspects of drug development and review have also changed since 1992. Review times have probably been accelerated by information technology, submission of higher-quality new drug applications, and patient advocacy. Evaluation of drug safety has also been affected by changes in postmarketing surveillance, knowledge acquired from dealing with previous drug problems, different therapeutic mechanisms, and perhaps a changing culture at the FDA. Therefore, any crude comparison of safety events before and after the PDUFA is not likely to produce valid or revealing answers about its effects. We conducted a quantitative analysis to determine whether a much more specific and plausible mechanism - the deadlines required by the PDUFA legislation - has changed the pattern of FDA decision making, and with what consequences.
After 1992, when the PDUFA was passed and deadlines for drug approvals were introduced, FDA drug approval decisions were concentrated in the 2 months just before the deadlines. For drugs approved since January 1993, we found that approvals made in the last 2 months before a deadline were more likely to be associated with subsequent safety problems than were approvals made at other times. This difference was seen when just-before-deadline approvals were compared with just-after-deadline approvals as well as with early approvals (those made 3 months or more before a deadline). As noted in two major reports of deficits in the FDA's capacity for postmarketing safety surveillance,29,30 ongoing assessments of drug risk are not conducted systematically. As a result, if a safety problem is not detected during the initial preapproval evaluation, it may not be fully defined or included as a prominent warning for years afterward, as occurred with the risks of potentially fatal drug-drug interactions with mibefradil (Posicor, Roche) and terfenadine (Seldane, Hoechst Marion Roussel), myocardial infarction or stroke with rofecoxib (Vioxx), hepatotoxicity with troglitazone (Rezulin, Warner-Lambert), and rhabdomyolysis with cerivastatin (Baycol, Bayer AG).
Our findings are not consistent with analyses that compared safety problems before and after enactment of the PDUFA and showed no increase in problems after the user-fee system was introduced.6,15 One potential explanation for the inconsistency is that our analyses explored the law's specific mechanisms instead of making blanket comparisons of all drugs approved before enactment and all those approved after enactment. Although our analysis corroborates other research suggesting that the PDUFA's acceleration of drug-review times was influenced by the incentives it provides for acceleration of drug review,2 a previous study has shown that the main factor accounting for accelerated approval times was the availability of more staff at the FDA; greater resources made it possible to hire more staff, and approval times were already falling rapidly in the years before the PDUFA.31 Approvals earlier in the review cycle were not inherently more likely to lead to postmarketing safety problems; it appears to be the deadline, not the speed of approval, that explains the difference in the risk of such problems.
This analysis has all the limitations of observational studies. In particular, approval times were not assigned randomly to drugs, and we cannot rule out the possibility that some unobserved factor was associated with both just-before-deadline reviews and postmarketing safety problems. We have, however, implemented extensive multivariate controls and performed numerous other checks on our analyses (see the Supplementary Appendix).
Finally, we have not conducted a cost-benefit analysis of the legislation, nor have we systematically measured whether the public health burden of subsequent safety problems outweighs the benefits of the user-fee act in accelerating drug review.
Taken together, these findings suggest potential adverse effects of the deadlines governing FDA drug review. There are many ways to accelerate regulatory processes, most notably by providing more staff, which can be accomplished as readily by appropriations as by user fees.31 Deadlines may offer a blunt tool with which to accelerate review. But in keeping with reports from FDA scientists,14,32 our findings suggest that deadlines may also cause drugs approved under these constraints to have a higher likelihood of unanticipated safety problems once they are in widespread use. A plausible hypothesis is that relying more on staffing and less on deadlines could result in the same degree of review efficiency without increasing the risk (and resulting greater cost) of unanticipated drug-safety problems.
Supported in part by grants from the Robert Wood Johnson Foundation (Investigator Award in Health Policy Research, to Dr. Carpenter), the National Science Foundation (SES-0076452), and Harvard University (the Center for American Political Studies and Institute for Quantitative Social Science).
No potential conflict of interest relevant to this article was reported.
Timing of Approvals
Figure 1 shows approvals by month for the first 24 months of the review cycle for new molecular entities submitted under the PDUFA and for those submitted during the 30 years before its enactment. For standard new molecular entities (Panel A) and priority entities (Panel D), there was no apparent spike in approval patterns in the pre-PDUFA data. Examining both standard and priority entities submitted from January 1993 to December 2004 with the use of dynamic Cox models, we found that approval was 3.4 times as likely in the 2 months before the user-fee deadline as it was at all other times of the review cycle (95% confidence interval [CI], 2.2 to 5.2; P<0.001); we also found that approvals were 2.7 times as likely in the 2 months before the deadline as in the 2 months afterward (95% CI, 2.0 to 3.8; P<0.001).
Rates of Postapproval Safety Problems
Summary statistics for our measures of postapproval safety problems appear in Table 1, along with a cross-tabulation of the just-before-deadline approval indicator with the postmarketing safety variables. The rates of postapproval safety problems are shown in Figure 2, Figure 3, and Figure 4, which compare the risk of safety problems after just-before-deadline approvals with the risk of problems after approvals at other times; these odds ratios (from exact logistic regressions) are displayed along with their 95% confidence intervals.
The rate at which drug approvals were followed by postmarketing safety problems was significantly higher for products approved in the 2 months before the PDUFA deadlines than for drugs approved at all other times. For approved new molecular entities submitted between January 1993 and December 2004, drugs that were approved just before the deadline had a higher rate of subsequent safety-based withdrawals than drugs approved in other months of the review cycle (odds ratio, 5.5; 95% CI, 1.3 to 27.8; P=0.02). Just-before-deadline approvals were more likely to be followed by assignment of a black-box warning (odds ratio, 4.4; 95% CI, 1.2 to 20.5; P=0.02) or discontinuation of at least one dosage form (odds ratio, 3.3; 95% CI, 1.5 to 7.5; P=0.003) than were other approvals. Such discontinuations had a high positive correlation with safety-based market withdrawals over the past 15 years (odds ratio, 13.9; 95% CI, 3.3 to 69.1; P<0.001) and a positive correlation with the addition of a black-box warning to a drug's label (odds ratio, 4.6; 95% CI, 0.9 to 19.7; P=0.06).
We found a similar postmarketing safety pattern for the comparison of just-before-deadline approvals with approvals made just after the deadline (Figure 3). As compared with drugs approved after the deadline, those approved just before the deadline were more likely to be withdrawn from the market for safety reasons (odds ratio, 5.7; 95% CI, 1.2 to 37.6; P=0.03), more likely to have a black-box warning added after marketing (odds ratio, 4.0; 95% CI, 1.3 to 16.7; P=0.01), more likely to have one or both of these outcomes (odds ratio, 3.9; 95% CI, 1.3 to 13.1; P=0.01), and more likely to have at least one dosage form discontinued (odds ratio, 3.3; 95% CI, 1.4 to 8.1; P=0.005).
Comparison with Early Approvals
We also compared the rates of postmarketing safety problems for just-before-deadline approvals with the rates for drugs approved well in advance of the PDUFA deadlines (within 6 months after submission for standard new molecular entities and within 4 months after submission for priority entities) (Figure 4). As compared with drugs approved well before the deadline, those approved just before the deadline were more likely to be withdrawn for safety problems (odds ratio, 5.3; 95% CI, 1.3 to 26.5; P=0.005) and more likely to have at least one dosage form discontinued (odds ratio, 3.3; 95% CI, 1.4 to 7.7; P=0.003). Drugs approved just before the deadline were also more likely to have a black-box warning added subsequently, but the odds ratio was not significant (4.4; 95% CI, 0.9 to 28.4; P=0.07). In addition, just-before-deadline approvals were more likely than early approvals to be followed by withdrawal of the drug from the market, assignment of a black-box warning, or both (odds ratio, 4.3; 95% CI, 1.3 to 15.0; P=0.01). In other analyses (see the Supplementary Appendix), we found no evidence for a relationship between the duration of the review itself and postmarketing safety problems; including the approval time as a linear regressor did not result in any significant relationships (P>0.3 for all comparisons).
Correlates of Drug Risk
We found no evidence of significant associations between the timing of approval and correlates of drug risk. As compared with drugs approved at other times, those approved just before the deadline were not more likely to have first-in-class status (odds ratio, 0.9; 95% CI, 0.4 to 2.3; P=0.90),16 to undergo review by a premarketing advisory committee (odds ratio, 1.4; 95% CI, 0.8 to 2.4; P=0.29),27 or to be associated with high hospitalization rates for the primary indication (odds ratio, 1.0; 95% CI, 0.9 to 1.2; P=0.69).28 These results suggest that drugs approved immediately before the PDUFA deadlines were not inherently more risky than those approved at other times.
We used a data set of all new molecular entities (drugs whose active ingredient has never before been approved in the United States) reviewed and approved by the FDA during the period from January 1950 through December 2004.16 First, we determined whether the user-fee-imposed deadlines changed the pattern of FDA reviews by conducting a dynamic Cox proportional-hazards estimation of review times for all new molecular entities submitted between 1992 and 2005 (313 submissions). The Cox models controlled for the staffing resources of the FDA's drug-reviewing division, and they also included a random-effect term for the product's primary indication. We compared the month-specific timing of approvals for new molecular entities before the enactment of the PDUFA with the pattern of approvals after enactment of the user-fee laws in 1992 and their renewal in 1997. (Cox models were estimated with the use of S-Plus 8.0 software.17) The basic estimate reported is that of an approval rate ratio, the likelihood of an approval decision in the 2-month interval before the deadline imposed by the user-fee laws as compared with the likelihood of an approval decision at all other times. We hypothesized that under the PDUFA, approval decisions would be much more likely during the 2 months immediately before the deadline.
We next examined the potential consequences of the PDUFA deadlines. The first implementation of the PDUFA, in 1992 (PDUFA I), required the FDA to review and act on 90% of standard new molecular entities within 12 months and 90% of priority new molecular entities within 6 months. The 1997 renewal of the PDUFA (PDUFA II) required the FDA to review 30% of standard new molecular entities within 10 months by 1997 and to review 90% within 10 months by 2002; the same deadlines continued to apply to priority entities. The 2002 reauthorization of the law (PDUFA III) maintained the PDUFA II criteria. We compared the rate of postapproval safety problems for drugs approved just before a PDUFA deadline with the rate for drugs approved in other months. The indicators of safety problems that we used included new black-box warnings, withdrawals because of safety problems, and dosage-form discontinuations. In all analyses, we controlled for the number of years the drug was on the U.S. market. Safety-based withdrawals and black-box warnings are the most important indicators of unanticipated postmarketing safety problems.18
Measures of Postmarketing Safety Problems
For approved new molecular entities submitted between 1993 and 2004, withdrawals were determined from Scrip reports19 and Pharmaprojects,20 which maintain records of all safety-based withdrawals from national pharmaceutical markets. For safety-based withdrawals occurring before 2000, we checked these data against an article by Fung and colleagues.21
For drugs submitted between 1993 and 1999, we used an analysis by Lasser and colleagues to determine whether a black-box warning was added after marketing.18 For the period from 1999 through 2004, we referred to a listing maintained by the Kansas University Medical Center.22
A secondary measure was the rate at which dosage forms of the drug were discontinued from clinical use, an action that is often but not always connected with issues of safety. The DrugsatFDA database contains a comprehensive list of drugs approved by the FDA, as well as a list of all changes to a drug's approval package that require an official amendment.23 For each new molecular entity, we determined whether a dosage-form had ever been discontinued. Because dosage-form discontinuations are not always related to safety or efficacy, we sought to determine whether they were useful markers of safety problems by estimating their association with the other two safety measures - safety-based withdrawals and black-box warnings.
Measure of Deadline Effects
For most analyses, we created a composite measure of approval just before the deadline, which indicated whether a drug was a standard new molecular entity approved in the 11th or 12th month of review under PDUFA I, a standard new molecular entity approved in the 9th or 10th month of review under PDUFA III, or a priority new molecular entity approved in the 5th or 6th month under PDUFA I or later user-fee laws. Multiple-cycle reviews (in which the FDA issues what is called an approvable letter on or before the user-fee deadline and asks for a resubmission) were coded as having missed the initial deadline. According to a recent report, 42 (69%) of approvals of new molecular entities from 2002 to 2004 were first-cycle approvals.24 Using a single variable - the last 2 months before deadline - made it possible to combine the effects of deadlines for different review types and different user-fee requirements.
We first conducted logistic-regression analyses of the postmarketing safety variables as a function of the submission year (time trend), descriptors noting predeadline versus postdeadline approvals, and variables for the drug's primary indication and for its sponsoring firm. We then used exact methods25 to analyze the same data, using LogXact8 software26; more detailed methods and output are reported in the Supplementary Appendix, available with the full text of this article at www.nejm.org.
Using these estimates, we then assessed differences in the risk of postapproval safety problems for drugs approved within 2 months before the PDUFA deadline as compared with drugs approved under three control conditions: all other approved drugs (those approved in all months other than the last 2 months before the deadline), drugs approved in the 2 months after the deadline, and drugs approved well before the deadline (in the first 6 months for a standard new molecular entity and in the first 4 months for a priority entity). Results for all postmarketing safety analyses are reported as odds ratios based on exact logistic regressions.
We considered whether our findings could be confounded by the possibility that drugs approved in the last 2 months before the PDUFA deadlines might be inherently riskier than drugs approved at other times. To assess this possibility, we looked at whether just-before-deadline approval status was associated with three likely correlates of drug risk: whether the drug was the first in its therapeutic class to be approved, whether the drug was reviewed by an advisory committee before final approval, and the number of hospitalizations per year (per 100,000 population) for the drug's primary indication.