| |
Determination of Clinically Relevant Cutoffs for HIV-1 Phenotypic Resistance Estimates Through a Combined Analysis of Clinical Trial and Cohort Data
|
| |
| |
Dowload the PDF Virco Cutoffs here
JAIDS Journal of Acquired Immune Deficiency Syndromes:Volume 48(1)1 May 2008pp 26-34
Winters, Bart MSc*; Montaner, Julio MD; Harrigan, P Richard PhD; Gazzard, Brian MD; Pozniak, Anton MD; Miller, Michael D PhD; Emery, Sean PhD; van Leth, Frank MD; Robinson, Patrick MD#; Baxter, John D MD**; Perez-Elias, Marie MD; Castor, Delivette MPH; Hammer, Scott MD; Rinehart, Alex PhD; Vermeiren, Hans PhD*; Van Craenenbroeck, Elke PhD*; Bacheler, Lee PhD
From *Virco BVBA, Mechelen, Belgium; British Columbia Centre for Excellence in HIV/AIDS, Vancouver, British Columbia, Canada; Chelsea and Westminster Healthcare National Health Services Trust, London, United Kingdom; Gilead Sciences, Foster City, CA; National Centre in HIV Epidemiology and Clinical Research (NCHECR), University of New South Wales, Sydney, Australia; International Antiviral Therapy Evaluation Center (IATEC), Amsterdam, The Netherlands; #Boehringer Ingelheim, Ridgefield CT; **Cooper University Hospital/University of Medicine and Dentistry of New Jersey-Robert Wood Johnson Medical School, Camden, NJ; Ramn y Cajal Hospital, Madrid, Spain, New York Academy of Medicine, New York, NY; Columbia University, New York, NY; Tibotec Therapeutics, Durham, NC; and VircoLab, Inc., Durham, NC.
Received for publication July 27, 2007; accepted February 13, 2008.
Abstract
Background: Clinically relevant cutoffs are needed for the interpretation of HIV-1 phenotypic resistance estimates as predicted by virtual phenotype HIV resistance analysis.
Methods: Using a clinical data set containing 2596 treatment change episodes in 2217 patients in 8 clinical trials and 2 population-based cohorts, drug-specific linear regression models were developed to describe the relation between baseline characteristics (resistance, viral load, and treatment history), new treatment regimen selected, and 8-week virologic outcome.
Results: These models were used to derive clinical cutoffs (CCOs) for 6 nucleoside/nucleotide reverse transcriptase inhibitors (zidovudine, lamivudine, stavudine, didanosine, abacavir, and tenofovir), 3 unboosted protease inhibitors (PIs; indinavir, amprenavir, and nelfinavir), and 4 ritonavir-boosted PIs (indinavir/ritonavir, amprenavir/ritonavir, saquinavir/ritonavir, lopinavir/ritonavir). The CCOs were defined as the phenotypic resistance levels (fold change [FC]) associated with a 20% and 80% loss of predicted wild-type drug effect and depended on the drug-specific dynamic range of the assay.
Conclusions: The proposed CCOs were better correlated with virologic response than were biological cutoffs and provide a relevant tool for estimating the resistance to antiretroviral drug combinations used in clinical practice. They can be applied to diverse patient populations and are based on a consistent methodologic approach to interpreting phenotypic drug resistance.
Phenotypic resistance testing of HIV-1 strains can provide an accurate, quantitative assessment of alterations in drug susceptibility in comparison to a standardized reference strain.1,2 To use these quantitative test results optimally, especially in clinical practice, interpretation of the results is required. Initially, the designation of HIV resistance status used in phenotypic assay systems was based on technical assay performance. Typically, 2.5-, 4-, or 10-fold changes (FCs) in drug susceptibility were arbitrarily used to define drug resistance to specific antiretroviral drugs (ARVs). The finding that there are large differences in the distribution of phenotypic drug susceptibility to ARVs among HIV variants from treatment-naive individuals3,4 led to the redefining of these technical cutoffs according to the natural variation of phenotypic susceptibility. Although these biologic cutoffs (BCOs) were an improvement over the arbitrary technical cutoffs, there was still no direct association between these values and clinical outcome. Efforts to address this have been undertaken for some ARVs. For example, it has been reported that a 1.4-FC in the median inhibitory concentration (IC50; Antivirogram assay [AVG]; Virco, Mechelen, Belgium) was associated with a small reduction in virologic response to tenofovir (TDF) in ARV-experienced patients, whereas a 3.8-FC was associated with a strongly reduced response or no response at all.5 In other studies of patients failing HIV protease inhibitor (PI)-based therapy, clinically relevant phenotypic cutoffs associated with poorer virologic and clinical outcomes have been estimated at 4- to 8-fold for indinavir (IDV) and ritonavir (RTV) and at 2.5- to 8-fold for saquinavir (SQV) using an in-house recombinant virus PI susceptibility assay.6 Clinical breakpoints for abacavir (ABC) have also been reported for HIV phenotype determined with the Monogram Biosciences (South San Francisco, CA) PhenoSense (PS) assay (4.4- and 6.3-fold) or the AVG (3.2- and 7.5-fold).7 The need for clinically evaluated analyses for the interpretation of genotypic drug resistance tests has been highlighted in a position paper.8
In this study, we present a novel approach for deriving clinically relevant cutoffs of estimated HIV-1 phenotypic resistance information for HIV nucleoside/nucleotide reverse transcriptase inhibitors (NRTIs) and PIs. To create the predictions, we used virco TYPE HIV-1 v. 4.0.00 (vT), a resistance analysis system that predicts phenotypic FC from mutational sequences. This approach treats all drugs consistently and is applicable to diverse treatment combinations and patient populations. We have evaluated the model on independent test data whenever possible and assessed the performance of clinical cutoffs (CCOs) compared with BCOs for predicting virologic response of drug-resistant HIV variants.
DISCUSSION
Resistance to a drug is a continuum rather than a black and white phenomenon. Phenotypic FC in IC50 reflects this continuum; however, to interpret it, milestones are needed that link FCs or IC50 values to clinical response. FC can then be interpreted by comparing it with these CCOs, bearing in mind that clinical response decreases as the FC and resistance increase. Historically, CCOs have been proposed for several drugs; however, a wide variety of definitions and methods were used to derive them, making it difficult to interpret resistance to all available drugs in a consistent way. We decided to define CCOs using a relative measure of resistance (ie, comparing the viral load drop of a particular FC with the viral load drop of a wild-type virus under similar circumstances), adjusting for the backbone therapy, baseline viral load, and treatment experience. Baseline characteristics are generally unknown to the provider of a resistance test. The proposed CCO approach can be applied to patients with different baseline characteristics without knowing the specific values for each characteristic, in contrast to statistical approaches, which rely on absolute viral load responses that are confounded with other baseline characteristics such as the baseline activity of the backbone therapy. In some cases, the lower CCOs are close to the predicted FC of a wild-type virus, suggesting that resistance is starting to play a role as soon as the FC increases to greater than the predicted FC of the wild-type virus. These small differences may sometimes not be reliably detected using a conventional phenotypic assay because of inherent assay variability, but they can be reliably detected using a predicted phenotype approach that gains precision by evaluating the impact of resistance mutations in a large number of samples.
The interpretation of FC in the context of CCOs should not be used to compare whether, in general, a specific drug is more potent or has a higher genetic barrier than another drug. Because the dynamic range of the assay varies from drug to drug, it is also not appropriate to compare drugs based on absolute FC values; each FC should be interpreted in the context of the dynamic range for the drug and the corresponding CCOs. To illustrate this, we can compare the dynamic range and the CCOs of AZT and TDF. The predicted FC for AZT varies from 0.8 for wild-type viruses to 38 for highly resistant viruses, whereas the dynamic range of TDF varies from 0.8 to 4.1. In general, because of the wider dynamic range, the CCOs and predicted FCs for resistant viruses and the CCOs might be expected to be higher for AZT than for TDF; this is independent of the potency or genetic barrier of both drugs.
An important goal of ARV therapy is to provide durable suppression of viral load well beyond an initial 8-week response. Nevertheless, in defining CCOs, we chose to focus on the initial week 8 response rather than on responses at 24 or 48 weeks, mainly because of the differential dropout rate expected among failing patients and patients with high resistance to the received treatment. Such dropouts can be common, for poorly defined reasons, in the clinical cohort data that form a substantial proportion of the outcome data used in the current analysis. The relation between baseline susceptibility and treatment response is further diluted at extended time intervals by the impact of other important factors such as adherence and side effect profiles.
Although CCOs and BCOs are unrelated concepts (CCOs are determined based on virologic response in treated patients, whereas BCOs simply indicate the normal range of in vitro FC values among treatment-naive viruses), validation was done comparing the new CCOs with BCOs based on their correlation with virologic outcome. It was demonstrated that an interpretation using CCOs is better correlated with clinical outcome than an interpretation using BCOs. CCOs give the interpreter of the resistance test a better idea of the response continuum, and this enables the selection of drugs that retain a substantial degree of activity, making the CCOs an important tool, especially in those patients with limited treatment options.
CCOs were not derived for all available drugs. The clinical database did not contain enough observations for some older drugs that are rarely used today, such as zalcitabine (ddC) and RTV. In the case of some newer drugs (atanavir [ATV]/r, fosamprenavir [FPV]/r, tipranavir [TPV]/r, and darunavir [DRV/r]), derivation of the CCOs has depended more heavily on outcome data from phase 2 and phase 3 trials, in collaboration with the various pharmaceutic sponsors developing these drugs. Use of data from these select patient populations presents additional issues requiring specific attention and discussion of the drawbacks and possible solutions, which are to be addressed in future articles.
These proposed CCOs should be refined and validated on an ongoing basis, not only by gathering more clinical data to ensure broad applicability but to take new therapeutic strategies into account. Finally, an in-depth analysis of the treatment effect over time should give better insight into the durability of the regimen selected based on a resistance test at baseline. The CCO values determined here should not be extrapolated to other phenotypic tests, because each assay has its specific properties that may affect the CCO values.
In summary, the CCOs presented here were determined in a uniform way using a heterogeneous patient population taking a wide range of ARV regimens. As such, we believe they are broadly applicable for use in clinical practice. They are likely to increase the value of genotypic HIV drug resistance testing using the vT approach. The CCOs described here have been implemented in the vT resistance analysis.
METHODS
Clinical Data Sources
Clinical data from 2 clinical cohorts (British Columbia Center for Excellence in HIV/AIDS, Vancouver, British Columbia, Canada; Chelsea and Westminster Healthcare National Health Service Trust, London, United Kingdom) and 8 controlled clinical studies [2NN trial,9 CREST,10,11 Gilead GS-99 to 907(14), VIRA3001,12 GART,13 RESA14 2026, CERT,15 and a trial of modified directly observed therapy New York Academy of Medicine (NYAM)16] were used to construct a clinical database in Oracle.
Treatment regimens included in the analysis had to meet the following inclusion criteria: a partial or complete regimen change (defined as a discontinuation or a dose change of 1 or more drugs in the regimen or the addition of a drug that was not present in the regimen) must have occurred after a resistance test, baseline sequence and viral load data within 3 months of starting a new regimen had to be available, the new regimen had to be stable for at least 4 weeks, no experimental ARV treatments were allowed in the background regimen, and viral load data 8 weeks after beginning a new regimen had to be available. Viral load data at week 8 was selected as the viral load closest to day 54 of the treatment within a window ranging from 25 to 84 days. Separate data sets were created for each drug; each treatment regimen contributed to data sets for each of the drugs in the regimen. Ritonavir-boosted (r) and nonboosted PIs were modeled separately. Only enteric-coated tablets were selected for the didanosine (ddI) data set. The data set for SQV/r only contained the hard-gel formulation at daily doses of at least 2000 mg.
Part of the clinical data set was set aside for validation purposes. This validation data set consisted of 1888 additional treatment episodes derived from clinical cohorts and from several clinical trials not used for clinical cutoff (CCO) development received after July 2005. Notice that patients from these studies with a regimen containing a drug for which <200 records were available in the development set were assigned to the development data set rather than to the validation set to increase the robustness of the CCO estimates.
Development of Statistical Models of Virologic Response
Predicted phenotypic drug susceptibility was quantified using the vT analysis system, which predicts phenotypic drug resistance from HIV genotype using linear regression models.17
Parametric linear regression models18 for censored data (which are also used for time-to-event modeling using the LIFEREG procedure in SAS v8.2 [SAS Institute, Cary, NC] as described by Hughes19) were developed to model the change in plasma viral load from baseline to week 8 on the new treatment regimen using the following model:
The model included terms for the intercept, baseline viral load (VLBaseline), baseline FC of the drug under investigation (FCBaseline), baseline phenotypic sensitivity score of the entire background regimen (cPSSTotal [number of active drugs taken in addition to the drug under investigation]20,21), drug class-specific phenotypic sensitivity scores (PSSNRTI and PSSPI), and terms for treatment history (treatment naive [Naive] and naive to NRTIs [NRTI_Naive] for NRTI models or PIs [PI_Naive] for PI models). A nonnucleoside reverse transcriptase inhibitor (NNRTI)-specific activity score was not included to avoid overparametrization, because the sum of the drug class-specific activity scores corresponds with the cPSSTotal. FC was transformed using a power transformation. Powers (p) ranging from -3 to 1 were evaluated in steps of 0.1. The power resulting in the model with the lowest standard deviation (SD) of the error term was used in the final model.
All the cutoffs were optimized simultaneously, and the CCO estimates for one drug had an effect on the CCO estimates for the other drugs. Initially, the cPSSTotal was calculated using the vT BCO,21 and a drug was considered active if the predicted FC was less than or equal to the BCO. When the first version of the CCO was available, the CCO estimates were used to calculated the cPSSTotal. Drugs were considered to be fully active if the predicted FC was less than or equal to the lower CCO, and they were considered to be inactive if the predicted FC was greater than the upper CCO. If the predicted FC was between the CCO estimates, the activity was determined using linear interpolation, as described elsewhere.20,21 Analyses were iterated with subsequent CCO estimates until CCO estimates remained stable. A standard ARV dose was assumed if this information was missing. RTV doses up to 800 mg/d were considered boosting doses, whereas doses ≥800 mg/d were considered fully active. Additional parameters (number of active NRTIs or PIs taken in the background and treatment history parameters) were selected by backward elimination at a 5% significance level.
Statistical models for the NNRTIs were not pursued for NNRTI CCO determination, because the utility of NNRTI CCOs remains questionable at this time.
Definition of Clinical Cutoffs
Using the treatment response models, the impact of baseline viral resistance to individual drugs on overall regimen response (defined as the change in viral load 8 weeks after initiating the new regimen) was assessed. The difference in predicted response between a wild-type susceptible virus strain and a fully resistant strain (defined as percentile 97.5 of the vT linear model-predicted FC values among >200,000 genotypes of clinical isolates) was taken as a measure of the effect of a single drug. Phenotypic resistance levels (FC) associated with 20% and 80% losses of this single drug effect were determined and defined as lower (CCO1) and upper (CCO2) CCOs. The variability of the proposed CCOs was assessed by bootstrapping based on 1000 repeats, and 95% confidence intervals were determined.
Model Performance and Validation of Clinical Cutoffs
A global performance comparison between the newly defined vT CCOs and previously used BCOs was made by testing the association of the cPSS of the entire regimen and response using 3 metrics. Area under the receiver-operator characteristic curve (as a measure for diagnostic accuracy) and odds ratios per unit increase in cPSS unit were used to express the association between cPSS and response rate. The Pearson correlation coefficient was used to assess the correlation between the cPSS and viral load drop. This analysis was conducted on the data set used for CCO development and on the unseen validation data set.
To illustrate the relevance of resistance classes as determined by CCOs and compare them with resistance classes defined by BCOs, the response rate and the median viral load drop per resistance class and per drug were determined. A responder at week 8 was defined as achieving a drop of at least 1 log compared with baseline at week 8 or an undetectable viral load at week 8. A responder at week 24 was defined as an individual with a drop of 1 log compared with baseline at week 24 or an undetectable viral load at week 24. Dropouts were considered as nonresponders in the week 24 analysis.
RESULTS
Description of the Analysis Data Set
The development data set contained 2596 treatment change episodes in 2217 patients (Table 1). Most of the patients were male (82%) and treatment experienced (88%). The median baseline CD4 cell count and viral load varied around 200 cells/ƒÊL and 4.5 log10 copies/mL, respectively (Table 2). Most of the regimens consisted of at least 3 drugs (ranging from 93% in the ddI population to 100% in the TDF population), and most patients took 1 or 2 active drugs in addition to the drug for which CCOs were being defined (from 47% in the boosted SQV population to 78% in the unboosted IDV population). Patients taking PIs tended to be more treatment experienced, as shown in Table 2. There also seemed to be a difference within the NRTIs with the ABC, TDF, and ddI populations containing more treatment-experienced individuals. Baseline characteristics of individual drug data sets are shown in Table 2. The development data set included 738 different drug combinations. The most common combinations included an NNRTI with 2 NRTIs (EFV + zidovudine [AZT] + lamivudine [3TC] [n = 115], EFV + stavudine [d4T] + 3TC [n = 127], NVP + d4T + 3TC [n = 166], and NVP + AZT + 3TC [n = 84]).
Approximately two thirds of the treatment regimen data originated from the clinical cohort data sets, and the remaining records came from clinical trials.
The same limit of detection was not used in all studies; nevertheless, the proportion of regimens with censored 8-week viral load values in each data set was moderate (ranging from 18% [IDV/r] to 33% [amprenavir (APV)] of the values).
Predicted Virologic Response to NRTIs and PIs and Determination of Clinical Cutoffs
Figure 1A illustrates the predicted 8-week change in viral load from baseline for AZT-containing treatment regimens as a function of baseline AZT FC for 3 different combinations of baseline characteristics as an example. The linear regression model predicts the greatest virologic response for patients whose virus is fully susceptible to AZT; the overall response to the new AZT-containing treatment regimen decreases as baseline resistance to AZT increases. Overall predicted regimen response also varies with other factors used in the model (eg, cPSS, baseline viral load) in addition to the baseline AZT FC; the response is reduced in patients whose regimen included fewer active drugs in combination with AZT (background cPSS = 1 or 0) or higher baseline viral load. No significant interaction effects were detected between baseline FC and other factors in the model with the current amount of available clinical data. Some model properties correlating predicted and observed viral load change (SD of the error term; power used to transform the baseline FC; and the c-index, a widely applicable measure of predictive discrimination22) are presented in Table 3. We defined a wild-type or reference response to AZT as the difference between the overall regimen response predicted for a wild-type fully AZT-susceptible virus and the diminished regimen response predicted for a strain fully resistant to AZT for patients with identical baseline characteristics. The shape of the response curve is determined by the transformation of the baseline FC, which is optimized on a drug-by-drug basis. These power transformations are presented by drug in Table 3. In Figure 1B, the predicted response was expressed as a percentage of the reference response rather than as an absolute value. Importantly, by comparing the response of a patient's virus with the response of wild-type virus in a patient with identical baseline characteristics, we can normalize the responses of all patients. Although the absolute magnitude of the viral load response varies among patients with different baseline characteristics, the predicted percentage of a reference response is independent of these baseline characteristics.
The predicted loss of virologic response as a function of baseline resistance is shown for NRTIs (Fig. 2A) and PIs (see Fig. 2B). ARV activity of all NRTIs except AZT was rapidly lost as susceptibility decreased. For all NRTIs except AZT, the models predicted >80% loss of response within a 2-fold increase in IC50 greater than the FC associated with a 20% loss of response. AZT, conversely, exhibited a much more gradual loss of ARV activity in response to decreasing susceptibility and was predicted to retain approximately half of its activity even after a 3-FC in IC50. For the PI class, all unboosted PIs rapidly lost ARV activity with increasing resistance levels, whereas PI/r exhibited more sustained activity despite increasing resistance. Lopinavir (LPV)/r, IDV/r, SQV/r, and APV/r were predicted to retain approximately half of their ARV activity after a 30-fold, 25-fold, 17-fold, and 3-fold increase in IC50, respectively. Note that the predicted loss of response at an identical FC differs from drug to drug. Furthermore, the graphs show substantial differences in dynamic range within and between drug classes, with a wider dynamic range for AZT, 3TC, and the PIs as compared with the other NRTIs.
Using the response models illustrated in Figure 2, CCOs indicating the baseline FC values associated with 20% and 80% loss of the 8-week reference response of a wild-type virus can be defined easily. An overview of the obtained CCOs for each antiretroviral agent and their 95% confidence intervals is presented in Table 3. The relative precision was higher for some CCO estimates (eg, d4T, LPV/r) than for others (eg, ddI, APV/r), and the variability around the NRTI estimates was generally lower than around the PI estimates. There were less PI observations available, and the FCs of the PIs were spread over a wider range of possible values than those of the NRTIs. The CCO estimates are likely to become more precise over time as more data become available.
TABLE 3. Overview of Some Model Fit Characteristics and the Estimated CCOs and Their 95% Confidence Intervals
Validation
Baseline resistance assessed by BCOs or CCOs was strongly associated with response in both data sets. The cPSS as determined by CCOs was better associated with actual virologic response at 8 weeks as compared with the cPSS by BCOs in the development data set as well as in the independent validation data set. Table 4 shows a significant improvement in prediction of week 8 virologic response in favor of the CCOs for all 3 measures in the development and the validation data set.
Illustrations of Virologic Response by Resistance Class in the Clinical Database
The response rate and the median viral load drop per resistance class and per drug are depicted in Figure 3 (week 8) and Figure 4 (week 24). Although activity of a single drug is confounded with the background activity in combination therapy, it is clear from the figures that CCOs reflect the continuous aspect of phenotypic susceptibility better, and therefore allow a more subtle interpretation of resistance. A consistent decline of response rate was observed as resistance increased, looking at the week 8 response and the week 24 response, even though the dropout rate was high at week 24 (ranging from 37% [TDF] to 50% [NFV]).
Limitations
Our approach to CCOs evaluates individual components of a combination therapy. It does not reflect the response to the entire regimen, and it does not indicate how individual drugs should be combined. In using this system of resistance interpretation, it should be borne in mind that drug potency is not included in the proposed CCO models; thus, for example, 50% activity of an extremely potent drug may be more desirable than 80% activity of a less potent drug. Furthermore, the percentage loss of wild-type response only addresses the loss of response attributable to resistance. Many complexities of therapy (eg, residual activity, adherence, drug interactions) that could affect virologic outcome were not considered in this analysis.23
|
|
| |
| |
|
|
|
