icon-folder.gif   Conference Reports for NATAP  
  AIDS 2002 Barcelona
Barcelona, Spain July 7-12 2002
Back grey_arrow_rt.gif
GUESS Study: inconsistencies by experts in interpreting resistance test results
Reported by Jules Levin
  This report contains results from 2 studies reflecting the difficulties in using resistance testing. Also in this report is a commentary by me on using resistance testing. I think resistance testing can be helpful in making treatment decisions in the hands of the right person but I also think test results in the wrong hands can be harmful. Education about resistance testing is crucial.
At the Barcelona Conference, Andew Zolopa, MD, (Stanford University) reported results from the GUESS Study in an oral session on resistance testing. He said that although the use of a resistance test may help in outcome (in picking a regimen that is more effective), he conducted this study to evaluate how much agreement exists within the expert community in the interpretation of genotypic testing. He looked at 45 patient specimens and matching phenotypes from Virco. By looking at the genotype experts were able to predict the phenotype about 30-40% of the time for nukes and protease inhibitors, except for 3TC which was correctly predicted about 70% of the time. For NNRTIs they could predict it about 67% of the time. The experts tended to over-estimate the phenotypic fold-change (amount of phenotypic resistance) for most nukes. The experts tended to disagree quite a bit for nukes and protease inhibitors (50%), but less so for NNRTIs. Although there was a surprising amount of disagreement for the NNRTIs and 3TC as well-about 20-25%. Zolopa reported the experts disagreed in predicting drug activity about 50% of the time for nukes and protease inhibitors, and 27% of the time for 3TC and about 20% of the time for NNRTIs. There was concordance, agreement by the experts, of about 80% in whether to recommend or not recommend a specific drug based on the genotype. Experts predicted phenotype fold-change within 1 category 70% of the time. Zolopa concluded: (1) there was inconsistent accuracy among experts in predicting phenotypic resistance from genotype resistance test results; (2) there was variable agreement among experts in translating genotype to phenotype or drug activity; (3) there were high levels of agreement in the treatment recommendations by the panel of experts; (4) outcomes of recommended treatments needs to be assessed in future studies. I recall that Zolopa stated he would expect similar discrepancies in interpretation by experts in using phenotypic resistance testing.
Refining The Use of Resistance Testing (Commentary)
Resistance testing is a relatively new diagnostic development in HIV. It started to emerge about 4 years ago. Resistance testing was the subject of a good deal of discussion and attention at the Resistance Workshop. I think it's apparent that it needs further refinement in the way it is used. This field is very confusing, in that studies suggest interpreting resistance testing results and using them to make treatment decisions is more complicated than we thought. More studies suggesting this are emerging. At the Resistance Wksp several researchers commented that we need to you learn how & when to use resistance testing; and I agree. I think the resistance testing should be used situationally. There are some circumstances in which testing can be helpful, if the doctor interpreting test results understands the test results, the limits of the testing & results, and how to mesh the results with the patient treatment history. These are a lot of ifs. For certain drugs testing may be more helpful but for certain other drugs testing may not be so helpful. For certain patient situations testing may be helpful & for certain other patient situations testing may not be so helpful.
Another question is genotype vs phenotype. Should you use genotype or phenotype or both. Whichever test you use, patient treatment history is crucial. There are little data comparing which may be better - the genotype test or the phenotype test. There is some preliminary research comparing the various resistance testing technologies: geno vs pheno, geno vs virtual pheno, but more research will be needed to better evaluate comparisons between the tests. We also need better studies to identify how & when using a test can benefit a patient. Following that comes the task of teaching doctors how to use the tests. The challenges to refining the use of testing are many and it will take time and efforts to address them. If you were to give the same set of genotype or phenotype test results to the leading resistance experts in the world and asked them to interpret them, you might get some differences in opinion. Andrew Zolopa, MD, Stanford University, reported results from the GUESS Study at Barcelona in support of this.
Some researchers believe the genotype test is more reliable while others believe the phenotype is more reliable. If you have the resources it might be better to do both tests but make sure your treatment history is adequately considered, and that whoever is making a treatment recommendation to you knows what they are doing. Bear in mind, improper interpretation of a test result or printout from the lab conducting the test can result in a bad treatment decision. Also bear in mind that reliance on the lab test recommendation can be a mistake, as their interpretation can be wrong.
If you get a genotypic mutation or several reported on a test result this can rule out the usefulness of certain drugs. But the absence of detectable resistance does not mean that you don't have resistance; it may just not be detectable. See how complicated resistance testing can be. Some lab test results, for example a Quest lab test result, may say you are sensitive to AZT but you may have very little sensitivity to AZT. Does your doctor understand all these subtle complications? Or does he/she just read the sometimes faulty recommendations on the printed lab test result the doctor receives?
There are a number of algorithms doctors can use to interpret test results. An algorithm is a system into which you can plug your resistance test results and it will interpret the results for you. The problem is that there are numerous algorithms and they may offer different interpretations. There was discussion at the Wksp on how to bring these different systems into agreement, but I think this task has tremendous barriers & challenges. Another suggestion that can help doctors interpret tests are certain websites where expert resistance researchers are available to help the doctor actually interpret the test results. An example of such a site is the Stanford University site. Does your doctor know about this? You can suggest it to him & see if he/she is open to using it. It takes time to do this. It is less time consuming for a doctor to just accept the recommendations on the printed lab report they receive.
At the Intl Conference a research group reported discrepancies in interpretation between 3 algorithms they studied. Samples were taken from 293 HIV infected patients with treatment failure to evaluate the concordance of genotypic data between 3 algorithms: Stanford University Database, TruGene (Visible Genetics), and VirtualPhenotype (Virco). The obtained genotypes were interpreted by the 3 algorithms to predict susceptibility for 14 ART drugs. Complete concordance results among the 3 systems for all the drugs studied were found in 40/293 (14%) of samples. Low concordance (50%) was observed for most NRTIs, while results more highly agreed (70%) for all NNRTIs and most of Pis. Discordant interpretations between Stanford & Virco were found in over 50% of samples for ddI, d4T, and abacavir. The level of disagreement between VGI & Virco exceeded 40% for the same drugs. Major discrepancies (high level resistance interpretation by 1 algorithm with sensitive interpretation by another) were observed between VGI & Virco in over 10% of cases for ddI, ddC, d4T, and abacavir. Most interpretations agree on 3TC 90% of the time. The authors concluded that this study demonstrates the great level of discordance in the interpretation of genotyping results among algorithms, clearly showing the need for clinical validation. They suggest a joint effort is needed to achievw a consensus on interpretation of genotypic data (MoPeB3125, GH Kijak, Harrigan, Montaner, Cahn et al). At the Resistance Wksp researchers discussed this and agreed such an effort is required but I have concerns regarding the success of such an effort. Nonetheless, until such consensus is found the interpretation of resistance test results is subject to many vagaries and the risk for mistakes in treatment decisions.
This is a discussion and commentary by me on the limitations of resistance testing but it's not meant to be exhaustive. My point is there are limitations and refinement is needed in how to use testing and advances are required in educating doctors & patients in using testing properly. Improper interpretation of test results is common & can result in wrong treatment decisions & failing a drug and drug class. A high price to pay.