Re: "Derivation and Validation of the Denver Human Immunodeficiency Virus (HIV) Risk Score for Targeted HIV Screening"

Abstract

Haukoos et al. developed a commendable risk score for prescreening for human immunodeficiency virus (HIV) testing in clinical settings (1). Their thoughtfully constructed model contains a number of important predictors of HIV infection. Although the authors concluded the contrary, the predictive utility of the model, as displayed in the receiver operating characteristics curve, seemed to us to strongly endorse the United States Centers for Disease Control and Prevention (CDC) guidelines for general HIV screening with highly sensitive tests (2). The Denver sexually transmitted infection clinic data reflect a 0.5% prevalence of undiagnosed infection and an annual caseload of 10,000 patients (50 HIV-infected patients). Although values of sensitivity/specificity used to create Figure 2B in the article by Haukoos et al. were not reported, the joint maximum for both appears to be 80%/80% sensitivity/specificity on the curve for the derivation data. Applying the risk score at this cutpoint would result in referral of 2,030 patients for HIV-testing and detection of 40 HIV-positive patients, with 10 HIV-positive patients who received low risk scores not being tested. A value of the receiver operating characteristics curve that attains 95% sensitivity appears to occur at 20% specificity. Applying these properties, all but 2 HIV-infected patients would be detected, with the trade-off of conducting HIV testing on 8,008 patients. To identify all 50 infected patients, the figure suggests that one would need to screen nearly the entire population. The authors state as much, highlighting for the validation sample, “the top 3 risk groups represented 62.5%… of all patients diagnosed with HIV infection” (1, p 843), meaning that 37.5% of those infected would not be identified unless one also tested the lower-risk groups. The risk-score model appears to offer a very modest advantage over general HIV testing in the settings examined at the cost of obtaining extensive risk data to construct the score. This is aligned with a number of previous studies of targeted screening in clinical populations (3–5) and the CDC guidelines (2). We have additional concerns about the populations and methods used. First, in selecting a model derivation population of individuals reporting to a sexually transmitted infection clinic for HIV testing and a validation population of individuals clinically and behaviorally indicated for HIV testing, the authors have partially conditioned their analytic data on risk factors included in the final model. This likely biases observed associations with HIV infection and may help to explain why some traditionally strong predictors of undiagnosed HIV infection (e.g., partner number, condom use) were not retained in the final model. Because populations at increased risk were used for the score development, it is also unclear how this model would perform for more general healthcare settings with patients who have a broader distribution of risks and for which the CDC guidelines were intended. Second, it is unclear how the authors ascertained whether HIV-positive individuals were previously undiagnosed. Substantial numbers of individuals who tested HIV-positive may have had a previous diagnosis, and this may be an important consideration because risk factors for failure to disclose a previous diagnosis may differ from risk factors for an undiagnosed HIV infection (6–8). Last, we question the validity of constructing a risk score based on the addition of model slope parameter estimates from a logistic regression model (Table 2). This method is typically reserved for linear regression, whereas in the context of a logistic regression model, these parameters contribute multiplicatively to risk (9). Risk-score approaches to guide screening decisions are appealing, but in this case we feel that the data source limits the generalizability of the score to general populations. Even if it were generalizable, the results of the receiver operating characteristics curve indicate that a screening program, with higher costs required to identify patients for screening, would still require screening nearly everyone in a population to find those all of those living with HIV.