SASSI: A RESPONSE TO LAZOWSKI & MILLER (2007)

Abstract

In response to our review of research on the reliability and validity of the Substance Abuse Subtle Screening Inventory (SASSI) [1], Lazowski & Miller [2], of the SASSI Institute, make several claims. The first is that we used ‘incorrect methods to calculate accuracy’. This is not so. We specified our calculation methods clearly, which are the traditional psychometric procedures for computing sensitivity (the percentage of true positives identified as positive) and specificity (the percentage of true negatives identified as negative). The alleged ‘inaccuracy’ pertains to one parenthetical point from page 46 of our review: how to weight evidence from different studies. The sensitivity values reported in eight independent studies were 0.33, 0.52, 0.59, 0.60, 0.65, 0.72, 0.85 and 0.87. We reported an average of 69.8% when weighting studies by N, essentially pooling all tested cases. The simple arithmetic average, which gives equal weight to smaller and larger studies, is less favorable still: 64.1%, and the median is 62.5%. The SASSI authors favor weighting studies by prevalence rate and arrived at a calculated average sensitivity of 84%, a figure that we were unable to reproduce. It is true that of the 36 studies we reviewed, only 10 included a criterion measure against which SASSI accuracy could be judged, and that only three of these used the SASSI-3. This is because, to the best of our knowledge, that is the entire peer-reviewed literature on the validity of the SASSI. The fact remains that no peer-reviewed study has replicated the sensitivity rate of 0.94 claimed in the test manual and the average, however computed, is well below that figure. The SASSI authors also allege ‘unsubstantiated claims of bias’ by stating that a screener may validly report a higher positive rate for an ethnic group if the true prevalence rate is also higher for that group. True prevalence rates for ethnic subgroups were not reported in the studies we cited, but US population studies generally show aggregate prevalence rates for substance use disorders in Hispanic and African American groups to be similar to or lower than those for the US majority population [3]. Furthermore, other screening instruments such as the Alcohol Use Disorders Identification Test (AUDIT) do not overclassify ethnic minorities. We believe that there remains reason for concern from the four studies pointing to racial/ethnic differences in SASSI classifications [4,5] and scale scores [6,7]. Lazowski & Miller did identify one typographical error in our review. The sensitivity of the SASSI direct scales as reported in a footnote in their test manual was indeed 74%, and not 79%. We apologize for this error, noting that the method used to compute sensitivity in their table is the same one used in our review. The central claim of the SASSI authors' letter, and indeed of each version of the SASSI test manual [8,9], is that ‘the subtle scales improve detection’. In defense of this claim, they cite statistics from their own test manual. Although the indirect scales do increase the proportion classified as positive by the SASSI, we stand by our conclusion that independent studies do not support this claim for any version of the SASSI. Six studies in our review found that the direct scales performed on a par with or better than the SASSI when its indirect scales were included [7,10–14]. This is consistent with the larger research literature across five decades, indicating that allegedly indirect scales are no more accurate than direct self-report scales [15]. Their final claimed advantage, that the SASSI goes ‘beyond mere screening’, is precisely a reason for concern. Although sold as a screening instrument, the SASSI is sometimes used (inappropriately in our view) to establish a diagnosis and make treatment decisions. Indeed, the SASSI manuals provide guidelines for using the instrument to determine severity of substance dependence, need for supervised detoxification, risk for criminality and suggested treatment approach and directions. We found no independent research evidence that the SASSI provides additional valid information unavailable from direct scales or client self-report. Properly used, a valid screening instrument signals the need for more careful evaluation and clinical confirmation of diagnosis, and does not itself serve or replace these functions.