Improved diagnostic accuracy of thallium-201 stress test using multiple observers and criteria derived from interobserver analysis of variance

Abstract

This study was performed to determine the interobserver variance for the interpretation of segmental thallium-201 image scores and the changes in such scores between the exercise and delayed images. Fifty patients, 40 with and 10 without significant coronary artery disease, underwent thallium imaging immediately after exercise and 2 hours later in the anterior and 50 ° left anterior oblique projections. The left ventricle was divided into six segments and graded by four independent observers on a scale of 0 (minimal activity) to +2 (normal activity). Interobserver variance was determined for each segment. For a change in thallium activity between exercise and delayed images, the interobserver variance was greatest for the apical and apical-inferior segments, intermediate for the inferior and septal segments and least for the anterolateral and posterior segments. When segmental thallium scores were interpreted by an individual observer using a standard method employing one set of arbitrary criteria (method 1), the thallium stress test had a sensitivity of 77 percent and a specificity of 75 percent for coronary artery disease. When the scores were interpreted by an individual observer using criteria derived from interobserver variance analysis (method 2), the sensitivity increased to 86 percent (probability [p] < 0.05); the specificity (78 percent) did not change significantly. When multiple observers arrived at a single diagnosis by consensus (method 3), the sensitivity was 75 percent and specificity 90 percent. However, when the multiple observers' scores were averaged and the averaged scores interpreted by a single set of criteria applied to all six ventricular segments (method 4), the sensitivity and specificity increased to 90 percent (p < 0.05 and not significant, respectively [method 4 versus method 1]). Thus, there is considerable interobserver variance in the qualitative scoring of segmental thallium activity. More caution must be observed in interpreting changes in activity from exercise to delayed images in the apical and apical-inferior segments than in the anterolateral and posterior segments. When only one observer is available, diagnostic accuracy is improved by using criteria based on interobserver variance analysis. However, when feasible, thallium stress tests should be interpreted by multiple observers because averaging of the multiple observers' scores maximizes both sensitivity and specificity.