Computer-assisted detection of pulmonary embolism: performance evaluation in consensus with experienced and inexperienced chest radiologists

Abstract

The value of a computer-aided detection tool (CAD) as second reader in combination with experienced and inexperienced radiologists for the diagnosis of acute pulmonary embolism (PE) was assessed prospectively. Computed tomographic angiography (CTA) scans (64 x 0.6 mm collimation; 61.4 mm/rot table feed) of 56 patients (31 women, 34-89 years, mean = 66 years) with suspected PE were analysed by two experienced (R1, R2) and two inexperienced (R3, R4) radiologists for the presence and distribution of emboli using a five-point confidence rating, and by CAD. Informed consent was obtained from all patients. Results were compared with an independent reference standard. Inter-observer agreement was calculated by kappa, confidence assessed by ROC analysis. A total of 1,116 emboli [within mediastinal (n = 72), lobar (n = 133), segmental (n = 465) and subsegmental arteries (n = 455)] were included. CAD detected 343 emboli (sensitivity = 30.74%, correct-positive rate = 6.13/patient; false-positive rate = 4.1/patient). Inter-observer agreement was good (R1, R2: kappa = 0.84, 95% CI = 0.81-0.87; R3, R4: kappa = 0.79, 95% CI = 0.76-0.81). Extended inter-observer agreement was higher in mediastinal and lobar than in segmental and subsegmental arteries (kappa = 0.84-0.86 and kappa = 0.51-0.58 for mediastinal/lobar and segmental/subsegmental arteries, respectively P < 0.05). Agreement between experienced and inexperienced readers was improved by CAD (kappa = 0.60-0.62 and kappa = 0.69-0.72 before and after CAD consensus, respectively P < 0.05). The experienced outperformed the inexperienced readers (Az = 0.95, 0.93, 0.89 and 0.86 for R1-4, respectively, P < 0.05). CAD significantly improved overall performances of readers 3 and 4 (Az = 0.86 for R3, R4 and Az = 0.89 for R3, R4 with CAD, P < 0.05), by enhancing sensitivities in segmental/subsegmental arteries. CAD improved experienced readers' sensitivities in segmental/subsegmental arteries (sens. = 0.93 and 0.90 for R1, R2 before and 0.97 and 0.94 for R1, R2 after CAD consensus, P < 0.05), without significant improvement of their overall performances (P > 0.05). Particularly inexperienced readers benefit from consensus with CAD data, greatly improving detection of segmental and subsegmental emboli. This system is advocated as a second reader.