SU‐E‐I‐46: Sample‐Size Dependence of Model Observers for Estimating Low‐Contrast Detection Performance From CT Images

Abstract

Purpose:Recently, task‐based assessment of diagnostic CT systems has attracted much attention. Detection task performance can be estimated using human observers, or mathematical observer models. While most models are well established, considerable bias can be introduced when performance is estimated from a limited number of image samples. Thus, the purpose of this work was to assess the effect of sample size on bias and uncertainty of two channelized Hotelling observers and a template‐matching observer.Methods:The image data used for this study consisted of 100 signal‐present and 100 signal‐absent regions‐of‐interest, which were extracted from CT slices. The experimental conditions included two signal sizes and five different x‐ray beam current settings (mAs). Human observer performance for these images was determined in 2‐alternative forced choice experiments. These data were provided by the Mayo clinic in Rochester, MN. Detection performance was estimated from three observer models, including channelized Hotelling observers (CHO) with Gabor or Laguerre‐Gauss (LG) channels, and a template‐matching observer (TM). Different sample sizes were generated by randomly selecting a subset of image pairs, (N=20,40,60,80). Observer performance was quantified as proportion of correct responses (PC). Bias was quantified as the relative difference of PC for 20 and 80 image pairs.Results:For n=100, all observer models predicted human performance across mAs and signal sizes. Bias was 23% for CHO (Gabor), 7% for CHO (LG), and 3% for TM. The relative standard deviation, σ(PC)/PC at N=20 was highest for the TM observer (11%) and lowest for the CHO (Gabor) observer (5%).Conclusion:In order to make image quality assessment feasible in the clinical practice, a statistically efficient observer model, that can predict performance from few samples, is needed. Our results identified two observer models that may be suited for this task.