Channelized Hotelling Observer Performance Research Articles

OA245] Channelized hotelling observer performance in acquired mammographic images of an anthropomorphic breast phantom

Purpose To study the feasibility of a quality control framework composed of an anthropomorphic breast phantom and a mathematical model observer (MO) for the objective evaluation of detectability of calcification-like signals in system-processed mammographic images. Methods A 3D-printed anthropomorphic breast phantom was imaged with two brands of digital mammography systems (Selenia, Hologic, USA & Amulet Innovality, Fujifilm, Japan) at four different dose levels. The phantom consisted of two transverse slabs, allowing for the insertion of a sheet with calcification-like signals. These signals were 0.25 mm diameter gold discs deposited on aluminum squares of 1 cm width. From the acquired processed and unprocessed images, signal-present and signal-absent regions of interest (ROIs) were extracted. The ROIs were evaluated with a channelized Hotelling observer (CHO) MO using four different formulations of difference-of-Gaussian (DoG) channel-sets, and by three human observers in two-alternative forced-choice experiments. We compared the human and CHO performance in detecting calcification-like discs in the processed and unprocessed ROIs. Detection performance was measured using the proportion of correct responses of the humans ( PC H ) and the CHO ( PC CHO ) . The correlation between PC H and PC CHO was analyzed using a mixed-effect regression model. Correlation results, including the goodness of fit (r 2 ) of PC H and PC CHO for all parameters investigated, were evaluated to determine whether CHO MO can be used to predict human observer performance. Results A strong correlation of the CHO detection performance with the performance of the humans was obtained, with r 2 = 0.93–0.96. However, the slight, but not significant, differences in intercept and slope suggest some system dependence (p > 0.05). The correlation was not considerably affected by the choice of the DoG channel-set. The image processing did not have a significant impact on the PC H or PC CHO results (p > 0.05). Conclusions The CHO with DoG channel-sets can predict human performance on the detection of calcification-like signals under the tested conditions. Therefore, there is potential for the framework to be used for the evaluation of detectability of calcification-like signals in system-processed mammographic images. However, additional systems need to be evaluated to investigate this system-dependency in the correlation between human and the MO, further.

Practical implementation of Channelized Hotelling Observers: Effect of ROI size.

Fundamental to the development and application of channelized Hotelling observer (CHO) models is the selection of the region of interest (ROI) to evaluate. For assessment of medical imaging systems, reducing the ROI size can be advantageous. Smaller ROIs enable a greater concentration of interrogable objects in a single phantom image, thereby providing more information from a set of images and reducing the overall image acquisition burden. Additionally, smaller ROIs may promote better assessment of clinical patient images as different patient anatomies present different ROI constraints. To this end, we investigated the minimum ROI size that does not compromise the performance of the CHO model. In this study, we evaluated both simulated images and phantom CT images to identify the minimum ROI size that resulted in an accurate figure of merit (FOM) of the CHO's performance. More specifically, the minimum ROI size was evaluated as a function of the following: number of channels, spatial frequency and number of rotations of the Gabor filters, size and contrast of the object, and magnitude of the image noise. Results demonstrate that a minimum ROI size exists below which the CHO's performance is grossly inaccurate. The minimum ROI size is shown to increase with number of channels and be dictated by truncation of lower frequency filters. We developed a model to estimate the minimum ROI size as a parameterized function of the number of orientations and spatial frequencies of the Gabor filters, providing a guide for investigators to appropriately select parameters for model observer studies.

Exact confidence intervals for channelized Hotelling observer performance in image quality studies.

Task-based assessments of image quality constitute a rigorous, principled approach to the evaluation of imaging system performance. To conduct such assessments, it has been recognized that mathematical model observers are very useful, particularly for purposes of imaging system development and optimization. One type of model observer that has been widely applied in the medical imaging community is the channelized Hotelling observer (CHO), which is well-suited to known-location discrimination tasks. In the present work, we address the need for reliable confidence interval estimators of CHO performance. Specifically, we show that the bias associated with point estimates of CHO performance can be overcome by using confidence intervals proposed by Reiser for the Mahalanobis distance. In addition, we find that these intervals are well-defined with theoretically-exact coverage probabilities, which is a new result not proved by Reiser. The confidence intervals are tested with Monte Carlo simulation and demonstrated with two examples comparing X-ray CT reconstruction strategies. Moreover, commonly-used training/testing approaches are discussed and compared to the exact confidence intervals. MATLAB software implementing the estimators described in this work is publicly available at http://code.google.com/p/iqmodelo/.

New Theoretical Results on Channelized Hotelling Observer Performance Estimation With Known Difference of Class Means

Task-based assessments of image quality constitute a rigorous, principled approach to the evaluation of imaging system performance. To conduct such assessments, it has been recognized that mathematical model observers are very useful, particularly for purposes of imaging system development and optimization. One type of model observer that has been widely applied in the medical imaging community is the channelized Hotelling observer (CHO). Since estimates of CHO performance typically include statistical variability, it is important to control and limit this variability to maximize the statistical power of image-quality studies. In a previous paper, we demonstrated that by including prior knowledge of the image class means, a large decrease in the bias and variance of CHO performance estimates can be realized. The purpose of the present work is to present refinements and extensions of the estimation theory given in our previous paper, which was limited to point estimation with equal numbers of images from each class. Specifically, we present and characterize minimum-variance unbiased point estimators for observer signal-to-noise ratio (SNR) that allow for unequal numbers of lesion-absent and lesion-present images. Building on this SNR point estimation theory, we then show that confidence intervals with exactly-known coverage probabilities can be constructed for commonly-used CHO performance measures. Moreover, we propose simple, approximate confidence intervals for CHO performance, and we show that they are well-behaved in most scenarios of interest.

Estimation of Channelized Hotelling Observer Performance With Known Class Means or Known Difference of Class Means

This paper concerns task-based image quality assessment for the task of discriminating between two classes of images. We address the problem of estimating two widely-used detection performance measures, SNR and AUC, from a finite number of images, assuming that the class discrimination is performed with a channelized Hotelling observer. In particular, we investigate the advantage that can be gained when either 1) the means of the signal-absent and signal-present classes are both known, or 2) when the difference of class means is known. For these two scenarios, we propose uniformly minimum variance unbiased estimators of SNR(2), derive the corresponding sampling distributions and provide variance expressions. In addition, we demonstrate how the bias and variance for the related AUC estimators may be calculated numerically by using the sampling distributions for the SNR(2) estimators. We find that for both SNR(2) and AUC, the new estimators have significantly lower bias and mean-square error than the traditional estimator, which assumes that the class means, and their difference, are unknown.