Human Observer Performance Research Articles

Localization of liver lesions in abdominal CT imaging: I. Correlation of human observer performance between anatomical and uniform backgrounds**Selected sections presented during oral presentation at 2017 RSNA Annual Meeting.

The purpose of this study was to determine the correlation between human observer performance for localization of small low contrast lesions within uniform water background versus an anatomical liver background, under the conditions of varying dose, lesion size, and reconstruction algorithm.Liver lesions (5 mm, 7 mm, and 9 mm, contrast: −21 HU) were digitally inserted into CT projection data of ten normal patients in vessel-free liver regions. Noise was inserted into the projection data to create three image sets: full dose and simulated half and quarter doses. Images were reconstructed with a standard filtered back projection (FBP) and an iterative reconstruction (IR) algorithm. Lesion and noise insertion procedures were repeated with water phantom data. Two-dimensional regions of interest (66 lesion-present, 34 lesion-absent) were selected, randomized, and independently reviewed by three medical physicists to identify the most likely location of the lesion and provide a confidence score. Locations and confidence scores were assessed using the area under the localization receiver operating characteristic curve (AzLROC). We examined the correlation between human performance for the liver and uniform water backgrounds as dose, lesion size, and reconstruction algorithm varied.As lesion size or dose increased, reader localization performance improved. For full dose IR images, the AzLROC for 5, 7, and 9 mm lesions were 0.53, 0.91, and 0.97 (liver) and 0.51, 0.96, and 0.99 (uniform water), respectively. Similar trends were seen with other parameters. Performance values for liver and uniform backgrounds were highly correlated for both reconstruction algorithms, with a Spearman correlation of ρ = 0.97, and an average difference in AzLROC of 0.05 ± 0.04.For the task of localizing low contrast liver lesions, human observer performance was highly correlated between anatomical and uniform backgrounds, suggesting that lesion localization studies emulating a clinical test of liver lesion detection can be performed using a uniform background.

Localization of liver lesions in abdominal CT imaging: II. Mathematical model observer performance correlates with human observer performance for localization of liver lesions in abdominal CT imaging**Selected sections presented during digital poster presentation at 2017 RSNA Annual Meeting.

Determination of the effect of protocol modifications on diagnostic performance in CT with human observers is extremely time-consuming, limiting the applicability of such methods in routine clinical practice. In this work, we sought to determine whether a channelized Hotelling observer (CHO) could predict human observer performance for the task of liver lesion localization as background, reconstruction algorithm, dose, and lesion size were varied.Liver lesions (5 mm, 7 mm, and 9 mm) were digitally inserted into the CT projection data of patients with normal livers and water phantoms. The projection data were reconstructed with filtered back projection (FBP) and iterative reconstruction (IR) algorithms for three dose levels: full dose (liver CTDIvol = 10.5 ± 8.5 mGy, water phantom CTDIvol = 9.6 ± 0.1 mGy) and simulated half and quarter doses. For each of 36 datasets (3 dose levels × 2 reconstruction algorithms × 2 backgrounds × 3 sizes), 66 signal-present and 34 signal-absent 2D images were extracted from the reconstructed volumes. Three medical physicists independently reviewed each dataset and noted the lesion location and a confidence score for each image. A CHO with Gabor channels was calculated to estimate the performance for each of the 36 localization tasks. The CHO performances, quantified using localization receiver operating characteristic (LROC) analysis, were compared to the human observer performances.Performance values between human and model observers were highly correlated for equivalent parameters (same lesion size, dose, background, and reconstruction), with a Spearman’s correlation coefficient of 0.93 (95% CI: 0.82–0.98). CHO performance values for the uniform background were strongly correlated (ρ = 0.94, CI: 0.80–1.0) with the human observer performance values for the liver background.Performance values between human observers and CHO were highly correlated as dose, reconstruction type and object size were varied for the task of localization of patient liver lesions in both uniform and liver backgrounds.

Confidence boosts serial dependence in orientation estimation.

In the absence of external feedback, a decision maker must rely on a subjective estimate of their decision accuracy in order to appropriately guide behavior. Normative models of perceptual decision-making relate subjective estimates of internal signal quality (e.g., confidence) directly to the internal signal quality itself, thereby making it unknowable whether the subjective estimate or the underlying signal is what drives behavior. We constructed stimuli that dissociated the human observer's performance on a visual estimation task from their subjective estimates of confidence in their performance, thus violating normative principles. To understand whether confidence influences future decision-making, we examined serial dependence in observer's responses, a phenomenon whereby the estimate of a stimulus on the current trial can be biased toward the stimulus from the previous trial. We found that when decisions were made with high confidence, they conferred stronger biases upon the following trial, suggesting that confidence may enhance serial dependence. Critically, this finding was true also when confidence was experimentally dissociated from task performance, indicating that subjective confidence, independent of signal quality, can amplify serial dependence. These findings demonstrate an effect of confidence on future behavior, independent of task performance, and suggest that perceptual decisions incorporate recent history in an uncertainty-weighted manner, but where the uncertainty carried forward is a subjectively estimated and possibly suboptimal readout of objective sensory uncertainty.

A deep learning- and partial least square regression-based model observer for a low-contrast lesion detection task in CT.

This work aims to develop a new framework of image quality assessment using deep learning-based model observer (DL-MO) and to validate it in a low-contrast lesion detection task that involves CT images with patient anatomical background. The DL-MO was developed using the transfer learning strategy to incorporate a pretrained deep convolutional neural network (CNN), a partial least square regression discriminant analysis (PLS-DA) model and an internal noise component. The CNN was previously trained to achieve the state-of-the-art classification accuracy over a natural image database. The earlier layers of the CNN were used as a deep feature extractor, with the assumption that similarity exists between the CNN and the human visual system. The PLSR model was used to further engineer the deep feature for the lesion detection task in CT images. The internal noise component was incorporated to model the inefficiency and variability of human observer (HO) performance, and to generate the ultimate DL-MO test statistics. Seven abdominal CT exams were retrospectively collected from the same type of CT scanners. To compare DL-MO with HO, 12 experimental conditions with varying lesion size, lesion contrast, radiation dose, and reconstruction types were generated, each condition with 154 trials. CT images of a real liver metastatic lesion were numerically modified to generate lesion models with four lesion sizes (5, 7, 9, and 11mm) and three contrast levels (15, 20, and 25HU). The lesions were inserted into patient liver images using a projection-based method. A validated noise insertion tool was used to synthesize CT exams with 50% and 25% of routine radiation dose level. CT images were reconstructed using the weighted filtered back projection algorithm and an iterative reconstruction algorithm. Four medical physicists performed a two-alternative forced choice (2AFC) detection task (with multislice scrolling viewing mode) on patient images across the 12 experimental conditions. DL-MO was operated on the same datasets. Statistical analyses were performed to evaluate the correlation and agreement between DL-MO and HO. A statistically significant positive correlation was observed between DL-MO and HO for the 2AFC low-contrast detection task that involves patient liver background. The corresponding Pearson product moment correlation coefficient was 0.986 [95% confidence interval (0.950, 0.996)]. Bland-Altman agreement analysis did not indicate statistically significant differences. The proposed DL-MO is highly correlated with HO in a low-contrast detection task that involves realistic patient liver background. This study demonstrated the potential of the proposed DL-MO to assess image quality directly based on patient images in realistic, clinically relevant CT tasks.

Can a channelized Hotelling observer assess image quality in acquired mammographic images of an anthropomorphic breast phantom including image processing?

To study the feasibility of a channelized Hotelling observer (CHO) to predict human observer performance in detecting calcification-like signals in mammography images of an anthropomorphic breast phantom, as part of a quality control (QC) framework. A prototype anthropomorphic breast phantom with inserted gold disks of 0.25 mm diameter was imaged with two different digital mammography x-ray systems at four different dose levels. Regions of interest (ROIs) were extracted from the acquired processed and unprocessed images, signal-present and signal-absent. The ROIs were evaluated by a CHO using four different formulations of the difference of Gaussian (DoG) channel sets. Three human observers scored the ROIs in a two-alternative forced-choice experiment. We compared the human and the CHO performance on the simple task to detect calcification-like disks in ROIs with and without postprocessing. The proportion of correct responses of the human reader (PCH ) and the CHO (PCCHO ) was calculated and the correlation between the two was analyzed using a mixed-effect regression model. To address the signal location uncertainty, the impact of shifting the DoG channel sets in all directions up to two pixels was evaluated. Correlation results including the goodness of fit (r2 ) of PCH and PCCHO for all different parameters were evaluated. Subanalysis by system yielded strong correlations between PCH and PCCHO , with r2 between PCH and PCCHO was found to be between 0.926 and 0.958 for the unshifted and between 0.759 and 0.938 for the shifted channel sets, respectively. However, the linear fit suggested a slight system dependence. PCCHO with shifted channel sets increased CHO performance but the correlation with humans was decreased. These correlations were not considerably affected by of the DoG channel set used. There is potential for the CHO to be used in QC for the evaluation of detectability of calcification-like signals. The CHO can predict the PC of humans in images of calcification-like signals of two different systems. However, a global model to be used for all systems requires further investigation.

CNN as model observer in a liver lesion detection task for x-ray computed tomography: A phantom study.

The purpose of this study was the evaluation of anthropomorphic model observers trained with neural networks for the prediction of a human observer's performance. To simulate liver lesions, a phantom with contrast targets (acrylic spheres, varying diameters, +30HU) was repeatedly scanned on a computed tomography scanner. Image data labeled with confidence ratings assessed in a reader study for a detection task of liver lesions were used to build several anthropomorphic model observers. Models were trained with images reconstructed with iterative reconstruction and evaluated with images reconstructed with filtered backprojection. A neural network, based on softmax regression (SR-MO), and convolutional neural networks (CNN-MO) were used to predict the performance of a human observer and compared to a channelized Hotelling observer [with Gabor channels and internal channel noise (CHOi)]. Model observers were evaluated by a receiver operating characteristic curve analysis and compared to the results in the reader study. Two strategies were used to train the SR-MO and CNN-MO: A) building a separate model for each lesion size; B) building one model that was applied to lesions of all sizes. All tested model observers and the human observer were highly correlated at each lesion size and dose level. With strategy A, Pearson's product-moment correlation coefficients r were 0.926 (95% confidence interval (CI): 0.679-0.985) for SR-MO and 0.979 (95% CI: 0.902-0.996) for CNN-MO. With strategy B, r was 0.860 (95% CI: 0.454-0.970) for SR-MO and 0.918 (95% CI: 0.651-0.983) for CNN-MO. For CHOi, r was 0.945 (95% CI: 0.755-0.989). With strategy A, mean absolute percentage differences (MAPD) between the model observers and the human observer were 3.7% for SR-MO and 1.2% for CNN-MO. With strategy B, MAPD were 3.7% for SR-MO and 3.0% for CNN-MO. For the CHOi the MAPD was 2.2%. Convolutional neural network model observers can accurately predict the performance of a human observer for all lesion sizes and dose levels in the evaluated signal detection task.

Multivariate Approach for Alzheimer's Disease Detection Using Stationary Wavelet Entropy and Predator-Prey Particle Swarm Optimization.

The number of patients with Alzheimer's disease is increasing rapidly every year. Scholars often use computer vision and machine learning methods to develop an automatic diagnosis system. In this study, we developed a novel machine learning system that can make diagnoses automatically from brain magnetic resonance images. First, the brain imaging was processed, including skull stripping and spatial normalization. Second, one axial slice was selected from the volumetric image, and stationary wavelet entropy (SWE) was done to extract the texture features. Third, a single-hidden-layer neural network was used as the classifier. Finally, a predator-prey particle swarm optimization was proposed to train the weights and biases of the classifier. Our method used 4-level decomposition and yielded 13 SWE features. The classification yielded an overall accuracy of 92.73±1.03%, a sensitivity of 92.69±1.29%, and a specificity of 92.78±1.51%. The area under the curve is 0.95±0.02. Additionally, this method only cost 0.88 s to identify a subject in online stage, after its volumetric image is preprocessed. In terms of classification performance, our method performs better than 10 state-of-the-art approaches and the performance of human observers. Therefore, this proposed method is effective in the detection of Alzheimer's disease.

OA245] Channelized hotelling observer performance in acquired mammographic images of an anthropomorphic breast phantom

Purpose To study the feasibility of a quality control framework composed of an anthropomorphic breast phantom and a mathematical model observer (MO) for the objective evaluation of detectability of calcification-like signals in system-processed mammographic images. Methods A 3D-printed anthropomorphic breast phantom was imaged with two brands of digital mammography systems (Selenia, Hologic, USA & Amulet Innovality, Fujifilm, Japan) at four different dose levels. The phantom consisted of two transverse slabs, allowing for the insertion of a sheet with calcification-like signals. These signals were 0.25 mm diameter gold discs deposited on aluminum squares of 1 cm width. From the acquired processed and unprocessed images, signal-present and signal-absent regions of interest (ROIs) were extracted. The ROIs were evaluated with a channelized Hotelling observer (CHO) MO using four different formulations of difference-of-Gaussian (DoG) channel-sets, and by three human observers in two-alternative forced-choice experiments. We compared the human and CHO performance in detecting calcification-like discs in the processed and unprocessed ROIs. Detection performance was measured using the proportion of correct responses of the humans ( PC H ) and the CHO ( PC CHO ) . The correlation between PC H and PC CHO was analyzed using a mixed-effect regression model. Correlation results, including the goodness of fit (r 2 ) of PC H and PC CHO for all parameters investigated, were evaluated to determine whether CHO MO can be used to predict human observer performance. Results A strong correlation of the CHO detection performance with the performance of the humans was obtained, with r 2 = 0.93–0.96. However, the slight, but not significant, differences in intercept and slope suggest some system dependence (p > 0.05). The correlation was not considerably affected by the choice of the DoG channel-set. The image processing did not have a significant impact on the PC H or PC CHO results (p > 0.05). Conclusions The CHO with DoG channel-sets can predict human performance on the detection of calcification-like signals under the tested conditions. Therefore, there is potential for the framework to be used for the evaluation of detectability of calcification-like signals in system-processed mammographic images. However, additional systems need to be evaluated to investigate this system-dependency in the correlation between human and the MO, further.

Analyzing thresholds and efficiency with hierarchical Bayesian logistic regression

Ideal observer analysis is a fundamental tool used widely in vision science for analyzing the efficiency with which a cognitive or perceptual system uses available information. The performance of an ideal observer provides a formal measure of the amount of information in a given experiment. The ratio of human to ideal performance is then used to compute efficiency, a construct that can be directly compared across experimental conditions while controlling for the differences due to the stimuli and/or task specific demands. In previous research using ideal observer analysis, the effects of varying experimental conditions on efficiency have been tested using ANOVAs and pairwise comparisons. In this work, we present a model that combines Bayesian estimates of psychometric functions with hierarchical logistic regression for inference about both unadjusted human performance metrics and efficiencies. Our approach improves upon the existing methods by constraining the statistical analysis using a standard model connecting stimulus intensity to human observer accuracy and by accounting for variability in the estimates of human and ideal observer performance scores. This allows for both individual and group level inferences.

Classification images for localization performance in ramp-spectrum noise.

This study investigates forced localization of targets in simulated images with statistical properties similar to trans-axial sections of x-ray computed tomography (CT) volumes. A total of 24 imaging conditions are considered, comprising two target sizes, three levels of background variability, and four levels of frequency apodization. The goal of the study is to better understand how human observers perform forced-localization tasks in images with CT-like statistical properties. The transfer properties of CT systems are modeled by a shift-invariant transfer function in addition to apodization filters that modulate high spatial frequencies. The images contain noise that is the combination of a ramp-spectrum component, simulating the effect of acquisition noise in CT, and a power-law component, simulating the effect of normal anatomy in the background, which are modulated by the apodization filter as well. Observer performance is characterized using two psychophysical techniques: efficiency analysis and classification image analysis. Observer efficiency quantifies how much diagnostic information is being used by observers to perform a task, and classification images show how that information is being accessed in the form of a perceptual filter. Psychophysical studies from five subjects form the basis of the results. Observer efficiency ranges from 29% to 77% across the different conditions. The lowest efficiency is observed in conditions with uniform backgrounds, where significant effects of apodization are found. The classification images, estimated using smoothing windows, suggest that human observers use center-surround filters to perform the task, and these are subjected to a number of subsequent analyses. When implemented as a scanning linear filter, the classification images appear to capture most of the observer variability in efficiency (r2 = 0.86). The frequency spectra of the classification images show that frequency weights generally appear bandpass in nature, with peak frequency and bandwidth that vary with statistical properties of the images. In these experiments, the classification images appear to capture important features of human-observer performance. Frequency apodization only appears to have a significant effect on performance in the absence of anatomical variability, where the observers appear to underweight low spatial frequencies that have relatively little noise. Frequency weights derived from the classification images generally have a bandpass structure, with adaptation to different conditions seen in the peak frequency and bandwidth. The classification image spectra show relatively modest changes in response to different levels of apodization, with some evidence that observers are attempting to rebalance the apodized spectrum presented to them.

Performance of human observers and an automatic 3-dimensional computer-vision-based locomotion scoring method to detect lameness and hoof lesions in dairy cows

The objective of this study was to determine if a 3-dimensional computer vision automatic locomotion scoring (3D-ALS) method was able to outperform human observers for classifying cows as lame or nonlame and for detecting cows affected and nonaffected by specific type(s) of hoof lesion. Data collection was carried out in 2 experimental sessions (5 mo apart). In every session all cows were assessed for (1) locomotion by 2 observers (Obs1 and Obs2) and by a 3D-ALS; and (2) identification of different types of hoof lesions during hoof trimming (i.e., skin and horn lesions and combinations of skin/horn lesions and skin/hyperplasia). Performances of observers and 3D-ALS for classifying cows as lame or nonlame and for detecting cows affected or nonaffected by types of lesion were estimated using the percentage of agreement (PA), kappa coefficient (κ), sensitivity (SEN), and specificity (SPE). Observers and 3D-ALS showed similar SENlame values for classifying lame cows as lame (SENlame comparison Obs1-Obs2 = 74.2%; comparison observers-3D-ALS = 73.9-71.8%). Specificity values for classifying nonlame cows as nonlame were lower for 3D-ALS when compared with observers (SPEnonlame comparison Obs1-Obs2 = 88.5%; comparison observers-3D-ALS = 65.3-67.8%). Accordingly, overall performance of 3D-ALS for classifying cows as lame and nonlame was lower than observers (Obs1-Obs2 comparison PAlame/nonlame = 84.2% and κlame/nonlame = 0.63; observers-3D-ALS comparisons PAlame/nonlame = 67.7-69.2% and κlame/nonlame = 0.33-0.36). Similarly, observers and 3D-ALS had comparable and moderate SENlesion values for detecting horn (SENlesion Obs1 = 68.6%; Obs2 = 71.4%; 3D-ALS = 75.0%) and combinations of skin/horn lesions (SENlesion Obs1 = 51.1%; Obs2 = 64.5%; 3D-ALS = 53.3%). The SPEnonlesion values for detecting cows without lesions when classified as nonlame were lower for 3D-ALS than for observers (SPEnonlesion Obs1 = 83.9%; Obs2 = 80.2%; 3D-ALS = 60.2%). This was translated into a poor overall performance of 3D-ALS for detecting cows affected and nonaffected by horn lesions (PAlesion/nonlesion Obs1 = 80.6%; Obs2 = 78.3%; 3D-ALS = 63.5% and κlesion/nonlesion Obs1 = 0.48; Obs2 = 0.44; 3D-ALS = 0.25) and skin/horn lesions (PAlesion/nonlesion Obs1 = 75.1%; Obs2 = 75.9%; 3D-ALS = 58.6% and κlesion/nonlesion Obs1 = 0.35; Obs2 = 0.42; 3D-ALS = 0.10), when compared with observers. Performance of observers and 3D-ALS for detecting skin lesions was poor (SENlesion for Obs1, Obs2, and 3D-ALS <40%). Comparable SENlame and SENlesion values for observers and 3D-ALS are explained by an overestimation of lameness by 3D-ALS when compared with observers. Thus, comparable SENlame and SENlesion were reached at the expense high number of false positives and low SPEnonlame and SPEnonlesion. Considering that observers and 3D-ALS showed similar performance for classifying cows as lame and for detecting horn and combinations of skin/horn lesions, the 3D-ALS could be a useful tool for supporting dairy farmers in their hoof health management.

Assessing CT acquisition parameters with visual-search model observers.

A principal difference between the channelized Hotelling (CH) and visual-search (VS) model observers is how they respond to noise texture in images. We compared the two observers in lesion-detection studies to evaluate linear and angular sampling parameters for CT. Simulated lung images were generated from a single two-dimensional mathematical torso phantom containing circular lesions of fixed radius and relative contrast. Projection datasets were produced for two detector pixel sizes and from 15 to 128 projections at 15 and 65M counts per set. Filtered backprojection reconstructions were obtained with dimensions of [Formula: see text] and [Formula: see text]. A localization receiver operating characteristic study was conducted with two human observers, three single-feature VS observers, and a feature-adaptive VS observer. The effects of the sampling parameters on performance were similar for all of these observers. The CH observer, applied in location-known studies with and without background variability, was not affected by the variations in angular sampling. The two-stage VS framework was an effective modification of the CH observer for assessing the effects of noise texture on human-observer performance in this study.

Human and model observer performance for lesion detection in breast cone beam CT images with the FDK reconstruction.

We investigate the detectability of breast cone beam computed tomography images using human and model observers and the variations of exponent, β, of the inverse power-law spectrum for various reconstruction filters and interpolation methods in the Feldkamp-Davis-Kress (FDK) reconstruction. Using computer simulation, a breast volume with a 50% volume glandular fraction and a 2mm diameter lesion are generated and projection data are acquired. In the FDK reconstruction, projection data are apodized using one of three reconstruction filters; Hanning, Shepp-Logan, or Ram-Lak, and back-projection is performed with and without Fourier interpolation. We conduct signal-known-exactly and background-known-statistically detection tasks. Detectability is evaluated by human observers and their performance is compared with anthropomorphic model observers (a non-prewhitening observer with eye filter (NPWE) and a channelized Hotelling observer with either Gabor channels or dense difference-of-Gaussian channels). Our results show that the NPWE observer with a peak frequency of 7cyc/degree attains the best correlation with human observers for the various reconstruction filters and interpolation methods. We also discover that breast images with smaller β do not yield higher detectability in the presence of quantum noise.

Determination of Optimal Image Type and Lowest Detectable Concentration for Iodine Detection on a Photon Counting Detector-Based Multi-Energy CT System.

Photon counting detector (PCD) based multi-energy CT is able to generate different types of images such as virtual monoenergetic images (VMIs) and material specific images (e.g., iodine maps) in addition to the conventional single energy images. The purpose of this study is to determine the image type that has optimal iodine detection and to determine the lowest detectable iodine concentration using a PCD-CT system. A 35 cm body phantom with iodine inserts of 4 concentrations and 2 sizes was scanned on a research PCD-CT system. For each iodine concentration, 80 repeated scans were performed and images were reconstructed for each energy threshold. In addition, VMIs at different keVs and iodine maps were also generated. CNR was measured for each type of images. A channelized Hotelling observer was used to assess iodine detectability after being validated with human observer studies, with area under the ROC curve (AUC) as a figure of merit. The agreement between model and human observer performance indicated that model observer could serve as an effective approach to determine optimal image type for the clinical practice and to determine the lowest detectable iodine concentration. Results demonstrated that for all size and concentration combinations, VMI at 70 keV had similar performance as that of threshold low images, both of which outperformed the iodine map images. At the AUC value of 0.8, iodine concentration as low as 0.2 mgI/cc could be detected for an 8 mm object and 0.5 mgI/cc for a 4 mm object with a 5 mm slice thickness.

Correlation between model observers in uniform background and human observers in patient liver background for a low-contrast detection task in CT.

Channelized Hotelling observer (CHO) has demonstrated strong correlation with human observer (HO) in both single-slice viewing mode and multi-slice viewing mode in low-contrast detection tasks with uniform background. However, it remains unknown if the simplest single-slice CHO in uniform background can be used to predict human observer performance in more realistic tasks that involve patient anatomical background and multi-slice viewing mode. In this study, we aim to investigate the correlation between CHO in a uniform water background and human observer performance at a multi-slice viewing mode on patient liver background for a low-contrast lesion detection task. The human observer study was performed on CT images from 7 abdominal CT exams. A noise insertion tool was employed to synthesize CT scans at two additional dose levels. A validated lesion insertion tool was used to numerically insert metastatic liver lesions of various sizes and contrasts into both phantom and patient images. We selected 12 conditions out of 72 possible experimental conditions to evaluate the correlation at various radiation doses, lesion sizes, lesion contrasts and reconstruction algorithms. CHO with both single and multi-slice viewing modes were strongly correlated with HO. The corresponding Pearson's correlation coefficient was 0.982 (with 95% confidence interval (CI) [0.936, 0.995]) and 0.989 (with 95% CI of [0.960, 0.997]) in multi-slice and single-slice viewing modes, respectively. Therefore, this study demonstrated the potential to use the simplest single-slice CHO to assess image quality for more realistic clinically relevant CT detection tasks.

Toward image quality assessment in mammography using model observers: Detection of a calcification-like object.

Model observers (MOs) are of interest in the field of medical imaging to assess image quality. However, before procedures using MOs can be proposed in quality control guidelines for mammography systems, we need to know whether MOs are sensitive to changes in image quality and correlations in background structure. Therefore, as a proof of principle, in this study human and model observer (MO) performance are compared for the detection of calcification-like objects using different background structures and image quality levels of unprocessed mammography images. Three different phantoms, homogeneous polymethyl methacrylate, BR3D slabs with swirled patterns (CIRS, Norfolk, VA, USA), and a prototype anthropomorphic breast phantom (Institute of Medical Physics and Radiation Protection, Technische Hochschule Mittelhessen, Germany) were imaged on an Amulet Innovality (FujiFilm, Tokyo, Japan) mammographic X-ray unit. Because the complexities of the structures of these three phantoms were different and not optimized to match the characteristics of real mammographic images, image processing was not applied in this study. In addition, real mammograms were acquired on the same system. Regions of interest (ROIs) were extracted from each image. In half of the ROIs, a 0.25-mm diameter disk was inserted at four different contrast levels to represent a calcification-like object. Each ROI was then modified, so four image qualities relevant for mammography were simulated. The signal-present and signal-absent ROIs were evaluated by a non-pre-whitening model observer with eye filter (NPWE) and a channelized Hotelling observer (CHO) using dense difference of Gaussian channels. The ROIs were also evaluated by human observers in a two alternative forced choice experiment. Detectability results for the human and model observer experiments were correlated using a mixed-effect regression model. Threshold disk contrasts for human and predicted human observer performance based on the NPWE MO and CHO were estimated. Global trends in threshold contrast were similar for the different background structures, but absolute contrast threshold levels differed. Contrast thresholds tended to be lower in ROIs from simple phantoms compared with ROIs from real mammographic images. The correlation between human and model observer performance was not affected by the range of image quality levels studied. The correlation between human and model observer performance does not depend on image quality. This is a promising outcome for the use of model observers in image quality analysis and allows for subsequent research toward the development of MO-based quality control procedures and guidelines.

Method for quantitative assessment of retinal vessel tortuosity in optical coherence tomography angiography applied to sickle cell retinopathy.

Tortuosity is an important geometric vessel parameter and among the first microvascular alterations observed in various retinopathies. In the current study, a quantitative vessel tortuosity index (VTI) based on a combination of local and global centerline features is presented. Performance of VTI and previously established tortuosity indices were compared against human observers' evaluation of tortuosity. An image-processing pipeline was developed for application of VTI in retinal vessels imaged by optical coherence tomography angiography (OCTA) in perifoveal (6 mm × 6 mm) and parafoveal (3 mm × 3 mm) regions centered on the fovea. Forty-one subjects (12 healthy control (NC) and 29 sickle cell retinopathy (SCR)) and 10 subjects (5 NC and 5 SCR) were imaged in the perifoveal and parafoveal regions, respectively. The relationship between VTI and age was examined in the perifoveal regions in NC subjects. VTI was measured from the OCTA images and compared between NC and SCR subjects using generalized least square regression with and without adjusting for age and race. VTI was found to correlate better than the 4 previous indices with performance of human observers. In the perifoveal region, a significant correlation was observed between VTI and age (r = -0.4, P<0.001, N = 12). VTI was higher in SCR than NC subjects in perifoveal and parafoveal regions (P≤0.001). The results demonstrate that the proposed method shows promise for detection of increased tortuosity in vessels due to retinal disorders.

Modified ideal observer model (MIOM) for high-contrast and high-spatial resolution CT imaging tasks.

Although a variety of mathematical observer models have been developed to predict human observer performance for low contrast lesion detection tasks, their predictive power for high contrast and high spatial resolution discrimination imaging tasks, including those in CT bone imaging, could be limited. The purpose of this work was to develop a modified observer model that has improved correlation with human observer performance for these tasks. The proposed observer model, referred to as the modified ideal observer model (MIOM), uses a weight function to penalize components in the task function that have less contribution to the actual human observer performance for high contrast and high spatial resolution discrimination tasks. To validate MIOM, both human observer and observer model studies were performed, each using exactly the same CT imaging task [discrimination of a connected component in a high contrast (1000 HU) high spatial resolution bone fracture model (0.3 mm)] and experimental CT image data. For the human observer studies, three physicist observers rated the connectivity of the fracture model using a five-point Likert scale; for the observer model studies, a total of five observer models, including both conventional models and the proposed MIOM, were used to calculate the discrimination capability of the CT images in resolving the connected component. Images used in the studies encompassed nine different reconstruction kernels. Correlation between human and observer model performance for these kernels were quantified using the Spearman rank correlation coefficient (ρ). After the validation study, an example application of MIOM was presented, in which the observer model was used to select the optimal reconstruction kernel for a High-Resolution (Hi-Res, GE Healthcare) CT scan technique. The performance of the proposed MIOM correlated well with that of the human observers with a Spearman rank correlation coefficient ρ of 0.88 (P = 0.003). In comparison, the value of ρ was 0.05 (P = 0.904) for the ideal observer, 0.05 (P = 0.904) for the non-prewhitening observer, -0.18 (P = 0.634) for the non-prewhitening observer with eye filter and internal noise, and 0.30 (P = 0.427) for the prewhitening observer with eye filter and internal noise. Using the validated MIOM, the optimal reconstruction kernel for the Hi-Res mode to perform high spatial resolution and high contrast discrimination imaging tasks was determined to be the HD Ultra kernel at the center of the scan field of view (SFOV), or the Lung kernel at the peripheral region of the SFOV. This result was consistent with visual observations of nasal CT images of an in vivo canine subject. Compared with other observer models, the proposed modified ideal observer model provides significantly improved correlation with human observers for high contrast and high spatial resolution CT imaging tasks.

Correlation between a 2D channelized Hotelling observer and human observers in a low-contrast detection task with multislice reading in CT.

Model observers have been successfully developed and used to assess the quality of static 2D CT images. However, radiologists typically read images by paging through multiple 2D slices (i.e., multislice reading). The purpose of this study was to correlate human and model observer performance in a low-contrast detection task performed using both 2D and multislice reading, and to determine if the 2D model observer still correlate well with human observer performance in multislice reading. A phantom containing 18 low-contrast spheres (6 sizes × 3 contrast levels) was scanned on a 192-slice CT scanner at five dose levels (CTDIvol = 27, 13.5, 6.8, 3.4, and 1.7 mGy), each repeated 100 times. Images were reconstructed using both filtered-backprojection (FBP) and an iterative reconstruction (IR) method (ADMIRE, Siemens). A 3D volume of interest (VOI) around each sphere was extracted and placed side-by-side with a signal-absent VOI to create a 2-alternative forced choice (2AFC) trial. Sixteen 2AFC studies were generated, each with 100 trials, to evaluate the impact of radiation dose, lesion size and contrast, and reconstruction methods on object detection. In total, 1600 trials were presented to both model and human observers. Three medical physicists acted as human observers and were allowed to page through the 3D volumes to make a decision for each 2AFC trial. The human observer performance was compared with the performance of a multislice channelized Hotelling observer (CHO_MS), which integrates multislice image data, and with the performance of previously validated CHO, which operates on static 2D images (CHO_2D). For comparison, the same 16 2AFC studies were also performed in a 2D viewing mode by the human observers and compared with the multislice viewing performance and the two CHO models. Human observer performance was well correlated with the CHO_2D performance in the 2D viewing mode [Pearson product-moment correlation coefficient R = 0.972, 95% confidence interval (CI): 0.919 to 0.990] and with the CHO_MS performance in the multislice viewing mode (R = 0.952, 95% CI: 0.865 to 0.984). The CHO_2D performance, calculated from the 2D viewing mode, also had a strong correlation with human observer performance in the multislice viewing mode (R = 0.957, 95% CI: 879 to 0.985). Human observer performance varied between the multislice and 2D modes. One reader performed better in the multislice mode (P = 0.013); whereas the other two readers showed no significant difference between the two viewing modes (P = 0.057 and P = 0.38). A 2D CHO model is highly correlated with human observer performance in detecting spherical low contrast objects in multislice viewing of CT images. This finding provides some evidence for the use of a simpler, 2D CHO to assess image quality in clinically relevant CT tasks where multislice viewing is used.

Illusory Increases in Font Size Improve Letter Recognition

Visual performance of human observers depends not only on the optics of the eye and early sensory encoding but also on subsequent cortical processing and representations. In two experiments, we demonstrated that motion adaptation can enhance as well as impair visual acuity. Observers who experienced an expanding motion aftereffect exhibited improved letter recognition, whereas observers who experienced a contracting motion aftereffect showed impaired letter recognition. We conclude that illusory enlargement and shrinkage of a visual stimulus can modulate visual acuity.