Non-prewhitening Model Observer Research Articles

3D printed phantom with 12 000 submillimeter lesions to improve efficiency in CT detectability assessment.

The detectability performance of a CT scanner is difficult to precisely quantify when nonlinearities are present in reconstruction. An efficient detectability assessment method that is sensitive to small effects of dose and scanner settings is desirable. We previously proposed a method using a search challenge instrument: a phantom is embedded with hundreds of lesions at random locations, and a model observer is used to detect lesions. Preliminary tests in simulation and a prototype showed promising results. In this work, we fabricated a full-size search challenge phantom with design updates, including changes to lesion size, contrast, and number, and studied our implementation by comparing the lesion detectability from a nonprewhitening (NPW) model observer between different reconstructions at different exposure levels, and by estimating the instrument sensitivity to detect changes in dose. Designed to fit into QRM anthropomorphic phantoms, our search challenge phantom is a cylindrical insert 10cm wide and 4cm thick, embedded with 12 000 lesions (nominal width of 0.6mm, height of 0.8mm, and contrast of -350 HU), and was fabricated using PixelPrint, a 3D printing technique. The insert was scanned alone at a high dose to assess printing accuracy. To evaluate lesion detectability, the insert was placed in a QRM thorax phantom and scanned from 50 to 625 mAs with increments of 25 mAs, once per exposure level, and the average of all exposure levels was used as high-dose reference. Scans were reconstructed with three different settings: filtered-backprojection (FBP) with Br40 and Br59, and Sinogram Affirmed Iterative Reconstruction (SAFIRE) with strength level 5 and Br59 kernel. An NPW model observer was used to search for lesions, and detection performance of different settings were compared using area under the exponential transform of free response ROC curve (AUC). Using propagation of uncertainty, the sensitivity to changes in dose was estimated by the percent change in exposure due to one standard deviation of AUC, measured from 5 repeat scans at 100, 200, 300, and 400 mAs. The printed insert lesions had an average position error of 0.20mm compared to printing reference. As the exposure level increases from 50 mAs to 625 mAs, the lesion detectability AUCs increase from 0.38 to 0.92, 0.42 to 0.98, and 0.41 to 0.97 for FBP Br40, FBP Br59, and SAFIRE Br59, respectively, with a lower rate of increase at higher exposure level. FBP Br59 performed best with AUC 0.01 higher than SAFIRE Br59 on average and 0.07 higher than FBP Br40 (all P<0.001). The standard deviation of AUC was less than 0.006, and the sensitivity to detect changes in mAs was within 2% for FBP Br59. Our 3D-printed search challenge phantom with 12 000 submillimeter lesions, together with an NPW model observer, provide an efficient CT detectability assessment method that is sensitive to subtle effects in reconstruction and is sensitive to small changes in dose.

Improvements in dose efficiency with high resolution scan modes in photon counting CT.

For the detection of very small objects, high resolution detectors are expected to provide higher dose efficiency. We assessed this impact of increased resolution on a clinical photon counting detector CT (PCD-CT) by comparing its detectability in high resolution and standard resolution (with 2×2 binning and larger focal spot) modes. A 50μm-thin metal wire was placed in a thorax phantom and scanned in both modes at three exposure levels (12, 15, and 18 mAs); acquired data were reconstructed with three reconstruction kernels (Br40, Br68, and Br76, from smooth to sharp). A scanning nonprewhitening model observer searched for the wire location within each slice independently. Detection performance was quantified as area under the exponential transform of the free response ROC curve. The high-resolution mode had the mean AUCs at 18 mAs of 0.45, 0.49, and 0.65 for Br40, Br68, and Br76, respectively, which were 2 times, 3.6 times, and 4.6 times those of the standard resolution mode. The high-resolution mode achieved greater AUC at 12 mAs than the standard resolution mode at 18 mAs for every reconstruction kernel, but improvements were larger at sharper kernels. The results are consistent with the greater suppression of noise aliasing expected at higher frequencies with high resolution CT. This work illustrates that PCD-CT can provide large dose efficiency gains for detection tasks of small, high contrast lesions.

Performance of high-resolution CT for detection and discrimination tasks related to stenotic lesions - A phantom study using model observers.

Accurate detection and grading of atheromatous stenotic lesions within the cardiac, renal, and intracranial vasculature is imperative for early recognition of disease and guiding treatment strategies. In this work, a stenotic lesion phantom was used to compare high resolution and normal resolution modes on the same CT scanner in terms of detection and size discrimination performance. The phantom is comprised of three acrylic cylinders (each 15.0cm in diameter and 1.3cm thick) with a matching array of holes in each module. The outer two modules contain holes that are slightly larger than the corresponding hole in the central module to simulate stenotic narrowing in vasculature. The stack of modules was submerged in an iodine solution simulating contrast-enhanced stenotic lesions with a range of lumen diameters (1.32-10.08mm) and stenosis severity (0%, 50%, 60%, 70%, and 80%). The phantom was imaged on the Canon Aquilion Precision high-resolution CT scanner in high-resolution (HR) mode (0.25mm × 0.50mm detector element size) and normal-resolution (NR) mode (0.50mm × 0.50mm) using 120 kV and two dose levels (14 and 21mGy SSDE) with 30 repeat scans acquired for each combination. Filtered back-projection (FBP) and a hybrid-iterative reconstruction (AIDR) were used with the FC18 kernel, as well as a deep learning algorithm (AiCE) which is only available for HR. A non-prewhitening model observer with an eye filter was implemented to quantify performance for detection and size discrimination tasks in the axial plane. Detection performance improved with increasing diameter, dose, and for AIDR in comparison to FBP for a fixed resolution mode. Performance in the HR mode was generally higher than NR for the smaller lumen diameters (1-5mm) with decreasing differences as the diameter increased. Performance in NR mode surpassed HR mode for lumen diameters greater than ∼4mm and ∼5mm for 14mGy and 21mGy, respectively. AiCE provided consistently higher detection performance compared with AIDR-FC18 (48% higher for a 6mm lumen diameter). Discrimination performance increased with increasing nominal diameter, dose, and for larger differences in stenosis severity. When comparing discrimination performance in HR to NR modes, the largest relative differences occur at the smallest nominal diameters and smallest differences in stenosis severity. The AiCE reconstruction algorithm produced the highest overall discrimination performance values, and these were significantly higher than AIDR-FC18 for nominal diameters of 7.14 and 10.08mm. HR mode outperforms NR for detection up to a specific diameter and the results improve with AiCE and for higher dose levels. For the task of size discrimination, HR mode consistently outperforms NR if AIDR-FC18 is used for dose levels of at least 21mGy, and the results improve with AiCE and for the smallest differences in stenosis severity investigated (50% vs. 60%). High-resolution CT appears to be beneficial for detecting smaller simulated lumen diameters (<5mm) and is generally advantageous for discrimination tasks related to stenotic lesions, which inherently contain information at higher frequencies, given the right reconstruction algorithm and dose level.

Experimental feasibility of xenon-enhanced dual-energy radiography for imaging of lung function

Chronic obstructive pulmonary disease (COPD) is a leading cause of death worldwide. We experimentally investigated the feasibility of two-dimensional xenon-enhanced dual-energy (XeDE) radiography for imaging of lung function. We optimized image quality under quantum-noise-limited conditions using a chest phantom consisting of a rectangular chamber representing the thoracic volume and PMMA slabs simulating x-ray attenuation by soft tissue. A sealed, air-filled cavity with thin PMMA walls was positioned inside the chamber to simulate a 2 cm thick ventilation defect. The chamber was ventilated with xenon and dual-energy imaging was performed using a diagnostic x-ray tube and a flat-panel detector. The contrast-to-noise ratio of ventilation defects normalized by patient x-ray exposure maximized at a kV-pair of approximately 60/140-kV and when approximately one third of the total exposure was allocated to the HE image. We used the optimized technique to image a second phantom that contained lung-parenchyma-mimicking PMMA clutter, rib-mimicking aluminum slats and an insert that simulated ventilation defects with thicknesses ranging from 0.5 cm to 2 cm and diameters ranging from 1 cm to 2 cm. From the resulting images we computed the area under the receiver operating characteristic curve (AUC) of the non-prewhitening model observer with an eye filter and internal noise. For a xenon concentration of 75%, good AUCs (i.e. 0.8–0.9) to excellent AUCs (i.e. >0.9) were obtained when the defect diameter is greater than 1.3 cm and defect thickness is 1 cm. When the xenon concentration was reduced to 50%, the AUC was ∼0.9 for defects 1.2 cm in diameter and ∼1.5 cm in thickness. Two-dimensional XeDE radiography may therefore enable detection of functional abnormalities associated with early-stage COPD, for which xenon ventilation defects can occupy up to 20% of the lung volume, and should be further developed as a low-cost alternative to MRI-based approaches and a low-dose alternative to CT-based approaches.

Xenon-enhanced dual-energy tomosynthesis for functional imaging of respiratory disease-Concept and phantom study.

Xenon-enhanced dual-energy (DE) computed tomography (CT) and hyperpolarized noble-gas magnetic resonance imaging (MRI) provide maps of lung ventilation that can be used to detect chronic obstructive pulmonary disease (COPD) early in its development and predict respiratory exacerbations. However, xenon-enhanced DE-CT requires high radiation doses and hyper-polarized noble-gas MRI is expensive and only available at a handful of institutionsglobally. To present xenon-enhanced dual-energy tomosynthesis (XeDET) for low-dose, low-cost functional imaging of respiratory disease in an experimental phantomstudy. We propose using digital tomosynthesis to produce Xe-enhanced low-energy (LE) and high-energy (HE) coronal images. DE subtraction of the LE and HE images is used to suppress soft tissues. We used an imaging phantom to investigate image quality in terms of the area under the reciever operating characteristic curve (AUC) for the Non-PreWhitening model observer with an Eye filter and internal noise (NPWEi). The phantom simulated anatomic clutter due to lung parenchyma and attenuation due to soft tissue and lung tissue. Aluminum slats were used to simulate rib structures. A stepwedge consisting of an acrylic casing with sealed cylindrical air-filled cavities was used to simulate ventilation defects with step thicknesses of 0.5, 1, and 2cm and cylindrical radii of 0.5, 0.75, and 1cm. The phantom was ventilated with Xe and projection data were acquired using a flat-panel detector, a tube-voltage combination of 60/140kV with 1.2 mm of copper filtration on the HE spectrum and an angular range of in 1° increments. The AUC of a NPWEi observer that has access only to a single coronal slice was calculated from measurements of the three-dimensional noise power spectrum and signal template. The AUC was calculated as a function of ventilation defect thickness and radius for total patient entrance air kermas ranging from 1.42 to 2.84mGy with and without rib-simulating Al slats. For the AUC analysis, the observer internal noise level was obtained from an ad hoc calibration to a high-dose dataset. XeDET was able to suppress parenchyma-simulating clutter in coronal images enabling visualization of the simulated ventilation defects, but the limited angle acquisition resulted in residual clutter due to out-of-plane bone-mimmicking structures. The signal power of the defects increased linearly with defect radius and showed a ten-fold to fifteen-fold increase in signal power when the defect thickness increased from 0.5 to 2cm. These trends agreed with theoretical predictions. Along the depth dimension, the power of the defects decreased exponentially with distance from the center of the defects with full-width half maxima that varied from 1.85to 2.85cm depending on the defect thickness and radius. The AUCs of the 1-cm-radius defect that was 2cm in thickness ranged from good (0.8-0.9) to excellent (0.9-1.0) over the range of air kermasconsidered. Xenon-enhanced DE tomosynthesis has the potential to enable functional imaging of respiratory disease and should be further investigated as a low-cost alternative to MRI-based approaches and a low-dose alternative to CT-basedapproaches.

A tin filter’s dose reduction effect revisited: Using the detectability index in low-dose computed tomography for the chest

PurposeTo reevaluate a tin filter's (TF) dose reduction effect in computed tomography (CT) using a combination of an anthropomorphic chest phantom and a rod-shaped phantom. Methods and materialsA third-generation dual-source CT system equipped with a built-in TF was employed. A chest phantom was scanned under low-dose conditions of 0.2 to 1.0 mGy with the TF at 100 kV (TF100kV) and without it at 100 kV and 120 kV (NF100kV and NF120kV). To eliminate effects other than that of the TF, only filtered back projection (FBP) was used for image reconstruction. On the images of the rod phantom placed inside the lung field, the CT number and the spatial resolution using the modulation transfer function (MTF) were measured. Using these indices plus the noise power spectrum (NPS) that was also measured, the detectability index based on the non-prewhitening model observer (d'NPW) was calculated. ResultsThe CT numbers and MTFs were almost identical across the three conditions. The area under the NPS curve was decreased by 13–17% with the TF compared with non-TF conditions. NPS increases at low frequencies of < 0.06 mm−1 observed in NF120kV and NF100kV were eliminated by TF100kV. The potential dose reduction by the TF, estimated using the d'NPW values, turned out to be 22 to 25%. ConclusionBased on the analysis of the FBP images of a chest phantom, the dose reduction attributable only to the TF was estimated at 22–25%, notably lower than those reported in previous studies.

Computed tomography protocols optimization using non-prewhitening model observer

The non-prewhitening model observer with eye filter (NPWE) allows assessing of performances of a whole imaging system, taking into account specific aspects of a considered clinical task and also the aspects of human visual system. The aim of this work is to determine the exposure parameters for computed tomography (CT) scan protocols so that they produce the satisfied performances of a human observer (HO). For each of the chosen scan protocols, the assessment of image quality parameters, based on the Fourier metric, as well as the volume CT dose index (CTDIvol) measurements, have been performed. The scan protocols using the smooth reconstruction kernel, for an object diameter of 3 mm, 7 mm and 10 mm, and a contrast level of 20 Hounsfield units (HU) and 40 HU, gave the values of the area under curve (AUC) greater than 0.75 for CTDIvol > 7.16 mGy. For a sharp kernel, all AUC values were smaller than 0.75 for all investigated scan protocols, object diameters and contrast levels. Through this work it was shown that the NPWE model-observer can be used as an effective and comprehensive optimization tool suitable for application in the various clinical tasks, considering also CTDIvol values.

Validation of synthesized normal-resolution image data generated from high-resolution acquisitions on a commercial CT scanner.

To validate a normal-resolution (NR) simulation (NRsim) algorithm that uses high-resolution (HR) or super-high resolution (SHR) acquisitions on a commercial HR computed tomography (CT) scanner by comparing image quality between NRsim-generated images and actual NR images. NRsim is intended to allow direct comparison between normal-resolution CT and HR/SHR reconstructions in clinical investigations, without repeating exams. The Aquilion Precision CT (Canon Medical Systems Corporation) HR CT scanner has three resolution modes resulting from detector binning in the channel (x-y) and row (z) directions. For NR, each detector element is 0.5mm×0.5mm along the channel and row directions, 0.25mm×0.5mm for HR, and 0.25mm×0.25mm for SHR. The NRsim algorithm simulates NR acquisitions from HR or SHR acquisitions (termed NRHR and NRSHR , respectively) by downsampling the pre-log raw data in the channel direction for the HR acquisitions and in the channel and row direction for the SHR acquisition. The downsampled data are then reconstructed using the same process as NR. The axial modulation transfer function (MTF), slice sensitivity profile (SSP), and CT number accuracy were measured using the Catphan 600 phantom, and the three-dimensional noise power spectrum (NPS) was measured in water-equivalent phantoms for standard protocols across a range of size-specific dose estimates (SSDE): head (6.2-29.8mGy), lung (2.2-18.2mGy), and body (5.6-19.4mGy). The MTF and NPS measurements were combined to estimate low-contrast detectability (LCD) using a non-prewhitening model observer with an eye filter for a 5-mm disk with 10HU contrast. All metrics were compared for NR, NRHR , and NRSHR images reconstructed using filtered back projection (FBP) and an iterative reconstruction algorithm (AIDR3D). We chose a 15% error threshold as a reasonable definition of success for NRsim when compared against actual NR based on published studies showing that a just-noticeable difference in image noise level for human observers is typically <15%. The axial MTF and SSPs for NRsim were in good agreement with NR demonstrated by a maximum difference of 5.1% for the MTF at 10% and 50% across materials (air, Teflon, LDPE, and polystyrene) and a maximum SSP difference of 2.2%. Noise magnitude differences were within 15% across the SSDE levels with the exception of below 4.5mGy for the lung protocol with FBP. The relative RMSE of normalized NPS comparisons were all <15%. Differences in CT numbers for NRsim reconstructions were within 2HU of NR. LCD for NRsim was within 15% of NR with the exception of NRSHR for the lung protocol SSDE levels below 3.7mGy with FBP. NRsim, an algorithm for simulating NR acquisitions using HR and SHR raw data, was introduced and shown to generate images with spatial resolution, noise, HU accuracy, and LCD largely equivalent to scans acquired using an actual NR acquisition. At SSDE levels below ~5mGy for the lung protocol, differences in noise magnitude and LCD for NRSHR were >15% which defines a region where NRsim degrades due to contributions from electronic noise.

In-plane image quality and NPWE detectability index in digital breast tomosynthesis

A rigorous 2D analysis of signal and noise transfer applied to reconstructed planes in digital breast tomosynthesis (DBT) is necessary for system characterization and optimization. This work proposes a method for assessing technical image quality and system detective quantum efficiency (DQEsys) for reconstructed planes in DBT. Measurements of 2D in-plane modulation transfer function (MTF) and noise power spectrum (NPS) were made on five DBT systems using different acquisition parameters, reconstruction algorithms and plane spacing. This work develops the noise equivalent quanta (NEQ), DQEsys and detectability index (d’) calculated using a non-prewhitening model observer with eye filter (NPWE) for reconstructed DBT planes. The images required for this implementation were acquired using a homogeneous test object of thickness 40 mm poly(methyl) methacrylate plus 0.5 mm Al; 2D MTF was calculated from an Al disc of thickness 0.2 mm and diameter 50 mm positioned within the phantom. The radiant contrast of the MTF disc and the air kerma at the system input were used as normalization factors. The NPWE detectability index was then compared to the in-plane contrast-detail (c-d) threshold measured using the CDMAM phantom.The MTF and NPS measured on the different systems showed a strong anisotropy, consistent with the cascaded models developed in the literature for DBT. Detectability indices calculated from the measured MTF and NPS successfully predicted changes in c-d detectability for details between 0.1 mm and 2.0 mm, for DBT plane spacings between 0.5 mm and 10 mm, and for air kerma values at the system input between 157 µGy and 1170 μGy. The linear Pearson correlation between the detectability index and threshold gold thickness of the CDMAM phantom was −0.996. The method implements a parametric means of assessing the technical image quality of reconstructed DBT planes, providing valuable information for optimization of DBT systems.

Model observers for Low Contrast Detectability evaluation in dynamic angiography: A feasible approach

PurposeTo evaluate the feasibility of spatio-temporal generalisation of mathematical methods for protocol optimisation in interventional radiology. Materials and methodsTwo model observers were considered:•Channelized Hotelling Observer with spatio-temporal Gabor channels;•Non-Prewhitening Model Observer with spatio-temporal contrast sensitivity function.Furthermore, Low Contrast Detectability (LCD) was evaluated with a generalised statistical method by taking into account the noise integration capability of the human eye.A series of two alternative force choices (2AFC) experiments performed by four observers were used to evaluate the reliability of the proposed models.The evaluation of the mathematical methods was performed by comparing their results to the human observer performances in two steps:1. Firstly, a series of simulated images were used to tune the models2. In the second phase, tuned models were applied both to simulated images and actual images obtained with a commercial phantom to evaluate detectability scores. ResultsEvaluation with simulated images shows a good agreement with 2AFC results (RMSE < 10%). Phantom-based evaluations show a general decrease of such agreement, characterized by an RMSE lower than 16%. ConclusionsThe agreement with human observer experiments supports the feasibility of the proposed generalisations. Thus, they could be introduced in quality control programmes for a deeper protocol-characterisation or for clinical protocol-optimization when dynamic images are involved.

Signal template generation from acquired images for model observer-based image quality analysis in mammography.

Mammography images undergo vendor-specific processing, which may be nonlinear, before radiologist interpretation. Therefore, to test the entire imaging chain, the effect of image processing should be included in the assessment of image quality, which is not current practice. For this purpose, model observers (MOs), in combination with anthropomorphic breast phantoms, are proposed to evaluate image quality in mammography. In this study, the nonprewhitening MO with eye filter and the channelized Hotelling observer were investigated. The goal of this study was to optimize the efficiency of the procedure to obtain the expected signal template from acquired images for the detection of a 0.25-mm diameter disk. Two approaches were followed: using acquired images with homogeneous backgrounds (approach 1) and images from an anthropomorphic breast phantom (approach 2). For quality control purposes, a straightforward procedure using a single exposure of a single disk was found adequate for both approaches. However, only approach 2 can yield templates from processed images since, due to its nonlinearity, image postprocessing cannot be evaluated using images of homogeneous phantoms. Based on the results of the current study, a phantom should be designed, which can be used for the objective assessment of image quality.

Update on the non-prewhitening model observer in computed tomography for the assessment of the adaptive statistical and model-based iterative reconstruction algorithms

The state of the art to describe image quality in medical imaging is to assess the performance of an observer conducting a task of clinical interest. This can be done by using a model observer leading to a figure of merit such as the signal-to-noise ratio (SNR). Using the non-prewhitening (NPW) model observer, we objectively characterised the evolution of its figure of merit in various acquisition conditions. The NPW model observer usually requires the use of the modulation transfer function (MTF) as well as noise power spectra. However, although the computation of the MTF poses no problem when dealing with the traditional filtered back-projection (FBP) algorithm, this is not the case when using iterative reconstruction (IR) algorithms, such as adaptive statistical iterative reconstruction (ASIR) or model-based iterative reconstruction (MBIR). Given that the target transfer function (TTF) had already shown it could accurately express the system resolution even with non-linear algorithms, we decided to tune the NPW model observer, replacing the standard MTF by the TTF. It was estimated using a custom-made phantom containing cylindrical inserts surrounded by water. The contrast differences between the inserts and water were plotted for each acquisition condition. Then, mathematical transformations were performed leading to the TTF. As expected, the first results showed a dependency of the image contrast and noise levels on the TTF for both ASIR and MBIR. Moreover, FBP also proved to be dependent of the contrast and noise when using the lung kernel. Those results were then introduced in the NPW model observer. We observed an enhancement of SNR every time we switched from FBP to ASIR to MBIR. IR algorithms greatly improve image quality, especially in low-dose conditions. Based on our results, the use of MBIR could lead to further dose reduction in several clinical applications.

Comparison of signal to noise ratios from spatial and frequency domain formulations of nonprewhitening model observers in digital mammography

Image quality indices based upon model observers are promising alternatives to laborious human readings of contrast-detail images. This is especially appealing in digital mammography as limiting values for contrast thresholds determine, according to some international protocols, the acceptability of these systems in the radiological practice. The objective of the present study was to compare the signal to noise ratios (SNR) obtained with two nonprewhitening matched filter model observer approaches, one in the spatial domain and the other in the frequency domain, and with both of them worked out for disks as present in the CDMAM phantom. The analysis was performed using images acquired with the Siemens Novation and Inspiration digital mammography systems. The spatial domain formulation uses a series of high dose CDMAM images as the signal and a routine exposure of two flood images to calculate the covariance matrix. The frequency domain approach uses the mathematical description of a disk and modulation transfer function (MTF) and noise power spectrum (NPS) calculated from images. For both systems most of the SNR values calculated in the frequency domain were in very good agreement with the SNR values calculated in the spatial domain. Both the formulations in the frequency domain and in the spatial domain show a linear relationship between SNR and the diameter of the CDMAM discs. The results suggest that both formulations of the model observer lead to very similar figures of merit. This is a step forward in the adoption of figures of merit based on NPS and MTF for the acceptance testing of mammography systems.

Toward objective and quantitative evaluation of imaging systems using images of phantoms

The use of imaging phantoms is a common method of evaluating image quality in the clinical setting. These evaluations rely on a subjective decision by a human observer with respect to the faintest detectable signal(s) in the image. Because of the variable and subjective nature of the human-observer scores, the evaluations manifest a lack of precision and a potential for bias. The advent of digital imaging systems with their inherent digital data provides the opportunity to use techniques that do not rely on human-observer decisions and thresholds. Using the digital data, signal-detection theory (SDT) provides the basis for more objective and quantitative evaluations which are independent of a human-observer decision threshold. In a SDT framework, the evaluation of imaging phantoms represents a "signal-known-exactly/background-known-exactly" ("SKE/ BKE") detection task. In this study, we compute the performance of prewhitening and nonprewhitening model observers in terms of the observer signal-to-noise ratio (SNR) for these "SK E/BKE" tasks. We apply the evaluation methods to a number of imaging systems. For example, we use data from a laboratory implementation of digital radiography and from a full-field digital mammography system in a clinical setting. In addition, we make a comparison of our methods to human-observer scoring of a set of digital images of the CDMAM phantom available from the internet (EUREF-European Reference Organization). In the latter case, we show a significant increase in the precision of the quantitative methods versus the variability in the scores from human observers on the same set of images. As regards bias, the performance of a model observer estimated from a finite data set is known to be biased. In this study, we minimize the bias and estimate the variance of the observer SNR using statistical resampling techniques, namely, "bootstrapping" and "shuffling" of the data sets. Our methods provide objective and quantitative evaluation of imaging systems with increased precision and reduced bias.