Two complementary model observers to evaluate reconstructions of simulated micro-calcifications in digital breast tomosynthesis

Abstract

New imaging modalities need to be properly evaluated before being introduced in clinical practice. The gold standard is to perform clinical trials or dedicated clinical performance related observer experiments with experienced readers. Unfortunately this is not feasible during development or optimization of new reconstruction algorithms due to their many degrees of freedom. Our goal is to design a set of model observers to evaluate the performance of newly developed reconstruction methods on the assessment of micro-calcifications in digital breast tomosynthesis. In order to do so, the model observers need to evaluate both detection and classification of micro-calcifications. A channelized Hotelling observer was created for the detection task and a Hotelling observer working on an extracted feature vector was implemented for the classification task. These observers were evaluated on their ability to predict the results of human observers. Results from a previous observer study were used as reference to compare performance between human and model observers. This study evaluated detection of small micro-calcifications (100 { 200 _m) by a free search task in a power law filtered noise background and classification of two types of larger micro-calcifications (200 {600 _m) in the same background. Scores from the free search study were evaluated using the weighted JAFROC method and the classification scores were analyzed using the DBM MRMC method. The same analysis methods were applied to the model observer scores. Results of the detection model observer were related linearly with the human observer results with a correlation coefficient of 0.962. The correlation coefficient for the classification task was 0.959 with a power law non-linear regression.