An algorithm for automatic analysis of portal images: clinical evaluation for prostate treatments

Abstract

The aim of this study is to assess the clinical value of an algorithm for automatic analysis of portal images by measuring the method's performance in a clinical study of treatment of prostate cancer. The algorithm is based on chamfer matching and measures displacements of patients relative to prescribed radiation beam positions. In this paper we propose a method to quantify the mean standard deviation (MSD) of the performance of automatic analysis relative to the MSD of the performance of trained radiographers using the clinical data set only, i.e. without using additional phantoms or simulations. The clinical data set in this study consists of 99 regional AP prostate images of 15 different patients. To assess the performance the automatic analysis in relation to that of the human observers, we studied the results of the unsupervised automatic analysis, as well as the results of a less-trained human observer and a well-trained human observer assisted by the automatic analysis (in this combination, automatic analysis is done first and the result is modified by the well-trained observer if the observer does not agree). First, the intra-observer variations of the well-trained observer are measured by repetitive analysis of a small subset of the clinical data, The distribution of differences in analysis between two arbitrary observers is described by the χ 2 distribution, and is tabulated in literature. We define the agreement histogram of an observer O as an estimator for the χ 2 distribution between O and the well-trained human observer, parametrized by the ratio of the intra-observer variations of O and the well-trained observer. The ratio parameter indicates the MSD of O relative to the MSD of the well-trained observer and is quantified by fitting the agreement histogram with the χ 2 distribution for three degrees of freedom (two translations and one rotation). The mean relative standard deviation of the well-trained human observer is 1,0 (about 0,6 mm translation and 0,4° rotation). The mean relative standard deviation of the lesst-rained human observer was found to be 2.1, 1.4 for the automatic analysis (discarding 4% failures), and 1.5 for the well-trained human observer assisted by the automatic analysis. The average computation time of the automatic analysis is around 2 s on a 66-MHz 80486 PC, compared with 30 s to 1 min for the human observers. From these numbers it is concluded that the well-trained human observer mainly corrects the failures in the automatic analysis. The small decrease in precision of the well-trained human observer when assisted by the automatic analysis is paid for by a large reduction in workload. Thirdly, the performance of the less-trained human observer can be improved when this observer is assisted by the automatic analysis in such way that the observer only corrects the failures of the automatic procedure.