A robust quality estimation method for medical image segmentation with small datasets

Abstract

The quality check of automatic image analysis results is a necessity to eliminate poor outcomes. There are few work on segmentation quality estimation when there is no reference mask available. Generative-based and regression approaches generally have strict assumptions on shape/contrast of anatomical structures, which may fail when there are abnormalities in images or in the presence of domain shift. Ensemble approach promises more generalisability to various types of images; however, they are costly to train. Also, none of these methods were designed for small datasets. To address these shortcomings, this paper presents a segmentation quality estimation method for small datasets with arbitrary dimensions, which is validated with 2D, 3D and 4D image datasets with roughly 20 training images, and on the segmentation of retinal vessel and left atrium segmentation. Although our method uses an ensemble for image segmentation, its design reduces its parameter size. We describe possible scenarios relating the amount of agreement across the base models’ outputs in an ensemble to quality scores; then, present a technique to deal with high quality score estimation for poor segmentation as a result of the base models to largely agree on mistakes. We assess the performance of our method in the presence of different sources of domain shift, and compare it with methods selected from the aforementioned approaches. We found robust quality score estimation, generalisable to different datasets. Our code would be available upon acceptance.