Abstract
BackgroundThe trend towards large-scale studies including population imaging poses new challenges in terms of quality control (QC). This is a particular issue when automatic processing tools such as image segmentation methods are employed to derive quantitative measures or biomarkers for further analyses. Manual inspection and visual QC of each segmentation result is not feasible at large scale. However, it is important to be able to automatically detect when a segmentation method fails in order to avoid inclusion of wrong measurements into subsequent analyses which could otherwise lead to incorrect conclusions.MethodsTo overcome this challenge, we explore an approach for predicting segmentation quality based on Reverse Classification Accuracy, which enables us to discriminate between successful and failed segmentations on a per-cases basis. We validate this approach on a new, large-scale manually-annotated set of 4800 cardiovascular magnetic resonance (CMR) scans. We then apply our method to a large cohort of 7250 CMR on which we have performed manual QC.ResultsWe report results used for predicting segmentation quality metrics including Dice Similarity Coefficient (DSC) and surface-distance measures. As initial validation, we present data for 400 scans demonstrating 99% accuracy for classifying low and high quality segmentations using the predicted DSC scores. As further validation we show high correlation between real and predicted scores and 95% classification accuracy on 4800 scans for which manual segmentations were available. We mimic real-world application of the method on 7250 CMR where we show good agreement between predicted quality metrics and manual visual QC scores.ConclusionsWe show that Reverse classification accuracy has the potential for accurate and fully automatic segmentation QC on a per-case basis in the context of large-scale population imaging as in the UK Biobank Imaging Study.
Highlights
The trend towards large-scale studies including population imaging poses new challenges in terms of quality control (QC)
We report the evaluation metrics for each class label: left ventricular (LV) cavity, LV myocardium (LVM) and right ventricular (RV) cavity (RVC)
Experiment B: Large-scale validation on manually-segmented UK Biobank (UKBB) Data, N = 4805 In this experiment we demonstrate that Reverse Classification Accuracy (RCA) is robust for employment in large-scale studies, and produces accurate predictions of segmentation quality on an individual basis
Summary
The trend towards large-scale studies including population imaging poses new challenges in terms of quality control (QC) This is a particular issue when automatic processing tools such as image segmentation methods are employed to derive quantitative measures or biomarkers for further analyses. It is important to understand the limitations of such pipelines and assess the quality of the results being reported This is a particular issue when we consider large-scale population imaging databases comprising thousands of images such as the UK Biobank (UKBB) Imaging Study [1]. It is infeasible to perform a manual, visual inspection of all outputs, and even more difficult to perform quality control (QC) within the pipeline itself We break down this challenge and focus on the automated QC of image segmentation
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have