Abstract Background Histologic remission (HR) is a critical treatment target in Ulcerative Colitis (UC). Among several scoring systems, the PICaSSO Histologic Remission Index (PHRI) simplifies HR assessment by evaluating the presence of neutrophils in the bowel tissue. Our artificial intelligence (AI) system built upon PHRI showed remarkable accuracy in HR assessment. PHRI assess neutrophils in four different regions of interest, so segmentation of these compartments is crucial to predict PHRI automatically. However, creating labelled histopathological datasets to train fully-supervised segmentation models takes time and effort. Hence, this study explores the impact of an active learning (AL) algorithm on enhancing image segmentation to alleviate the burden of detailed pathologists’ annotation and to standardise protocol annotation. Methods Biopsy samples from an international real-life prospective UC study were digitised into whole slide images (WSI). Initial annotations of superficial epithelium, lumen and epithelium of crypts and lamina propria for 33 WSI were employed to train a U-Net segmentation model at baseline. An AL framework was employed to iteratively select and annotate 15 unannotated images while selecting those with the highest uncertainty. Uncertainty was calculated using Least confidence sampling, Margin Sampling, and Shannon Entropy. The most informative samples, based on the average of the three uncertainty measure, were selected in consecutive batches of 5 images, and pathologists were enlisted in a human-in-the-loop process to refine annotations. Subsequently, the segmentation model was retrained by incorporating the newly refined annotated samples, and its performance was assessed using a fully annotated test set of 19 WSI. Results Following the baseline model training, the model’s segmentation performance assessed by the Dice score and Intersection over Union (IoU) was 0.622 and 0.386, respectively (see Table 1). Applying the AL algorithm with newly annotated images notably improved model performance, especially with 10 images (Dice=0.651, IoU=0.415). However, training the model with an additional 5 newly annotated images, which exhibited lower uncertainty, did not yield further improvement (Dice = 0.651, IoU = 0.415). Thus, the 10 annotated WSI demonstrating more uncertainty resulted crucial for the AL framework. Conclusion This novel AL-based iterative framework exhibits promise in standardising digital tissue annotation by our PHRI-based AI model. It offers a novel approach for both clinical trial and clinical practice, aiming to alleviate the burden of WSI labelling and reduce the bias of annotation, thereby improving histological assessment in UC.
Read full abstract