Segmentation Quality Research Articles

An Efficient and Light Transformer-Based Segmentation Network for Remote Sensing Images of Landscapes

High-resolution image segmentation for landscape applications has garnered significant attention, particularly in the context of ultra-high-resolution (UHR) imagery. Current segmentation methodologies partition UHR images into standard patches for multiscale local segmentation and hierarchical reasoning. This creates a pressing dilemma, where the trade-off between memory efficiency and segmentation quality becomes increasingly evident. This paper introduces the Multilevel Contexts Weighted Coupling Transformer (WCTNet) for UHR segmentation. This framework comprises the Mult-level Feature Weighting (MFW) module and Token-based Transformer (TT) designed to weigh and couple multilevel semantic contexts. First, we analyze the multilevel semantics within a local patch without image-level contextual reasoning. It avoids complex image-level contextual associations and eliminates the misleading information carried. Second, MFW is developed to weigh shallow and deep features for enhancing object-related attention at different grain sizes from multilevel semantics. Third, the TT module is introduced to couple multilevel semantic contexts and transform them into semantic tokens using spatial attention. Then, we can capture token interactions and obtain clearer local representations. The suggested contextual weighting and coupling of single-scale patches empower WCTNet to maintain a well-balanced relationship between accuracy and computational overhead. Experimental results show that WCTNet achieves state-of-the-art performance on two UHR datasets of DeepGlobe and Inria Aerial.

A transformer and self-cascade operation-based architecture for segmenting high-resolution bridge cracks

High-resolution (HR) crack image segmentation is crucial for accurate bridge safety diagnosis. However, accurately segmenting cracks with their slender topology and random distribution in HR images poses significant challenges. Additionally, the restricted memory capacity of graphics cards presents a significant limitation for HR image segmentation. To tackle this demanding task, a approach called Cascade CATransUNet, consisting of a coordinate attention-enhanced transformer architecture with a self-cascaded design, was proposed in this study. Firstly, CATransUNet, a transformer-based multi-scale feature extraction architecture with the embedment of the coordinate attention mechanism, is customized to enhance the extraction of the crack's main contour both horizontally and vertically. Then, a self-cascade refinement operation is introduced on the basis of CATransUNet to reconstruct the details of the extracted crack features from the global and local levels successively. Furthermore, an optimized boundary loss based on the joint cascade loss function is introduced to improve segmentation quality in boundary areas. The necessity and effectiveness of all the proposed improvements were demonstrated through ablation studies conducted on both open-sourced crack dataset and HR crack images collected on-site. Moreover, the advancement of the proposed method was confirmed in a parallel comparative experiment. The self-cascaded transformer architecture achieved impressive performance with mIoU, mBIoU, and DICE scores exceeding 89.83%, 85.78%, and 96.97% respectively, on HR (4 K) images. Finally, the proposed method was tested through an unmanned aerial vehicle (UAV)-based bridge crack inspection task. The utilization of the Cascade CATransUNet enabled the UAV to accurately detect cracks even from considerable distances. This improvement enhances safety and detection efficiency in UAV inspections, ensuring flexibility in selecting flight paths, thus providing technical support for the promotion and application of UAV-based bridge crack detection technology.

Not without Context—A Multiple Methods Study on Evaluation and Correction of Automated Brain Tumor Segmentations by Experts

Brain tumor segmentations are integral to the clinical management of patients with glioblastoma, the deadliest primary brain tumor in adults. The manual delineation of tumors is time-consuming and highly provider-dependent. These two problems must be addressed by introducing automated, deep-learning-based segmentation tools. This study aimed to identify criteria experts use to evaluate the quality of automatically generated segmentations and their thought processes as they correct them. Multiple methods were used to develop a detailed understanding of the complex factors that shape experts' perception of segmentation quality and their thought processes in correcting proposed segmentations.Data from a questionnaire and semistructured interview withneuro-oncologistsand neuroradiologists were collected between August and December 2021 and analyzed using a combined deductive and inductive approach. Brain tumors are highly complex and ambiguous segmentation targets.Therefore, physicians rely heavily on the given context related to the patient and clinical context in evaluating the quality and need to correct brain tumor segmentation.Most importantly, the intended clinical application determines the segmentation quality criteria and editing decisions.Physicians' personal beliefs and preferences about the capabilities of AI algorithms and whether questionable areas should not be included are additional criteria influencing the perception of segmentation quality and appearance of an edited segmentation. Our findings on experts' perceptions of segmentation quality will allow the design of improved frameworks for expert-centered evaluation of brain tumor segmentation models.In particular, the knowledge presented here can inspire the development of brain tumor-specific metrics for segmentation model training and evaluation.

NIMG-76. A DEEP LEARNING ALGORITHM FOR FULLY AUTOMATED VOLUMETRIC MEASUREMENT OF MENINGIOMA BURDEN

Abstract PURPOSE Meningiomas are the most common primary central nervous system tumor. A subset of these tumors is multiply recurrent despite aggressive treatment with surgery or radiation and causes significant morbidity and mortality. Longitudinal measurement of meningioma burden on post-contrast MRI is the basis of treatment response assessment. As per Response Assessment in Neuro-Oncology (RANO), estimates of tumor size are based on the product of bidirectional diameters of enhancing tumor. However, these 2D segmentations as well as manual volumetric segmentations have high rates of intra- and inter-rater variability, which prohibits accurate longitudinal assessment of tumor size. A rapid and reproducible quantitative method for determining meningioma volume is needed to expedite response assessment in clinical trials. To address this unmet need, we developed a deep learning algorithm that enables fully automated 3D segmentation and volumetric assessment of meningioma burden. METHODS 351 post-contrast T1-weighted brain MRIs from 86 patients with meningioma were obtained from Massachusetts General Hospital and Dana-Farber Cancer Institute. The cohort was unique compared to prior DL segmentation models as it encompassed diverse clinical scenarios (e.g., meningioma of all grades, post-operative and post-radiated meningioma). The MRIs were resampled to 1 mm isotropic resolution and preprocessed using N4 bias field correction and zero-mean intensity normalization. The dataset was split 70%/15%/15% for training/validation/testing on a per patient level. The training data were used with manually generated tumor segmentations to train a convolutional neural network (CNN) with a U-Net architecture and the Dice loss function. RESULTS The median Dice score was 0.6758 on the test set. Additional work is ongoing to optimize performance, with primary focus directed toward improved data preprocessing and ground truth segmentation as the variability in segmentation quality contributed to the lower Dice score, highlighting the need to develop a less biased and more reliable tool to segment meningiomas.

Abstract 13323: Impact of an Institutional Process Change Adopting End-Systolic Coronary CTA Acquisition on Patient Throughput and Image Quality

Introduction: Prospective ECG-triggered end-systolic coronary computed tomographic angiography (CCTA) acquisition is less prone to motion artifacts with arrhythmias and high heart rates, and with increasing utilization, has the potential to improve scan and CT laboratory efficiency by reducing the need for IV/oral beta-blockers (BB) prior to the scan. Methods: We implemented an institutional process change in CCTA performance effective January 2023, transitioning from prospective ECG-triggered mid-diastolic acquisitions supervised by a physician at the scanner to an algorithmic approach with prospective end-systolic acquisition (200-400 ms after R peak) without BB administration, all on 3 rd -generation 192 slice dual-source scanners. We reviewed 100 consecutive CCTAs done pre- and post-process change in Jan 2022 and Jan 2023. Two experienced CCTA readers assessed segmental image quality by consensus using an 18-segment SCCT model on a 5-point Likert scale (1=non-interpretable; 2=poor; 3=satisfactory; 4=good; 5=excellent). Results: Post-process change, median overall patient appointment [94 (78 - 127) min vs. 69 (53 - 86) min; P < 0.001] and scan times [12 (10 - 16) min vs. 7 (6 - 9) min; P <.001] were significantly lower, and patient throughput doubled [2 (2 - 3)/day vs. 4 (3 - 6)/day; P =.024]. Median segmental image quality score decreased from excellent to good [5 (4 - 5) vs. 4 (4 - 5); P <.001], with a decrease in signal-noise ratio [23.5 (18 - 28.8) vs. 11.8 (8.7 - 14.7); P <.001], and no significant difference in poor or non-interpretable segments in the end-systolic cohort. More scans were repeated with end-systolic scans (10% vs. 18%), with no significant change in radiation dose (Table 1). Conclusion: Utilizing prospective ECG-triggered end-systolic acquisition avoided the use of BB, significantly reduced scan times, doubled patient throughput, at similar radiation doses, and maintained diagnostic performance with only a mild compromise in image quality.

Exploring a novel HE image segmentation technique for glioblastoma: A hybrid slime mould and differential evolution approach

Glioblastoma is a primary brain tumor with high incidence and mortality rates, posing a significant threat to human health. It is crucial to provide necessary diagnostic assistance for its management. Among them, Multi-threshold Image Segmentation (MIS) is considered the most efficient and intuitive method in image processing. In recent years, many scholars have combined different metaheuristic algorithms with MIS to improve the quality of Image Segmentation (IS). Slime Mould Algorithm (SMA) is a metaheuristic approach inspired by the foraging behavior of slime mould populations in nature. In this investigation, we introduce a hybridized variant named BDSMA, aimed at overcoming the inherent limitations of the original algorithm. These limitations encompass inadequate exploitation capacity and a tendency to converge prematurely towards local optima when dealing with complex multidimensional problems. To bolster the algorithm’s optimization prowess, we integrate the original algorithm with a robust exploitative operator called Differential Evolution (DE). Additionally, we introduce a strategy for handling solutions that surpass boundaries. The incorporation of an advanced cooperative mixing model accelerates the convergence of BDSMA, refining its precision and preventing it from becoming trapped in local optima. To substantiate the effectiveness of our proposed approach, we conduct a comprehensive series of comparative experiments involving 30 benchmark functions. The results of these experiments demonstrate the superiority of our method in terms of both convergence speed and precision. Moreover, within this study, we propose a MIS technique. This technique is subsequently employed to conduct experiments on IS at both low and high threshold levels. The effectiveness of the BDSMA-based MIS technique is further showcased through its successful application to the medical image of brain glioblastoma. The evaluation of these experimental outcomes, utilizing image quality metrics, conclusively underscores the exceptional efficacy of the algorithm we have put forth.

MLN-net: A multi-source medical image segmentation method for clustered microcalcifications using multiple layer normalization

The accurate segmentation of clustered microcalcifications in mammography is crucial for the diagnosis and treatment of breast cancer. Despite exhibiting expert-level accuracy, recent deep learning advancements in medical image segmentation provide insufficient contribution to practical applications, due to the domain shift resulting from differences in patient postures, individual gland density, and imaging modalities, etc. In this paper, a novel framework named MLN-net is proposed for clustered microcalcification segmentation. It can segment multi-source images using only single source images. Specifically, to rich domain distribution information, we introduce a source domain image augmentation for generating multi-source images. A structure of multiple layer normalization (LN) layers is then used to construct the segmentation network, which can be found efficient for clustered microcalcification segmentation in different domains. Additionally, a branch selection strategy is designed for measuring the similarity of the source domain data and the target domain data. To validate the proposed MLN-net, extensive analyses including ablation experiments are performed, comparison of 12 baseline methods. MLN-net enhances segmentation quality of full-field digital mammography (FFDM) and digital breast tomosynthe (DBT) images from the FFDM-DBT dataset, achieving the average Dice similarity coefficient (DSC) of 86.52% and the average Hausdorff distance (HD) of 20.49 mm on the source domain DBT. And it outperforms the baseline models for the task in FFDM images from both the CBIS-DDSM and the FFDM-DBT dataset, achieving the average DSC of 50.78% and the average HD of 35.12 mm on the source domain CBIS-DDSM. Extensive experiments validate the effectiveness of MLN-net in segmenting clustered microcalcifications from different domains and its segmentation accuracy surpasses state-of-the-art methods. Code will be available at https://github.com/yezanting/MLN-NET-VERSON1.

Estimating surface utilization factors for BIPV applications using pix2pix on street captured façade images

While techniques for assessing solar potential, particularly on roofs, are well-established, estimating solar potential on building facades often requires more work due to the complexity of urban features and the elaborate design of building facades. Existing methods to assess the solar potential of building facades often neglect the characteristics of individual facades. This study presents an image-based method for a more accurate estimation of the PV potential of facades. The proposed method is composed of four steps: (1) data acquisition, preprocessing, and manual labeling, (2) training a pixel-wise semantic image segmentation model based on Generative Adversarial Networks (GANs), (3) color content analysis of segmented images, and (4) estimating the annual solar energy potential. We apply the proposed workflow to several buildings in Zurich, Switzerland, and evaluate segmentation quality and resulting changes in façade surface utilization factors. In a comparative analysis between the widely used web-based solar potential assessment tools, Sonnenfassade, Global Solar Atlas and the simulation software Climate Studio, we demonstrate the downstream impact of surface utilization factors on resulting solar potentials. Considerable deviations in façade availability for BIPV deployments among the various tools as compared to our segmentation approach indicate the potential impact of the proposed method on policy-making and the benefits for BIPV design and planning.

A New Ready-to-Eat Product Based on Enzymatically Peeled 'Hernandina' Clementine Segments and Citrus Syrup.

Ready-to-eat fresh fruit have an increasing presence in international markets due to their convenience and health benefits. However, these products are highly perishable and efficient technologies to increase their shelf life are needed. In the present research, different citrus fruit species and cultivars from organic farming were assessed to obtain enzymatically peeled citrus segments. The best results in terms of segment quality were observed for 'Hernandina' clementine, which was chosen to make a new ready-to-eat product based on peeled citrus segments that were packaged in glass jars with a light syrup made of citrus juice and organic sugar cane. Different citrus juice mixtures were assayed and the most appreciated syrup, based on the sensory scores given by panellists, was that containing 50-50 (v/v) of 'Fino' lemon and 'Hernandina' clementine juices. In addition, different pasteurization treatments were assessed for their effects on conserving the safety, nutritional quality and sensory properties of the product during cold storage. The results show that pasteurization treatment at 50 °C for 45 min was sufficient to prevent microbial contamination with mesophilic and psychrophilic aerobic bacteria or yeast and mould and to maintain sensory properties until five weeks of storage at 4 °C. In addition, only a 10% reduction in vitamin C concentrations was observed in fresh-segments or syrup until the end of the storage period, showing that a high bioactive compound content and health benefits were conserved in the new ready-to-eat product after pasteurization and prolonged cold storage.

Deep Reinforcement Learning-Based Approach for Video Streaming: Dynamic Adaptive Video Streaming over HTTP

Dynamic adaptive video streaming over HTTP (DASH) plays a crucial role in delivering video across networks. Traditional adaptive bitrate (ABR) algorithms adjust video segment quality based on network conditions and buffer occupancy. However, these algorithms rely on fixed rules, making it challenging to achieve optimal decisions considering the overall context. In this paper, we propose a novel deep-reinforcement-learning-based approach for DASH streaming, with the primary focus of maintaining consistent perceived video quality throughout the streaming session to enhance user experience. Our approach optimizes quality of experience (QoE) by dynamically controlling the quality distance factor between consecutive video segments. We evaluate our approach through a comprehensive simulation model encompassing diverse wireless network environments and various video sequences. We also conduct a comparative analysis with state-of-the-art methods. The experimental results demonstrate significant improvements in QoE, ensuring users enjoy stable, high-quality video streaming sessions.

Asymmetric dual-decoder-U-Net for pavement crack semantic segmentation

Accurate pavement crack segmentation is crucial for civil engineering and infrastructure maintenance. To address the challenge of imbalanced data resulting from the prevalence of non-crack pixels, this research seeks to improve the quality of pavement crack segmentation, particularly for thick and tiny cracks. This paper presents an Asymmetric Dual-Decoder-U-Net (ADDU-Net) model, which involves constructing an asymmetric dual decoder with a dual attention module to better capture the features of both thick and tiny cracks under diverse environmental conditions. Through evaluation with images from four benchmark datasets, the ADDU-Net model demonstrates its effectiveness and robustness in accurately segmenting various types of cracks. This segmentation model shows significant potential for improving crack segmentation in industrial applications.

Research on a random sampling method for bulk grain based on the M-Unet and SGBM algorithms

In response to the challenge of limited real-time capability in identifying and positioning sampling points during random sampling of bulk grains in industrial settings, this paper introduces a novel approach. It combines the utilization of a lightweight neural network model, known as M-Unet, with the Semi-Global Block Matching (SGBM) algorithm to achieve the random sampling of bulk grains. The process commences with the deployment of a binocular camera to capture image data from the grain loading area, followed by binocular correction. Recognizing the constraints associated with the traditional U-Net network model, characterized by large parameters and suboptimal real-time performance, we introduce the M-Unet network, which is a more efficient alternative. M-Unet incorporates MobileNet as a substitute for the U-Net network encoder, resulting in a significant improvement in real-time performance and segmentation quality. Subsequently, the M-Unet network is employed to partition the corrected left image into multiple regions based on predefined sampling principles, and random sampling points are generated within each region. To determine the three-dimensional spatial positioning of these sampling points, we employ the SGBM algorithm, a semi-global stereo matching technique, on the left and right images of the corrected grain loading area. Experimental results demonstrate that the proposed method exhibits robust real-time performance and effectively recognizes grain loading areas. Furthermore, this method is suitable for deployment on portable devices, offering novel insights and methodologies for automating and advancing the intelligence of grain sampling processes. This innovation effectively addresses the challenges associated with low efficiency and human-related factors in traditional grain sampling. The findings presented in this paper carry substantial significance in safeguarding the interests of grain farmers, reducing food losses, minimizing waste, and enhancing the intelligence of sampling processes.

Self-navigated coronary MR angiography for coronary aneurysm detection in Kawasaki disease at 3T: comparison with conventional diaphragm-navigated coronary MR angiography.

To assess the scan time, image quality, and diagnostic performance of self-navigated coronary MR angiography (SN-CMRA) for coronary aneurysm (CAA) detection in Kawasaki disease (KD) patients and compare it with diaphragm-navigated CMRA (DN-CMRA). SN-CMRA and DN-CMRA were performed on 76 pediatric patients with KD (48 males, 6.75 ± 3.59 years). Thirty-three of whom underwent coronary CT angiography (CCTA)/invasive coronary angiography (ICA). The scan time and qualitative and quantitative image quality assessment were compared between the two sequences. The diagnostic performance for CAA detection by the two approaches using CCTA/ICA as the reference standard was compared on per-patient, per-vessel, and per-segment basis. The scan time of SN-CMRA was significantly shorter than that of DN-CMRA (7.49 ± 2.31 min vs. 10.03 ± 4.47 min, p< 0.001). There was no difference in overall and segmental image quality to reach the clinical diagnostic criteria between the two sequences (all p> 0.05). No significant difference in vessel length of the three main coronary arteries was found between the two approaches (all p> 0.05). Moreover, SN-CMRA showed no difference from DN-CMRA in contrast ratio of blood-myocardium (1.25 (interquartile range [IQR], 1.06 to 1.51) vs. 1.18 (IQR, 0.95 to 1.64), p= 0.706). There was no difference in the diagnostic accuracy of SN-CMRA and DN-CMRA for CAA detection on per-patient, per-vessel, or per-segment basis (all p> 0.05). SN-CMRA at 3T showed reliable diagnostic performance and application value for CAA detection in children with KD. Compared with DN-CMRA, SN-CMRA can simplify the scanning procedure and shorten the scan time, achieving comparable image quality and diagnostic accuracy. Coronary aneurysm in children with Kawasaki disease (KD) can be detected by self-navigated coronary MR angiography (CMRA) non-invasively and without radiation, achieving comparable image quality and diagnostic performance as diaphragm-navigated CMRA while shortening scanning time. It can provide reference for risk stratification and treatment management of KD. • Evaluating the size of coronary aneurysm is important for risk stratification and treatment of Kawasaki disease. • Self-navigated coronary MR angiography (SN-CMRA) shortens scan time and achieves comparable image quality and diagnostic performance compared with diaphragm-navigated coronary MR angiography. • SN-CMRA can evaluate coronary aneurysm non-invasively and without radiation, providing information for risk stratification and treatment.

Optimal segmentation of image datasets by genetic algorithms using color spaces

A vision system comprises several steps, with each step exerting a significant impact on the final outcome. One of these crucial steps is segmentation, which isolates the region of interest within an object and removes the background. Segmentation is vital because it enhances the quality of the isolated region, improves the accuracy of extracted features, and reduces the noise introduced by poor-quality features. Several segmentation techniques are available in the literature, each requiring one or more adjustable parameters. Selecting the most appropriate technique for segmenting a particular image dataset can be challenging. Several factors can affect the segmentation quality, including preprocessing, the chosen segmentation method, parameter fine-tuning approaches, and the characteristics of the dataset. Moreover, variations in lighting and intensity can further influence segmentation quality. At times, an expert may need to manually choose the segmentation technique and fine-tune its associated parameters. This paper presents the development of an automated algorithm for the selection of segmentation techniques and their associated parameters. The developed techniques are implemented and compared using diverse datasets, and the resulting experimental outcomes are thoroughly discussed and analyzed. The algorithm aims to streamline and simplify the process of selecting appropriate segmentation techniques, determining the required parameters, and selecting suitable pre-processing techniques.

Face Image Segmentation Using Boosted Grey Wolf Optimizer.

Image segmentation methods have received widespread attention in face image recognition, which can divide each pixel in the image into different regions and effectively distinguish the face region from the background for further recognition. Threshold segmentation, a common image segmentation method, suffers from the problem that the computational complexity shows exponential growth with the increase in the segmentation threshold level. Therefore, in order to improve the segmentation quality and obtain the segmentation thresholds more efficiently, a multi-threshold image segmentation framework based on a meta-heuristic optimization technique combined with Kapur's entropy is proposed in this study. A meta-heuristic optimization method based on an improved grey wolf optimizer variant is proposed to optimize the 2D Kapur's entropy of the greyscale and nonlocal mean 2D histograms generated by image computation. In order to verify the advancement of the method, experiments compared with the state-of-the-art method on IEEE CEC2020 and face image segmentation public dataset were conducted in this paper. The proposed method has achieved better results than other methods in various tests at 18 thresholds with an average feature similarity of 0.8792, an average structural similarity of 0.8532, and an average peak signal-to-noise ratio of 24.9 dB. It can be used as an effective tool for face segmentation.

Segmentation of plant residues on soil X‐ray CT images using neural network

AbstractIn soil, plant residues have low contrast making them difficult to detect using X‐ray computed tomography. In this work, we tested a convolutional neural network (U‐Net) for its ability to improve the identification of crop residues in soil samples assembled from aggregates of different size fractions (small, large, water‐stable aggregates, and average aggregate composition). Soil CT images were obtained using a 244 μm resolution. About 2500 soil images were annotated to train the neural network, of which only 631 images were selected for the training data set. Intersection over Union (IOU) was used as a measure of success of segmentation by neural network, which takes values from 0 to 1. In the validation data set, IOU of background was 0.93, IOU of solid phase was 0.95, IOU of pore space was 0.77, and IOU of plant residues was 0.40. However, IOU of plant residues in the total data set increased to 0.7. Soil structure influences the quality of multiphase segmentation of soil CT images. The poorest segmentation of plant residues was in the soil samples composed of average aggregate size composition. The quality of pore space segmentation increased with increasing porosity of the soil sample. The model tends to generalize the large areas occupied by plant residues and overlooks the smaller ones. The low values of the IOU metric for plant residues in the training data set can also be related to insufficient quality of annotation of the original images.

Self-supervised Semantic Segmentation: Consistency over Transformation.

Accurate medical image segmentation is of utmost importance for enabling automated clinical decision procedures. However, prevailing supervised deep learning approaches for medical image segmentation encounter significant challenges due to their heavy dependence on extensive labeled training data. To tackle this issue, we propose a novel self-supervised algorithm, , which integrates a robust framework based on the proposed Inception Large Kernel Attention (I-LKA) modules. This architectural enhancement makes it possible to comprehensively capture contextual information while preserving local intricacies, thereby enabling precise semantic segmentation. Furthermore, considering that lesions in medical images often exhibit deformations, we leverage deformable convolution as an integral component to effectively capture and delineate lesion deformations for superior object boundary definition. Additionally, our self-supervised strategy emphasizes the acquisition of invariance to affine transformations, which is commonly encountered in medical scenarios. This emphasis on robustness with respect to geometric distortions significantly enhances the model's ability to accurately model and handle such distortions. To enforce spatial consistency and promote the grouping of spatially connected image pixels with similar feature representations, we introduce a spatial consistency loss term. This aids the network in effectively capturing the relationships among neighboring pixels and enhancing the overall segmentation quality. The approach iteratively learns pixel-level feature representations for image content clustering in an end-to-end manner. Our experimental results on skin lesion and lung organ segmentation tasks show the superior performance of our method compared to the SOTA approaches.

Sampling Propagation Attention With Trimap Generation Network for Natural Image Matting

Natural image matting aims to precisely separate foreground objects from backgrounds using alpha mattes. Fully automatic natural image matting without external annotations is challenging. Well-performed matting methods usually require accurate labor-intensive handcrafted trimap as an extra input while the performance of automatic trimap generation method, e.g., erosion/dilation manipulation on foreground segmentation, fluctuates with segmentation quality. Therefore, we argue that how to produce a high-quality trimap using coarse segmentation is a major issue in automatic matting. In this paper, we present a two-stage trimap-free natural image matting pipeline that does not need trimap and background as input. Specifically, guided by a coarse segmentation, Trimap Generation Network (TGN) estimates a trimap where the coarse segmentation can be produced by segmentation/salient object detection/matting approaches, which enables more flexibility for matting to adapt into different scenarios. Then, with an estimated trimap as guidance, our Sampling Propagation Attention Matting Network (SPAMattNet) estimates an alpha matte. Different from previous propagation-based matting networks, inspired by traditional sampling/propagation matting approaches, we propose Sampling Propagation Attention (SPA) for matting network to incorporate sampling and propagation procedures in deep learning based manner for network explainability and performance improvement. It explicitly investigates local spatial and global semantic relationships to reconstruct alpha features. To better harvest sampling/propagation and local/global information, a Cross-Fusion Contextual Module (CFC) is introduced to aggregate features from different sources. Extensive experiments are conducted to show that our matting approach is competitive compared to other state-of-the-art methods in both trimap-free and trimap-needed aspects on several challenging matting benchmarks.

A network score-based metric to optimize the quality assurance of automatic radiotherapy target segmentations

Background and purposeExisting methods for quality assurance of the radiotherapy auto-segmentations focus on the correlation between the average model entropy and the Dice Similarity Coefficient (DSC) only. We identified a metric directly derived from the output of the network and correlated it with clinically relevant metrics for contour accuracy. Materials and MethodsMagnetic Resonance Imaging auto-segmentations were available for the gross tumor volume for cervical cancer brachytherapy (106 segmentations) and for the clinical target volume for rectal cancer external-beam radiotherapy (77 segmentations). The nnU-Net’s output before binarization was taken as a score map. We defined a metric as the mean of the voxels in the score map above a threshold (λ). Comparisons were made with the mean and standard deviation over the score map and with the mean over the entropy map. The DSC, the 95th Hausdorff distance, the mean surface distance (MSD) and the surface DSC were computed for segmentation quality. Correlations between the studied metrics and model quality were assessed with the Pearson correlation coefficient (r). The area under the curve (AUC) was determined for detecting segmentations that require reviewing. ResultsFor both tasks, our metric (λ = 0.30) correlated more strongly with the segmentation quality than the mean over the entropy map (for surface DSC, r > 0.65 vs. r < 0.60). The AUC was above 0.84 for detecting MSD values above 2 mm. ConclusionsOur metric correlated strongly with clinically relevant segmentation metrics and detected segmentations that required reviewing, indicating its potential for automatic quality assurance of radiotherapy target auto-segmentations.

Comparison of AI-Based Auto-Segmentation Quality with Different Daily IGRT Imaging Modalities for Adaptive Radiotherapy Treatment Planning

Modern AI-based auto-segmentation tools may be applied on daily computed tomography (CT) images in an image-guided radiotherapy course to facilitate evaluation of anatomical variations and adaptive treatment planning. However, most AI auto-segmentation models are trained with high-quality diagnostic or planning CT image data. This study aims to evaluate and compare auto-segmentation quality by an AI-based auto-segmentation system with different daily CT imaging modalities. We retrospectively retrieved daily IGRT images for sixty patients. Among them, twenty patients were treated on helical tomotherapy with daily megavoltage CT (MVCT) scans, twenty patients were treated on conventional Linacs with daily cone-beam CT (CBCT) scans, and twenty patients were treated on a biology-guided radiotherapy (BgRT)-capable machine with daily fan-beam kilovoltage CT (kVCT) scans. With each treatment modality, ten patients received daily CT scans in the pelvic region while the other ten received daily CT scans in the thoracic region. An auto-segmentation system using a convolutional neural network algorithm was trained in-house with historical treatment planning CT and contouring data to generate auto-segmentation models for the pelvic and thoracic regions, respectively. Normal organs were first delineated the auto-segmentation system on the daily CT images and then drawn by an experienced planner. A set of metrics including the dice similarity coefficient (DSC), Jaccard similarity index, and Hausdorff distance were used to evaluate the quality of the auto segmentation results compared with manual contours. The auto-segmentation contours on the kVCT images showed higher average DSC compared to those on the MVCT and CBCT images for all the major organs in both the pelvic and thoracic regions including the bladder, rectum, bowel, left and right femurs, esophagus, heart, left and right lung, and spinal cord. With the kVCT images, the average DSC ranged from 0.52±0.22 to 0.996±0.005. In the pelvic region, the largest absolute difference in DSC was observed for the bowel volume with an average DSC of 0.69±0.16, 0.49±0.27, and 0.32±0.25 for the kVCT, MVCT, and CBCT images, respectively (p-value < 0.05 with unpaired t-tests between kVCT and the other two imaging modalities); in the thoracic region, the largest absolute difference in DSC was observed for the esophagus with an average DSC of 0.63±0.16, 0.22±0.08, and 0.15±0.18 for the kVCT, MVCT, and CBCT images, respectively (p-value < 0.05 with unpaired t-tests between kVCT and the other two imaging modalities). Similar results were observed with other metrics. The AI-based auto segmentation system showed improved agreement with manual contouring when using kVCT images from the BgRT capable machine compared to MVCT or CBCT images. However, manual correction is necessary on auto-segmentation results from all imaging modalities especially for organs with limited contrast from surrounding tissues.