Small Segmentation Research Articles

ACTNet: A Dual-Attention Adapter with a CNN-Transformer Network for the Semantic Segmentation of Remote Sensing Imagery

In recent years, the application of semantic segmentation methods based on the remote sensing of images has become increasingly prevalent across a diverse range of domains, including but not limited to forest detection, water body detection, urban rail transportation planning, and building extraction. With the incorporation of the Transformer model into computer vision, the efficacy and accuracy of these algorithms have been significantly enhanced. Nevertheless, the Transformer model’s high computational complexity and dependence on a pre-training weight of large datasets leads to a slow convergence during the training for remote sensing segmentation tasks. Motivated by the success of the adapter module in the field of natural language processing, this paper presents a novel adapter module (ResAttn) for improving the model training speed for remote sensing segmentation. The ResAttn adopts a dual-attention structure in order to capture the interdependencies between sets of features, thereby improving its global modeling capabilities, and introduces a Swin Transformer-like down-sampling method to reduce information loss and retain the original architecture while reducing the resolution. In addition, the existing Transformer model is limited in its ability to capture local high-frequency information, which can lead to an inadequate extraction of edge and texture features. To address these issues, this paper proposes a Local Feature Extractor (LFE) module, which is based on a convolutional neural network (CNN), and incorporates multi-scale feature extraction and residual structure to effectively overcome this limitation. Further, a mask-based segmentation method is employed and a residual-enhanced deformable attention block (Deformer Block) is incorporated to improve the small target segmentation accuracy. Finally, a sufficient number of experiments were performed on the ISPRS Potsdam datasets. The experimental results demonstrate the superior performance of the model described in this paper.

Fine-Grained Guided Model Fusion Network with Attention Mechanism for Infrared Small Target Segmentation

Infrared small target segmentation plays an important role in infrared guidance systems. In this paper, a fine-grained guided model fusion network with attention mechanism (FAMNet) is proposed for improving the performance of the infrared small target segmentation. An autonomous traditional feature extraction algorithm based on information entropy and four-directional gradient contrast is proposed for solving the difficulty of feature extraction from small targets. The deep features are extracted from an improved U-Net. The two kinds of features are fused by channel shuffle, which could enhance the communication efficiency between the channels of the two features. More context information from the target and background is introduced into the network by cross layer parallelization convolutional block attention module (CLPCBAM). Compared with some state-of-the-art techniques, the proposed FAMNet performs significantly better in terms of IoU, nIoU, and F1-score comprehensively.

Adaptive Local Cross-Channel Vector Pooling Attention Module for Semantic Segmentation of Remote Sensing Imagery

Adding an attention module to the deep convolution semantic segmentation network has significantly enhanced the network performance. However, the existing channel attention module focusing on the channel dimension neglects the spatial relationship, causing location noise to transmit to the decoder. In addition, the spatial attention module exemplified by self-attention has a high training cost and challenges in execution efficiency, making it unsuitable to handle large-scale remote sensing data. We propose an efficient vector pooling attention (VPA) module for building the channel and spatial location relationship. The module can locate spatial information better by performing a unique vector average pooling in the vertical and horizontal dimensions of the feature maps. Furthermore, it can also learn the weights directly by using the adaptive local cross-channel interaction. Multiple weight learning ablation studies and comparison experiments with the classical attention modules were conducted by connecting the VPA module to a modified DeepLabV3 network using ResNet50 as the encoder. The results show that the mIoU of our network with the addition of an adaptive local cross-channel interaction VPA module increases by 3% compared to the standard network on the MO-CSSSD. The VPA-based semantic segmentation network can significantly improve precision efficiency compared with other conventional attention networks. Furthermore, the results on the WHU Building dataset present an improvement in IoU and F1-score by 1.69% and 0.97%, respectively. Our network raises the mIoU by 1.24% on the ISPRS Vaihingen dataset. The VPA module can also significantly improve the network’s performance on small target segmentation.

Improving Small Lesion Segmentation in CT Scans using Intensity Distribution Supervision: Application to Small Bowel Carcinoid Tumor.

Finding small lesions is very challenging due to lack of noticeable features, severe class imbalance, as well as the size itself. One approach to improve small lesion segmentation is to reduce the region of interest and inspect it at a higher sensitivity rather than performing it for the entire region. It is usually implemented as sequential or joint segmentation of organ and lesion, which requires additional supervision on organ segmentation. Instead, we propose to utilize an intensity distribution of a target lesion at no additional labeling cost to effectively separate regions where the lesions are possibly located from the background. It is incorporated into network training as an auxiliary task. We applied the proposed method to segmentation of small bowel carcinoid tumors in CT scans. We observed improvements for all metrics (33.5% → 38.2%, 41.3% → 47.8%, 30.0% → 35.9% for the global, per case, and per tumor Dice scores, respectively.) compared to the baseline method, which proves the validity of our idea. Our method can be one option for explicitly incorporating intensity distribution information of a target in network training.

Fully-automated detection of small bowel carcinoid tumors in CT scans using deep learning.

Small bowel carcinoid tumor is a rare neoplasm and increasing in incidence. Patients with small bowel carcinoid tumors often experience long delays in diagnosis due to the vague symptoms, slow growth of tumors, and lack of clinician awareness. Computed tomography (CT) is the most common imaging study for diagnosis of small bowel carcinoid tumor. It is often used with positron emission tomography (PET) to capture anatomical and functional aspects of carcinoid tumors and thus to increase thesensitivity. We compared three different kinds of methods for the automatic detection of small bowel carcinoid tumors on CT scans, which is the first to the best of ourknowledge. Thirty-three preoperative CT scans of 33 unique patients with surgically-proven carcinoid tumors within the small bowel were collected. Ground-truth segmentation of tumors was drawn on CT scans by referring to available 18 F-DOPA PET scans and the corresponding radiology report. These scans were split into the trainval set (n = 24) and the test positive set (n= 9). Additionally, 22 CT scans of 22 unique patients who had no evidence of the tumor were collected to comprise the test negative set. We compared three different kinds of detection methods, which are detection network, patch-based classification, and segmentation-based methods. We also investigated the usefulness of small bowel segmentation for reduction of false positives (FPs) for each method. Free-response receiver operating characteristic (FROC) curves and receiver operating characteristic (ROC) curves were used for lesion- and patient-level evaluations, respectively. Statistical analyses comparing the FROC and ROC curves were alsoperformed. The detection network method performed the best among the compared methods. For lesion-level detection, the detection network method, without the small bowel segmentation-based filtering, achieved sensitivity values of (60.8%, 81.1%, 82.4%, 86.5%) at per-scan FP rates of (1, 2, 4 ,8), respectively. The use of the small bowel segmentation did not improve the performance ( ). For patient-level detection, again the detection network method, but with the small bowel segmentation-based filtering, achieved the highest AUC of 0.86 with a sensitivity of 78% and specificity of 82% at the Youdenpoint. The carcinoid tumors in this patient population were very small and potentially difficult to diagnose. The presented method showed reasonable sensitivity at small numbers of FPs for lesion-level detection. It also achieved a promising AUC for patient-level detection. The method may have clinical application in patients with this rare and difficult to detectdisease.

An underwater small target boundary segmentation method in forward-looking sonar images

Underwater target boundary segmentation is integral to forward-looking sonar image processing. However, small target boundary segmentation is always challenging due to the low resolution, low signal-to-noise ratio, and inhomogeneity of intensity of forward-looking sonar images. This paper proposes an improved level set segmentation method to accurately obtain the contour of small targets in the forward-looking sonar images: morphological reconstruction combined with the level set method (MRLSM). Compared with the classical level set and the level set method combined with the morphological method, MRLSM improves the accuracy and stability for forward-looking sonar image segmentation. Furthermore, the fuzzy C-means, Markov random field, and MRLSM are applied to the numerical simulations and experimental data for comparison. The deep learning methods are also used to compare the performances. The results demonstrated that the proposed method is more accurate, robust, and considerable for underwater small target boundary segmentation.

Deep Learning-Based Semantic Segmentation Methods for Pavement Cracks

As road mileage continues to expand, the number of disasters caused by expanding pavement cracks is increasing. Two main methods, image processing and deep learning, are used to detect these cracks to improve the efficiency and quality of pavement crack segmentation. The classical segmentation network, UNet, has a poor ability to extract target edge information and small target segmentation, and is susceptible to the influence of distracting objects in the environment, thus failing to better segment the tiny cracks on the pavement. To resolve this problem, we propose a U-shaped network, ALP-UNet, which adds an attention module to each encoding layer. In the decoding phase, we incorporated the Laplacian pyramid to make the feature map contain more boundary information. We also propose adding a PAN auxiliary head to provide an additional loss for the backbone to improve the overall network segmentation effect. The experimental results show that the proposed method can effectively reduce the interference of other factors on the pavement and effectively improve the mIou and mPA values compared to the previous methods.

Improved fully convolutional neuron networks on small retinal vessel segmentation using local phase as attention.

Retinal images have been proven significant in diagnosing multiple diseases such as diabetes, glaucoma, and hypertension. Retinal vessel segmentation is crucial for the quantitative analysis of retinal images. However, current methods mainly concentrate on the segmentation performance of overall retinal vessel structures. The small vessels do not receive enough attention due to their small percentage in the full retinal images. Small retinal vessels are much more sensitive to the blood circulation system and have great significance in the early diagnosis and warning of various diseases. This paper combined two unsupervised methods, local phase congruency (LPC) and orientation scores (OS), with a deep learning network based on the U-Net as attention. And we proposed the U-Net using local phase congruency and orientation scores (UN-LPCOS), which showed a remarkable ability to identify and segment small retinal vessels. A new metric called sensitivity on a small ship (Sesv ) was also proposed to evaluate the methods' performance on the small vessel segmentation. Our strategy was validated on both the DRIVE dataset and the data from Maastricht Study and achieved outstanding segmentation performance on both the overall vessel structure and small vessels.

CaraNet: context axial reverse attention network for segmentation of small medical objects.

Segmenting medical images accurately and reliably is important for disease diagnosis and treatment. It is a challenging task because of the wide variety of objects' sizes, shapes, and scanning modalities. Recently, many convolutional neural networks have been designed for segmentation tasks and have achieved great success. Few studies, however, have fully considered the sizes of objects; thus, most demonstrate poor performance for small object segmentation. This can have a significant impact on the early detection of diseases. We propose a context axial reverse attention network (CaraNet) to improve the segmentation performance on small objects compared with several recent state-of-the-art models. CaraNet applies axial reserve attention and channel-wise feature pyramid modules to dig the feature information of small medical objects. We evaluate our model by six different measurement metrics. We test our CaraNet on segmentation datasets for brain tumor (BraTS 2018) and polyp (Kvasir-SEG, CVC-ColonDB, CVC-ClinicDB, CVC-300, and ETIS-LaribPolypDB). Our CaraNet achieves the top-rank mean Dice segmentation accuracy, and results show a distinct advantage of CaraNet in the segmentation of small medical objects. We proposed CaraNet to segment small medical objects and outperform state-of-the-art methods.

Semantics recalibration and detail enhancement network for real‐time semantic segmentation

AbstractReal‐time semantic segmentation is a crucial technology in automatic driving scenarios, which needs to meet both high precision and real‐time. The authors observe that learning complex correlations between object categories is vital in the real‐time semantic segmentation task. Moreover, image spatial detail information plays an important role in small object segmentation and preserving edges and textures. A Semantics Recalibration and Detail Enhancement Network for real‐time semantic segmentation based on BiSeNet V2 is proposed. On the one hand, a lightweight Semantics Recalibration module is designed to effectively extract global semantic contextual information, which combines pyramid segmentation and adaptive recalibration operations to learn the correlations between object categories. On the other hand, a Detail Enhancement module takes the feature maps of the shallow layers in the semantics branch as input and refines the feature maps to highlight the detail information. Finally, quantitative and qualitative analyses on Cityscapes and CamVid datasets demonstrate the effectiveness and generalisation of the proposed method.

A retinal vessel segmentation method based improved U-Net model

There are two problems in retinal blood vessel segmentation, which are the insufficient segmentation of small vessels due to the complex curvature morphology of blood vessels and the segmentation difficulty of blood vessels due to uneven brightness background of lesion fundus images. To solve the problems, a series deformable convolution structure is proposed in this paper, which could improve theadaptive features extraction ability to the blood vessels with various shapes and sizes, enhance feature transmissions and alleviate exploding gradients. On this basis, a retinal vessel segmentation method with series deformable convolution and attention mechanism based on U-Net structure (SDAU-Net) is proposed. In SDAU-Net, the convolution module in U-Net is replaced by series deformable convolution module, the lightweight attention module and dual attention module are applied in the decoder part, which effectively improve the U-Net feature extraction ability for the small vessels with complex morphology and the retinopathy images. To verify the SDAU-Net effect, the comparative experiments are conducted on datasets of DRIVE, STARE, CHASE_DB1 and IOSTAR. The results show that SDAU-Net is superior to comparative methods in accuracy. The Se and Acc on the DRIVE and STARE are 0.8293, 0.9675, 0.8973 and 0.9833 respectively, which indicate that SDAU-Net has more advantages in small vessel segmentation and lesion images. To verify the generalization and extendibility, cross-dataset and cross-modality experiments are conducted on DRIVE, STARE and IOSTAR, the results demonstrate outstanding generalization and extendibility of SDAU-Net.

A Method of Polished Rice Image Segmentation Based on YO-LACTS for Quality Detection

The problem of small and multi-object polished rice image segmentation has always been one of importance and difficulty in the field of image segmentation. In the appearance quality detection of polished rice, image segmentation is a crucial part, directly affecting the results of follow-up physicochemical indicators. To avoid leak detection and inaccuracy in image segmentation qualifying polished rice, this paper proposes a new image segmentation method (YO-LACTS), combining YOLOv5 with YOLACT. We tested the YOLOv5-based object detection network, to extract Regions of Interest (RoI) from the whole image of the polished rice, in order to reduce the image complexity and maximize the target feature difference. We refined the segmentation of the RoI image by establishing the instance segmentation network YOLACT, and we eventually procured the outcome by merging the RoI. Compared to other algorithms based on polished rice datasets, this constructed method was shown to present the image segmentation, enabling researchers to evaluate polished rice satisfactorily.

Infrared Small Target Tracking Algorithm via Segmentation Network and Multistrategy Fusion

To solve the problem of infrared (IR) small target tracking loss or error caused by factors such as scale changes, motion blur, occlusion, etc., this paper proposes a multi-strategy fusion tracking algorithm using an IR small target segmentation network as the detection head, which mainly includes six strategies: target pixel clustering, target feature threshold adjustment, large area search, small area tracking, gate tracking, and coordinate solution. First, candidate targets are obtained through the IR small target segmentation network and pixel clustering strategy. Second, the range of candidate targets is further reduced through threshold adjustment strategies. Then, real-time tracking of IR small targets is achieved through large area search, small area tracking, and wave gate tracking strategies. Finally, the longitude, latitude, and altitude of the tracked target are obtained through coordinate calculation strategies. Both qualitative and quantitative experiments based on real IR small target sequences verify that our algorithm can achieve more satisfactory performances in terms of success rate, precision, and robustness compared with other typical visual trackers. In addition, we have deployed tracking algorithms on the Orange Pi 5 embedded platform, and the tracking speed meets the real-time requirements.

Shape-Aware 3D Small Vessel Segmentation with Local Contrast Guided Attention.

The automated segmentation and analysis of small vessels from in vivo imaging data is an important task for many clinical applications. While current filtering and learning methods have achieved good performance on the segmentation of large vessels, they are sub-optimal for small vessel detection due to their apparent geometric irregularity and weak contrast given the relatively limited resolution of existing imaging techniques. In addition, for supervised learning approaches, the acquisition of accurate pixel-wise annotations in these small vascular regions heavily relies on skilled experts. In this work, we propose a novel self-supervised network to tackle these challenges and improve the detection of small vessels from 3D imaging data. First, our network maximizes a novel shape-aware flux-based measure to enhance the estimation of small vasculature with non-circular and irregular appearances. Then, we develop novel local contrast guided attention(LCA) and enhancement(LCE) modules to boost the vesselness responses of vascular regions of low contrast. In our experiments, we compare with four filtering-based methods and a state-of-the-art self-supervised deep learning method in multiple 3D datasets to demonstrate that our method achieves significant improvement in all datasets. Further analysis and ablation studies have also been performed to assess the contributions of various modules to the improved performance in 3D small vessel segmentation. Our code is available at https://github.com/dengchihwei/LCNetVesselSeg.

MIC-Net: multi-scale integrated context network for automatic retinal vessel segmentation in fundus image.

Accurate retinal vessel segmentation is of great value in the auxiliary screening of various diseases. However, due to the low contrast between the ends of the branches of the fundus blood vessels and the background, and the variable morphology of the optic disc and cup in the retinal image, the task of high-precision retinal blood vessel segmentation still faces difficulties. This paper proposes a multi-scale integrated context network, MIC-Net, which fully fuses the encoder-decoder features, and extracts multi-scale information. First, a hybrid stride sampling (HSS) block was designed in the encoder to minimize the loss of helpful information caused by the downsampling operation. Second, a dense hybrid dilated convolution (DHDC) was employed in the connection layer. On the premise of preserving feature resolution, it can perceive richer contextual information. Third, a squeeze-and-excitation with residual connections (SERC) was introduced in the decoder to adjust the channel attention adaptively. Finally, we utilized a multi-layer feature fusion mechanism in the skip connection part, which enables the network to consider both low-level details and high-level semantic information. We evaluated the proposed method on three public datasets DRIVE, STARE and CHASE. In the experimental results, the Area under the receiver operating characteristic (ROC) and the accuracy rate (Acc) achieved high performances of 98.62%/97.02%, 98.60%/97.76% and 98.73%/97.38%, respectively. Experimental results show that the proposed method can obtain comparable segmentation performance compared with the state-of-the-art (SOTA) methods. Specifically, the proposed method can effectively reduce the small blood vessel segmentation error, thus proving it a promising tool for auxiliary diagnosis of ophthalmic diseases.

LW-IRSTNet: Lightweight Infrared Small Target Segmentation Network and Application Deployment

LW-IRSTNet: Lightweight Infrared Small Target Segmentation Network and Application Deployment

A Novel Lightweight Swin-Unet Network for Semantic Segmentation of COVID-19 Lesion in CT Images

The Corona Virus Disease 2019 (COVID-19) is highly infectious, has been spread worldwide, caused a global pandemic, and seriously endangered human health and life. The most effective methods for halting and stopping the transmission of the Corona Virus include early detection, quarantine, and successful treatment. Because it exhibits significant imaging characteristics for COVID-19 lesions in chest computed tomography (CT), it can be used to diagnose COVID-19. Aiming at the inaccuracies of uneven gray distribution, irregular regions, multi-scale, and multi-region segmentation in COVID-19 CT images. This paper proposed a novel Swin-Unet network to improve the accuracy of multi-scale lesion segmentation in COVID-19 CT images. First, in the double-layer Swin Transformer blocks of the Swin-Unet, a residual multi-layer perceptron (ResMLP) module was introduced and replaced the multi-layer perceptron (MLP) module to reduce the loss of features during the transmission process, thereby improving the segmentation precision of multi-scale lesion areas. Second, the uncertain region inpainting module (URIM) was added after Linear Projection, which can refine the uncertain regions in the segmentation features map, thereby improving the segmentation accuracy of different lesion regions. Third, a new loss function DF was designed. It can effectively improve the small target segmentation effect and thus improve the multi-scale segmentation result. Finally, the proposed method was compared to other methods on the public dataset. The Dice, Precision, Recall, and IOU of the proposed method are 0.812, 0.780, 0.848, and 0.683, respectively, which are better than the other models. Moreover, our model has fewer parameters and faster reasoning speed. The proposed method achieves excellent segmentation results for multi-scale and multi-region lesions, and it will be more beneficial in aiding COVID-19 diagnosis and treatment.

Retinal vessel segmentation based on multi-scale feature and style transfer.

Retinal vessel segmentation is very important for diagnosing and treating certain eye diseases. Recently, many deep learning-based retinal vessel segmentation methods have been proposed; however, there are still many shortcomings (e.g., they cannot obtain satisfactory results when dealing with cross-domain data or segmenting small blood vessels). To alleviate these problems and avoid overly complex models, we propose a novel network based on a multi-scale feature and style transfer (MSFST-NET) for retinal vessel segmentation. Specifically, we first construct a lightweight segmentation module named MSF-Net, which introduces the selective kernel (SK) module to increase the multi-scale feature extraction ability of the model to achieve improved small blood vessel segmentation. Then, to alleviate the problem of model performance degradation when segmenting cross-domain datasets, we propose a style transfer module and a pseudo-label learning strategy. The style transfer module is used to reduce the style difference between the source domain image and the target domain image to improve the segmentation performance for the target domain image. The pseudo-label learning strategy is designed to be combined with the style transfer module to further boost the generalization ability of the model. Moreover, we trained and tested our proposed MSFST-NET in experiments on the DRIVE and CHASE_DB1 datasets. The experimental results demonstrate that MSFST-NET can effectively improve the generalization ability of the model on cross-domain datasets and achieve improved retinal vessel segmentation results than other state-of-the-art methods.

Research on infrared small target segmentation algorithm based on improved mask R-CNN

The task of segmenting small infrared targets, which have few pixels and weak features, has been a difficult problem in the field of small target image processing. Small targets not only exist in general images, but also widely exist in drone cameras, communication base station cameras, rescue cameras, and vehicle cameras. Research on small target segmentation algorithm is very important for analyzing and utilizing these images, and it also has important application value in security, traffic, and rescue. Traditional small target segmentation algorithms are able to segment objects with simple target contour edges and large differences in signal strength. The traditional algorithm often has a high false detection rate and missed detection rate when facing several targets with weak signal strength. It does not perform well in complex scenes. In this paper, we introduce an infrared small target segmentation scheme facing multiple types and numbers of targets. This paper also produces an infrared UAV and pedestrian dataset for validation. Experiments show that the optimized algorithm has a better detection effect on targets in complex scenes.

Liver Lesion Segmentation Using Deep Learning Models

An estimated 9.6 million deaths, or one in every six deaths, were attributed to cancer in 2018, making it the second highest cause of death worldwide. Men are more likely to develop lung, prostate, colorectal, stomach, and liver cancer than women, who are more likely to develop breast, colorectal, lung, cervical, and thyroid cancer. The primary goals of medical image segmentation include studying anatomical structure, identifying regions of interest (RoI), and measuring tissue volume to track tumor growth. It is crucial to diagnose and treat liver lesions quickly in order to stop the tumor from spreading further. Deep learning model-based liver segmentation has become very popular in the field of medical image analysis. This study explores various deep learning-based liver lesion segmentation algorithms and methodologies. Based on the developed models, the performance, and their limitations of these methodologies are contrasted. In the end, it was concluded that small size lesion segmentation, in particular, is still an open research subject for computer-aided systems of liver lesion segmentation, for there are still a number of technical issues that need to be resolved.