Small Segmentation Research Articles

Segmenting Individual Tree from TLS Point Clouds Using Improved DBSCAN

Terrestrial laser scanning (TLS) can provide accurate and detailed three-dimensional (3D) structure information of the forest understory. Segmenting individual trees from disordered, discrete, and high-density TLS point clouds is the premise for obtaining accurate individual tree structure parameters of forest understory, pest control and fine modeling. In this study, we propose a bottom-up method to segment individual trees from TLS forest data based on density-based spatial clustering of applications with noise (DBSCAN). In addition, we also improve the DBSCAN based on the distance distribution matrix (DDM) to automatically and adaptively determine the optimal cluster number and the corresponding input parameters. Firstly, the proposed method is based on the improved DBSCAN to detect the trunks and obtain the initial clustering results. Then, the Hough circle fitting method is used to modify the trunk detection results. Finally, individual tree segmentation is realized based on regional growth layer-by-layer clustering. In this paper, we use TLS multi-station scanning data from Chinese artificial forest and German mixed forest, and then evaluate the efficiency of the method from three aspects: overall segmentation, trunk detection and small tree segmentation. Furthermore, the proposed method is compared with three existing individual tree segmentation methods. The results show that the total recall, precision, and F1-score of the proposed method are 90.84%, 95.38% and 0.93, respectively. Compared with traditional DBSCAN, recall, accuracy and F1-score are increased by 6.96%, 4.14% and 0.06, respectively. The individual tree segmentation result of the proposed method is comparable to those of the existing methods, and the optimal parameters can be automatically extracted and the small trees under tall trees can be accurately segmented.

AWSnet: An auto-weighted supervision attention network for myocardial scar and edema segmentation in multi-sequence cardiac magnetic resonance images.

Multi-sequence cardiac magnetic resonance (CMR) provides essential pathology information (scar and edema) to diagnose myocardial infarction. However, automatic pathology segmentation can be challenging due to the difficulty of effectively exploring the underlying information from the multi-sequence CMR data. This paper aims to tackle the scar and edema segmentation from multi-sequence CMR with a novel auto-weighted supervision framework, where the interactions among different supervised layers are explored under a task-specific objective using reinforcement learning. Furthermore, we design a coarse-to-fine framework to boost the small myocardial pathology region segmentation with shape prior knowledge. The coarse segmentation model identifies the left ventricle myocardial structure as a shape prior, while the fine segmentation model integrates a pixel-wise attention strategy with an auto-weighted supervision model to learn and extract salient pathological structures from the multi-sequence CMR data. Extensive experimental results on a publicly available dataset from Myocardial pathology segmentation combining multi-sequence CMR (MyoPS 2020) demonstrate our method can achieve promising performance compared with other state-of-the-art methods. Our method is promising in advancing the myocardial pathology assessment on multi-sequence CMR data. To motivate the community, we have made our code publicly available via https://github.com/soleilssss/AWSnet/tree/master.

A U-snake based deep learning network for right ventricle segmentation.

Ventricular segmentation is of great importance for the heart condition monitoring. However, manual segmentation is time-consuming, cumbersome, and subjective. Many segmentation methods perform poorly due to the complex structure and uncertain shape of the right ventricle, so we combine deep learning to achieve automatic segmentation. This paper proposed a method named U-Snake network which is based on the improvement of deep snake together with level set to segment the right ventricular in the MR images. U-snake aggregates the information of each receptive field which is learned by circular convolution of multiple dilation rates. At the same time, we also added dice loss functions and transferred the result of U-Snake to the level set so as to further enhance the effect of small object segmentation. Our method is tested on the test 1 and test 2 datasets in the right ventricular segmentation challenge (RVSC), which shows the effectiveness. The experiment showed that we have obtained good result in the RVSC. The highest segmentation accuracy on the right ventricular test set 2 reached a dice coefficient of 0.911, and the segmentation speed reached 5 fps. Our method, a new deep learning network named U-snake, has surpassed the previous excellent ventricular segmentation method based on mathematical theory and other classical deep learning methods, such as Residual U-net, Inception CNN, and Dilated CNN. However, it can only be used as an auxiliary tool instead of replacing the work of human beings.

Real-Time Tracking of Object Melting Based on Enhanced DeepLab v3+ Network.

In order to reveal the dissolution behavior of iron tailings in blast furnace slag, the main component of iron tailings, SiO2, was used for research. Aiming at the problem of information loss and inaccurate extraction of tracking molten SiO2 particles in high temperature, a method based on the improved DeepLab v3+ network was proposed to track, segment, and extract small object particles in real time. First, by improving the decoding layer of the DeepLab v3+ network, construct dense ASPP (atrous spatial pyramid pooling) modules with different dilation rates to optimize feature extraction, increase the shallow convolution of the backbone network, and merge it into the upper convolution decoding part to increase detailed capture. Secondly, integrate the lightweight network MobileNet v3 to reduce network parameters, further speed up image detection, and reduce the memory usage to achieve real-time image segmentation and adapt to low-level configuration hardware. Finally, improve the expression of the loss function for the binary classification model of small object in this paper, combining the advantages of the Dice Loss binary classification segmentation and the Focal Loss balance of positive and negative samples, solving the problem of unbalanced dataset caused by the small proportion of positive samples. Experimental results show that MIoU (mean intersection over union) of the proposed model for small object segmentation is 6% higher than that of the original model, the overall MIoU is increased by 3%, and the execution time and memory consumption are only half of the original model, which can be well applied to real-time tracking and segmentation of small particles.

GRAPH-BASED SMALL BOWEL PATH TRACKING WITH CYLINDRICAL CONSTRAINTS.

We present a new graph-based method for small bowel path tracking based on cylindrical constraints. A distinctive characteristic of the small bowel compared to other organs is the contact between parts of itself along its course, which makes the path tracking difficult together with the indistinct appearance of the wall. It causes the tracked path to easily cross over the walls when relying on low-level features like the wall detection. To circumvent this, a series of cylinders that are fitted along the course of the small bowel are used to guide the tracking to more reliable directions. It is implemented as soft constraints using a new cost function. The proposed method is evaluated against ground-truth paths that are all connected from start to end of the small bowel for 10 abdominal CT scans. The proposed method showed clear improvements compared to the baseline method in tracking the path without making an error. Improvements of 6.6% and 17.0%, in terms of the tracked length, were observed for two different settings related to the small bowel segmentation.

MR‐UNet: An UNet model using multi‐scale and residual convolutions for retinal vessel segmentation

AbstractThe overall performance of the retinal vessel segmentation network based on improved UNet is excellent, but there is still room for improvement in the small blood vessel segmentation. Therefore, this paper proposes an improved MR‐UNet, which designs two new blocks: the multi‐scale convolution (Multiconv) block and the residual convolution (Resconv) block. The Multiconv block uses different size convolution kernels to extract the characteristics of different thicknesses of retinal blood vessels, thereby improving the model's ability to segment small blood vessels. The Resconv block uses different convolutional layers to process the shallow semantic information in the encoding stage and then concatenates it to the decoding stage, reducing the semantic difference between the encoder and the decoder. On the retinal data sets DRIVE, STARE and CHASE_DB1, the accuracy (Acc) of this model is 0.9705, 0.9747, 0.9778, the specificity (Sp) is 0.9863, 0.9892, 0.9930, the AUC is 0.9872, 0.9849, 0.9925, and the F1‐Score is 0.8270, 0.8134, 0.8460, respectively. Compared with the original UNet, Se, Sp, and F1‐Score of MR‐UNet increase by 2.5%, 0.5%, and 0.4%, respectively, proving that MR‐UNet has better comprehensive performance.

Dual-Branch Network With Dual-Sampling Modulated Dice Loss for Hard Exudate Segmentation in Color Fundus Images

Automated segmentation of hard exudates in colour fundus images is a challenge task due to issues of extreme class imbalance and enormous size variation. This paper aims to tackle these issues and proposes a dual-branch network with dual-sampling modulated Dice loss. It consists of two branches: large hard exudate biased segmentation branch and small hard exudate biased segmentation branch. Both of them are responsible for their own duties separately. Furthermore, we propose a dual-sampling modulated Dice loss for the training such that our proposed dual-branch network is able to segment hard exudates in different sizes. In detail, for the first branch, we use a uniform sampler to sample pixels from predicted segmentation mask for Dice loss calculation, which leads to this branch naturally be biased in favour of large hard exudates as Dice loss generates larger cost on misidentification of large hard exudates than small hard exudates. For the second branch, we use a re-balanced sampler to oversample hard exudate pixels and undersample background pixels for loss calculation. In this way, cost on misidentification of small hard exudates is enlarged, which enforces the parameters in the second branch fit small hard exudates well. Considering that large hard exudates are much easier to be correctly identified than small hard exudates, we propose an easy-to-difficult learning strategy by adaptively modulating the losses of two branches. We evaluate our proposed method on two public datasets and the results demonstrate that ours achieves state-of-the-art performance.

SAUNet++: an automatic segmentation model of COVID-19 lesion from CT slices

The coronavirus disease 2019 (COVID-19) epidemic has spread worldwide and the healthcare system is in crisis. Accurate, automated and rapid segmentation of COVID-19 lesion in computed tomography (CT) images can help doctors diagnose and provide prognostic information. However, the variety of lesions and small regions of early lesion complicate their segmentation. To solve these problems, we propose a new SAUNet++ model with squeeze excitation residual (SER) module and atrous spatial pyramid pooling (ASPP) module. The SER module can assign more weights to more important channels and mitigate the problem of gradient disappearance; the ASPP module can obtain context information by atrous convolution using various sampling rates. In addition, the generalized dice loss (GDL) can reduce the correlation between lesion size and dice loss, and is introduced to solve the problem of small regions segmentation of COVID-19 lesion. We collected multinational CT scan data from China, Italy and Russia and conducted extensive comparative and ablation studies. The experimental results demonstrated that our method outperforms state-of-the-art models and can effectively improve the accuracy of COVID-19 lesion segmentation on the dice similarity coefficient (our: 87.38 $$\%$$ vs. U-Net++: 84.25 $$\%$$ ), sensitivity (our: 93.28 $$\%$$ vs. U-Net++: 89.85 $$\%$$ ) and Hausdorff distance (our: 19.99 mm vs. U-Net++: 26.79 mm), respectively.

A Multi-Task Framework for Infrared Small Target Detection and Segmentation

Due to the complicated background and noise of infrared images, infrared small target detection is one of the most difficult problems in the field of computer vision. In most existing studies, semantic segmentation methods are typically used to achieve better results. The centroid of each target is calculated from the segmentation map as the detection result. In contrast, we propose a novel end-to-end framework for infrared small target detection and segmentation in this paper. First, with the use of UNet as the backbone to maintain resolution and semantic information, our model can achieve a higher detection accuracy than other state-of-the-art methods by attaching a simple anchor-free head. Then, a pyramid pool module is used to further extract features and improve the precision of target segmentation. Next, we use semantic segmentation tasks that pay more attention to pixel-level features to assist in the training process of object detection, which increases the average precision and allows the model to detect some targets that were previously not detectable. Furthermore, we develop a multi-task framework for infrared small target detection and segmentation. Our multi-task learning model reduces complexity by nearly half and speeds up inference by nearly twice compared to the composite single-task model, while maintaining accuracy. The code and models are publicly available at https://github.com/Chenastron/MTUNet.

Deep Siamese Semantic Segmentation Network for PCB Welding Defect Detection

Deep learning has been widely used in recent years for printed circuit board (PCB) defect detection because of its excellent performance. However, deep-learning-based approaches often suffer from the over-fitting problem due to the lack of sufficient training data in real applications. Meanwhile, these approaches still have some challenges to detect these defects with small sizes and irregular shapes. To address these problems, this article has developed a novel deep Siamese semantic segmentation network which integrates the similarity measurement of the Siamese network with the encoder–decoder semantic segmentation network for PCB welding defect detection. This network includes two encoders sharing weighted values, a decoder, and some correlation modules, in which the decoder integrates deep features from two decoders with their feature difference computed by some correlation modules via skipping connections to recover spatial information on multiple output layers, and thus this proposed network can perform PCB welding small defect semantic segmentation. Moreover, via these correlation modules, this proposed network can pay more attention to semantic difference and further alleviate the over-fitting issue because of insufficient defect samples. Finally, we propose a combined loss function which combines the weighted cross-entropy loss, the Lovasz softmax loss, and the weighted precision–recall loss for network training to further improve small defect segmentation and recall improvement. Experimental results demonstrate that the proposed network can be trained on limited training images and achieve high efficiency and outstanding effects for PCB welding small defect segmentation.

PixelGame: Infrared Small Target Segmentation as a Nash Equilibrium

A key challenge of infrared small target segmentation (ISTS) is to balance false negative pixels (FNs) and false positive pixels (FPs). Traditional methods combine FNs and FPs into a single objective by weighted sum, and the optimization process is decided by one actor. Minimizing FNs and FPs with the same strategy leads to antagonistic decisions. To address this problem, we propose a competitive game framework (pixelGame) from a novel perspective for ISTS. In pixelGame, FNs and FPs are controlled by different player whose goal is to minimize their own utility function. FNs-player and FPs-player are designed with different strategies: One is to minimize FNs and the other is to minimize FPs. The utility function drives the evolution of the two participants in competition. We consider the Nash equilibrium of pixelGame as the optimal solution. In addition, we propose maximum information modulation (MIM) to highlight the target information. MIM effectively focuses on the salient region including small targets. Extensive experiments on two standard public datasets prove the effectiveness of our method. Compared with other state-of-the-art methods, our method achieves better performance in terms of F1-measure ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathrm{F_{1}}$</tex-math></inline-formula> ) and the intersection of union ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{IoU}$</tex-math></inline-formula> ).

FactSeg: Foreground Activation-Driven Small Object Semantic Segmentation in Large-Scale Remote Sensing Imagery

The small object semantic segmentation task is aimed at automatically extracting key objects from high-resolution remote sensing (HRS) imagery. Compared with the large-scale coverage areas for remote sensing imagery, the key objects, such as cars and ships, in HRS imagery often contain only a few pixels. In this article, to tackle this problem, the foreground activation (FA)-driven small object semantic segmentation (FactSeg) framework is proposed from perspectives of structure and optimization. In the structure design, FA object representation is proposed to enhance the awareness of the weak features in small objects. The FA object representation framework is made up of a dual-branch decoder and collaborative probability (CP) loss. In the dual-branch decoder, the FA branch is designed to activate the small object features (activation) and suppress the large-scale background, and the semantic refinement (SR) branch is designed to further distinguish small objects (refinement). The CP loss is proposed to effectively combine the activation and refinement outputs of the decoder under the CP hypothesis. During the collaboration, the weak features of the small objects are enhanced with the activation output, and the refined output can be viewed as the refinement of the binary outputs. In the optimization stage, small object mining (SOM)-based network optimization is applied to automatically select effective samples and refine the direction of the optimization while addressing the imbalanced sample problem between the small objects and the large-scale background. The experimental results obtained with two benchmark HRS imagery segmentation datasets demonstrate that the proposed framework outperforms the state-of-the-art semantic segmentation methods and achieves a good tradeoff between accuracy and efficiency. Code will be available at: <uri>http://rsidea.whu.edu.cn/FactSeg.htm</uri>

Hilbertscnet Self-Attention Networks for Small Target Segmentation of Aerial Drone Images

Hilbertscnet Self-Attention Networks for Small Target Segmentation of Aerial Drone Images

Deep Reinforcement Learning for Small Bowel Path Tracking using Different Types of Annotations.

Small bowel path tracking is a challenging problem considering its many folds and contact along its course. For the same reason, it is very costly to achieve the ground-truth (GT) path of the small bowel in 3D. In this work, we propose to train a deep reinforcement learning tracker using datasets with different types of annotations. Specifically, we utilize CT scans that have only GT small bowel segmentation as well as ones with the GT path. It is enabled by designing a unique environment that is compatible for both, including a reward definable even without the GT path. The performed experiments proved the validity of the proposed method. The proposed method holds a high degree of usability in this problem by being able to utilize the scans with weak annotations, and thus by possibly reducing the required annotation cost.

LFG-Net: Low-Level Feature Guided Network for Precise Ship Instance Segmentation in SAR Images

Ship instance segmentation of high-resolution SAR images is a valuable and challenging task due to the complex scattering and noise properties. In this article, we pioneered the construction of the low-level feature to discriminate the ships and complemented the super-resolution denoising techniques in the network modules, termed low-level feature guided network (LFG-Net), for precise ship instance segmentation in SAR images. LFG-Net consists of the low-level feature concerned pyramid (LFCP), the high-resolution interaction module (HR-FIM), and the compression recovery segmentation branch (CRSB). LFCP extends vanilla FPN with the P₁ layer and complements super-resolution techniques to capture the regional and texture information at the image level for small object segmentation. HR-FIM interacts the bounding box region of interest (RoI) feature and mask RoI feature at the instance level with high-resolution techniques to enhance the mask RoI feature. CRSB aims at recovering the high-resolution mask predictions to improve the ship segmentation performance. Comprehensive experiments on HRSID, PSeg-SSDD, and AirSARShip indicate that LFG-Net* achieves 11.7%, 6.3%, and 12.7% AP increments compared with the Mask R-CNN baseline, respectively. Besides, it receives 9.5%, 4.9%, and 7.3% AP increments compared with state-of-the-art method, respectively, which bridges the gap of instance segmentation precision in SAR images. In terms of the visualized instance segmentation results, LFG-Net* is capable of segmenting the complex scenes, e.g, the adjacent distributed ships and ships with strong reflection noise interference, in SAR images. Code is available at: https://github.com/Evarray/LFG-Net.

Explored Normalized Cut With Random Walk Refining Term for Image Segmentation.

The Normalized Cut (NCut) model is a popular graph-based model for image segmentation. But it suffers from the excessive normalization problem and weakens the small object and twig segmentation. In this paper, we propose an Explored Normalized Cut (ENCut) model that establishes a balance graph model by adopting a meaningful-loop and a k-step random walk, which reduces the energy of small salient region, so as to enhance the small object segmentation. To improve the twig segmentation, our ENCut model is further enhanced by a new Random Walk Refining Term (RWRT) that adds local attention to our model with the help of an un-supervising random walk. Finally, a move-making based strategy is developed to efficiently solve the ENCut model with RWRT. Experiments on three standard datasets indicate that our model can achieve state-of-the-art results among the NCut-based segmentation models.

Infrared Detection of Small-Moving Targets Using Spatial Local Vector Difference and Temporal Sample Consensus Measures

Detection small moving targets in infrared sequences with accuracy and low computation time is challenging work in infrared search and tracking systems. A frequent approach for this task is to strengthen small weak targets and diminish the clutter of background. However, the background and small targets’ pixel values are close to each other and thus only a few background suppressing models are suitable to infrared small target segmentation. To efficiently resolve the problem that most of the classic approaches cannot manage low signal to noise ratio and weak objects without details, a novel spatial-temporal local difference measure is introduced for moving objects segmentation in infrared sequences. First of all, to strengthen targets, a new local vector dissimilarity measure is employed to demonstrate the difference between the weak target and their surrounding backgrounds and calculate the spatial saliency feature figure. Then the local sample consensus of succession images is used to compute the temporal varying feature figure. Afterward, the combined saliency feature map is measured by taking spatial and temporal feature images into account. Finally, the small moving targets are segmented employing an adaptive threshold approach. Abundant quantitative and qualitative experimental results have demonstrated that the introduced approach is remarkable and has a superior accuracy in terms of performance on both public and real datasets in comparison to the state-of-the-art spatial and temporal models.

Learning multi-level structural information for small organ segmentation

Deep neural networks have achieved great success in medical image segmentation problems such as liver, kidney, the accuracy of which already exceeds the human level. However, small organ segmentation (e.g., pancreas) is still a challenging task. To tackle such problems, extracting and aggregating multi-scale robust features become essentially important. In this paper, we develop a multi-level structural loss by integrating the region, boundary, and pixel-wise information to supervise feature fusion and precise segmentation. The novel pixel-wise term can provide information complementary to the region and boundary loss, which helps to discover more local information from the image. We further develop a multi-branch network with a saliency guidance module to better aggregate the three levels of features. The coarse-to-fine segmentation architecture is adopted to use the prediction on the coarse stage to obtain the bounding box for the fine stage. Comprehensive evaluations are performed on three benchmark datasets, i.e., the NIH pancreas, ISICDM pancreas, and MSD spleen dataset, showing that our models can achieve significant increases in segmentation accuracy compared to several state-of-the-art pancreas and spleen segmentation methods. Furthermore, the ablation study demonstrates the multi-level structural features help both the training stability and the convergence of the coarse-to-fine approach.

AugMS-Net:Augmented multiscale network for small cervical tumor segmentation from MRI volumes

Cervical cancer is one of the leading causes of female-specific cancer death. Tumor region segmentation plays a pivotal role in both the clinical analysis and treatment planning of cervical cancer. Due to the heterogeneity and low contrast of biomedical images, current state-of-the-art tumor segmentation approaches are facing the challenge of the insensitive detection of small lesion regions. To tackle this problem, this paper proposes an augmented multiscale network (AugMS-Net) based on 3D U-Net to automatically segment cervical Magnetic Resonance Imaging (MRI) volumes. Since a multiscale strategy is considered one of the most promising algorithms to tackle small object recognition, we introduce a novel 3D module to explore more granular multiscale representations. Besides, we employ a deep multiscale supervision strategy to doubly supervise the side outputs hierarchically. To demonstrate the generalization of our model, we evaluated AugMS-Net on both a cervical dataset from MRI volumes and a liver dataset from Computerized Tomography (CT) volumes. Our proposed AugMS-Net shows superior performance over baseline models, yielding high accuracy while reducing the number of model parameters by nearly 20%. The source code and trained models are available at https://github.com/Cassieyy/AugMS-Net.

Blood Vessel Segmentation Based on the 3D Residual U-Net

In this paper, we propose blood vessel segmentation based on the 3D residual U-Net method. First, we integrate the residual block structure into the 3D U-Net. By exploring the influence of adding residual blocks at different positions in the 3D U-Net, we establish a novel and effective 3D residual U-Net. In addition, to address the challenges of pixel imbalance, vessel boundary segmentation, and small vessel segmentation, we develop a new weighted Dice loss function with a better effect than the weighted cross-entropy loss function. When training the model, we adopted a two-stage method from coarse-to-fine. In the fine stage, a local segmentation method of 3D sliding window is added. In the model testing phase, we used the 3D fixed-point method. Furthermore, we employ the 3D morphological closed operation to smooth the surfaces of vessels and volume analysis to remove noise blocks. To verify the accuracy and stability of our method, we compare our method with FCN, 3D DenseNet, and 3D U-Net. The experimental results indicate that our method has higher accuracy and better stability than the other studied methods and that the average Dice coefficients for hepatic veins and portal veins reach 71.7% and 76.5% in the coarse stage and 72.5% and 77.2% in the fine stage, respectively. In order to verify the robustness of the model, we conducted the same comparative experiment on the brain vessel datasets, and the average Dice coefficient reached 87.2%.