Object Segmentation Research Articles

MVSRF: Point cloud semantic segmentation and optimization method for granular construction objects

MVSRF: Point cloud semantic segmentation and optimization method for granular construction objects

Promoting camouflaged object detection through novel edge-target interaction and frequency-spatial fusion

The realm of camouflage object detection endeavors to precisely discern concealed targets that blend seamlessly into their surroundings. Recently, numerous research endeavors have converged on unraveling the disguise tactics employed by such objects, striving for efficient and accurate detection results. While these methods have garnered notable advancements in unveiling camouflaged targets, they still grapple with challenges posed by the high similarity between objects and their backgrounds, resulting in misidentifications, detection oversights, and the loss of intricate details. In light of these challenges, this paper introduces an innovative EFNet, aimed at elevating the integrity of target recognition and the finesse of detail capture through deepened interactions between Edge-target features and Frequency-spatial information. Notably, recognizing that the edges of camouflaged objects harbor a wealth of intrinsic details and serve as crucial delimiters distinguishing them from their surroundings, we have devised a dual-path architecture, where an auxiliary Edge Detection (ED) branch collaborates seamlessly with the primary Object Segmentation (OS) branch. Within the OS branch, we meticulously crafted the Edge-induced Segmentation Refinement module (ESR), which ingeniously harnesses the refined edge information provided by the ED branch to bolster target segmentation accuracy and augment detail fidelity. Moreover, in the supporting ED branch, we propose the Structure-induced Edge Enhancement module (SEE), designed to accentuate edge information while suppressing irrelevant noise with the assistance of the OS branch, thereby ensuring the precise extraction of edge information. To further enrich the information on dual-path, we innovatively introduce a Frequency Decomposition Module (FDM), which decomposes images into low- and high-frequency components, tailored to enhance target segmentation and edge detection, respectively. Additionally, to ensure seamless integration of frequency-domain and spatial-domain information, we devised the versatile Frequency-Spatial Domain Mixer (FSDM), achieving precise information alignment and fusion of these two domains. Quantitative and qualitative experimental results demonstrate that our proposed EFNet significantly outperforms existing state-of-the-art methods on four widely used benchmark datasets.

LBSNet: Lightweight Joint Boundary Detection and Semantic Segmentation for Transparent and Reflective Objects

LBSNet: Lightweight Joint Boundary Detection and Semantic Segmentation for Transparent and Reflective Objects

Using the Mask R-CNN model for segmentation of real estate objects in aerial photographs

Using the Mask R-CNN model for segmentation of real estate objects in aerial photographs

GCNet: A Deep Learning Framework for Enhanced Grape Cluster Segmentation and Yield Estimation Incorporating Occluded Grape Detection with a Correction Factor for Indoor Experimentation

Object segmentation algorithms have heavily relied on deep learning techniques to estimate the count of grapes which is a strong indicator for the yield success of grapes. The issue with using object segmentation algorithms for grape analytics is that they are limited to counting only the visible grapes, thus omitting hidden grapes, which affect the true estimate of grape yield. Many grapes are occluded because of either the compactness of the grape bunch cluster or due to canopy interference. This introduces the need for models to be able to estimate the unseen berries to give a more accurate estimate of the grape yield by improving grape cluster segmentation. We propose the Grape Counting Network (GCNet), a novel framework for grape cluster segmentation, integrating deep learning techniques with correction factors to address challenges in indoor yield estimation. GCNet incorporates occlusion adjustments, enhancing segmentation accuracy even under conditions of foliage and cluster compactness, and setting new standards in agricultural indoor imaging analysis. This approach improves yield estimation accuracy, achieving a R² of 0.96 and reducing mean absolute error (MAE) by 10% compared to previous methods. We also propose a new dataset called GrapeSet which contains visible imagery of grape clusters imaged indoors, along with their ground truth mask, total grape count, and weight in grams. The proposed framework aims to encourage future research in determining which features of grapes can be leveraged to estimate the correct grape yield count, equip grape harvesters with the knowledge of early yield estimation, and produce accurate results in object segmentation algorithms for grape analytics.

Comparing Above and Underwater Visibility of Fluorescent Quinine-Based Liquid and Solid Tracers to Estimate Suspended Sediment Concentrations Under Low Luminosity Conditions

While exploring the use and visibility of fluorescent tracers for estimating flow velocities under varying water turbidity, this study introduces a technique for estimating suspended sediment concentration (SSC) in water. Laboratory and field experiments were conducted using a dual above/underwater optical camera monitoring system to measure and compare the visibility (brightness intensity) of fluorescent tracers applied into water under varying SSC loads. Based on the fluorescent properties of quinine under low luminosity conditions, a quinine solution, in both liquid and solid (ice cube) states, was applied to the water surface. Its visibility was recorded using the monitoring system, and brightness intensity was quantified using image processing techniques such as grayscale conversion, object segmentation, and pixel intensity extraction. Results indicate that the tracer brightness intensity decreases as SSC loads increase, with the underwater camera recording lower visibility than the above-water camera, particularly at higher SSC loads. Empirical regression equations were developed to describe the relationship between the tracer brightness intensity recorded by the underwater camera and the SSC loads. These findings suggest the ability of the presented monitoring system, using quinine solution in both liquid and solid states, along with an underwater camera, to estimate a relevant range of SSC loads under low luminosity conditions.

A semi-supervised fracture-attention model for segmenting tubular objects with improved topological connectivity.

Ensuring connectivity and preventing fractures in tubular object segmentation are critical for downstream analyses. Despite advancements in deep neural networks that have significantly improved tubular object segmentation, existing methods still face limitations. They often rely heavily on precise annotations, hindering their scalability to large-scale unlabeled image datasets. Additionally, current evaluation metrics are insufficient for effectively capturing segmentation fractures. To address these challenges, we propose a semi-supervised fracture-attention model (SSFA) for tubular object segmentation. SSFA enhances connectivity, reduces fractures, and maintains volumetric accuracy. It outperforms state-of-the-art models in topological performance. Extensive experiments on four public datasets validate the effectiveness of SSFA. Furthermore, we introduce a novel evaluation metric, the fracture rate, which provides an intuitive and quantitative assessment of segmentation fractures. Our source code is available at http://github.com/Yanfeng-Zhou/SSFA.

Gradient Enhancement Techniques and Motion Consistency Constraints for Moving Object Segmentation in 3D LiDAR Point Clouds

The ability to segment moving objects from three-dimensional (3D) LiDAR scans is critical to advancing autonomous driving technology, facilitating core tasks like localization, collision avoidance, and path planning. In this paper, we introduce a novel deep neural network designed to enhance the performance of 3D LiDAR point cloud moving object segmentation (MOS) through the integration of image gradient information and the principle of motion consistency. Our method processes sequential range images, employing depth pixel difference convolution (DPDC) to improve the efficacy of dilated convolutions, thus boosting spatial information extraction from range images. Additionally, we incorporate Bayesian filtering to impose posterior constraints on predictions, enhancing the accuracy of motion segmentation. To handle the issue of uneven object scales in range images, we develop a novel edge-aware loss function and use a progressive training strategy to further boost performance. Our method is validated on the SemanticKITTI-based LiDAR MOS benchmark, where it significantly outperforms current state-of-the-art (SOTA) methods, all while working directly on two-dimensional (2D) range images without requiring mapping.

Unraveling Spatial Heterogeneity in Mass Spectrometry Imaging Data with GraphMSI.

Mass spectrometry imaging (MSI) provides valuable insights into metabolic heterogeneity by capturing in situ molecular profiles within organisms. One challenge of MSI heterogeneity analysis is performing an objective segmentation to differentiate the biological tissue into distinct regions with unique characteristics. However, current methods struggle due to the insufficient incorporation of biological context and high computational demand. To address these challenges, a novel deep learning-based approach is proposed, GraphMSI, which integrates metabolic profiles with spatial information to enhance MSI data analysis. Our comparative results demonstrate GraphMSI outperforms commonly used segmentation methods in both visual inspection and quantitative evaluation. Moreover, GraphMSI can incorporate partial or coarse biological contexts to improve segmentation results and enable more effective three-dimensional MSI segmentation with reduced computational requirements. These are facilitated by two optional enhanced modes: scribble-interactive and knowledge-transfer. Numerous results demonstrate the robustness of these two modes, ensuring that GraphMSI consistently retains its capability to identify biologically relevant sub-regions in complex practical applications. It is anticipated that GraphMSI will become a powerful tool for spatial heterogeneity analysis in MSI data.

Graph-based Moving Object Segmentation for underwater videos using semi-supervised learning

Graph-based Moving Object Segmentation for underwater videos using semi-supervised learning

Aligning Instance-Semantic Sparse Representation towards Unsupervised Object Segmentation and Shape Abstraction with Repeatable Primitives

Aligning Instance-Semantic Sparse Representation towards Unsupervised Object Segmentation and Shape Abstraction with Repeatable Primitives

Spatial Mask-based Adaptive Robust Training for Video Object Segmentation with Noisy Labels

Spatial Mask-based Adaptive Robust Training for Video Object Segmentation with Noisy Labels

SA3D-L: A lightweight model for 3D object segmentation using neural radiance fields

SA3D-L: A lightweight model for 3D object segmentation using neural radiance fields

Quantifying and Learning Static Vs. Dynamic Information in Deep Spatiotemporal Networks.

There is limited understanding of the information captured by deep spatiotemporal models in their intermediate representations. For example, while evidence suggests that action recognition algorithms are heavily influenced by visual appearance in single frames, no quantitative methodology exists for evaluating such static bias in the latent representation compared to bias toward dynamics. We tackle this challenge by proposing an approach for quantifying the static and dynamic biases of any spatiotemporal model, and apply our approach to three tasks, action recognition, automatic video object segmentation (AVOS) and video instance segmentation (VIS). Our key findings are: (i) Most examined models are biased toward static information. (ii) Some datasets that are assumed to be biased toward dynamics are actually biased toward static information. (iii) Individual channels in an architecture can be biased toward static, dynamic or jointly encode a combination static and dynamic information. (iv) Most models converge to their culminating biases in the first half of training. We then explore how these biases affect performance on dynamically biased datasets. For action recognition, we propose StaticDropout, a semantically guided dropout that debiases a model from static information toward dynamics. For AVOS, we design a better combination of fusion and cross connection layers compared with previous architectures.

Application of Decomposition Expression in Digital Video Object Segmentation

In the field of actual Video Object Segmentation (VOS), traditional techniques have poor adaptability and insufficient segmentation results. Therefore, based on existing problems, an Unsupervised Video Object Segmentation (UVOS) technique based on convolutional networks is proposed. Firstly, the method of decomposing expressions is used to handle the spatiotemporal relationship between the reference frame and the target frame, and video object reconstruction is achieved through similarity calculation. For target segmentation in motion scenes, a Single Linear Bottleneck Operator (SLBO) is introduced for feature extraction, and pooling compensation is used to optimize feature information loss. For general scene segmentation, a spatiotemporal similarity segmentation technique is introduced to achieve target video segmentation for complex scenes. In the foreground segmentation test of sports scenes, the Change Detection Benchmark Dataset 2014 (CDNet.20I4SM) dataset was selected to test the model's loss performance in different scenarios. In adverse weather scenario training, the proposed model tends to converge after 40 iterations, with a loss value of 0.276, which is superior to the Foreground image Segmentation (FgSegNet_), the Convolutional Networks for Biomedical Image Segmentation (MU Net), Cascade Convolutional Neural Network (Cascade CNN) models; In the accuracy test, the proposed FS-LBPC model tended to converge after 50 iterations, with a precision P-value of 0.963. It performed the best among the four segmentation models the FgSegNet_, MU Net, Cascade CNN, and a real-time Foreground Segmentation network based on single Linear Bottleneck and Pooling Compensation (FS-LBPC). Usually, the Densely Annotated VIdeo Segmentation (DAVIS16) dataset is selected for video scene segmentation, which has the best segmentation performance in horse racing and animal flight scenes, with segmentation accuracy of 0.976 and 0.965, respectively. In summary, the VOS technology has excellent application effects in practical scenarios, providing important technical references for the improvement of image and video processing and segmentation technology

Augmenting efficient real-time surgical instrument segmentation in video with point tracking and Segment Anything.

The Segment Anything model (SAM) is a powerful vision foundation model that is revolutionizing the traditional paradigm of segmentation. Despite this, a reliance on prompting each frame and large computational cost limit its usage in robotically assisted surgery. Applications, such as augmented reality guidance, require little user intervention along with efficient inference to be usable clinically. This study addresses these limitations by adopting lightweight SAM variants to meet the efficiency requirement and employing fine-tuning techniques to enhance their generalization in surgical scenes. Recent advancements in tracking any point have shown promising results in both accuracy and efficiency, particularly when points are occluded or leave the field of view. Inspired by this progress, a novel framework is presented that combines an online point tracker with a lightweight SAM model that is fine-tuned for surgical instrument segmentation. Sparse points within the region of interest are tracked and used to prompt SAM throughout the video sequence, providing temporal consistency. The quantitative results surpass the state-of-the-art semi-supervised video object segmentation method XMem on the EndoVis 2015 dataset with 84.8 IoU and 91.0 Dice. The method achieves promising performance that is comparable to XMem and transformer-based fully supervised segmentation methods on ex vivo UCL dVRK and in vivo CholecSeg8k datasets. In addition, the proposed method shows promising zero-shot generalization ability on the label-free STIR dataset. In terms of efficiency, the method was tested on a single GeForce RTX 4060/4090 GPU respectively, achieving an over 25/90 FPS inference speed. Code is available at: https://github.com/zijianwu1231/SIS-PT-SAM.

Adaptive Refinement of Segmented Object Contour Based on the Brightness of Neighboring Pixels Using the Ensemble Method

Improving the accuracy of computer vision algorithms plays a significant role in the tasks of medical image segmentation. After all, determining the boundaries of objects is a difficult task when using medical images, and especially X-ray images. The use of X-ray images in segmentation tasks is a complex process, since these images themselves can have a sufficient amount of noise and artifacts. Classical segmentation methods face significant challenges when segmenting X-ray images where there are objects with fuzzy boundaries. To solve such tasks, it is suggested to use segmentation with the help of machine learning, and to increase the accuracy of determining the boundaries of objects, it is necessary to use adaptive approaches. This paper proposes a new method to improve the accuracy of X-ray image segmentation, which analyzes the neighboring pixels of each contour element and adaptively reshapes it if necessary, and then combines all predictions using an ensemble method, which improves the previous version of the contour. The method was able to demonstrate an improvement in the quality of image segmentation on three datasets with different complexity of structures. Improvements in object boundary accuracy were obtained for all three sets.

Efficient video transmission using object segmentation over orthogonal frequency-division multiplexing-based underwater acoustic channel

Efficient video transmission using object segmentation over orthogonal frequency-division multiplexing-based underwater acoustic channel

FCB-YOLOv8s-Seg: A Malignant Weed Instance Segmentation Model for Targeted Spraying in Soybean Fields

Effective management of malignant weeds is critical to soybean growth. This study focuses on addressing the critical challenges of targeted spraying operations for malignant weeds such as Cirsium setosum, which severely threaten soybean yield in soybean fields. Specifically, this research aims to tackle key issues in plant protection operations, including the precise identification of weeds, the lightweight deployment of segmentation models, real-time requirements for spraying operations, and the generalization ability of models in diverse field environments. To address these challenges, this study proposes an improved weed instance segmentation model based on YOLOv8s-Seg, named FCB-YOLOv8s-Seg, for targeted spraying operations in soybean fields. The FCB-YOLOv8s-Seg model incorporates a lightweight backbone network to accelerate computations and reduce model size, with optimized Squeeze-and-Excitation Networks (SENet) and Bidirectional Feature Pyramid Network (BiFPN) modules integrated into the neck network to enhance weed recognition accuracy. Data collected from real soybean field scenes were used for model training and testing. The results of ablation experiments revealed that the FCB-YOLOv8s-Seg model achieved a mean average precision of 95.18% for bounding box prediction and 96.63% for segmentation, marking an increase of 5.08% and 7.43% over the original YOLOv8s-Seg model. While maintaining a balanced model scale, the object detection and segmentation accuracy of this model surpass other existing classic models such as YOLOv5s-Seg, Mask-RCNN, and YOLACT. The detection results in different scenes show that the FCB-YOLOv8s-Seg model performs well in fine-grained feature segmentation in complex scenes. Compared with several existing classical models, the FCB-YOLOv8s-Seg model demonstrates better performance. Additionally, field tests on plots with varying weed densities and operational speeds indicated an average segmentation rate of 91.30%, which is 6.38% higher than the original model. The proposed algorithm shows higher accuracy and performance in practical field instance segmentation tasks and is expected to provide strong technical support for promoting targeted spray operations.

A dynamic dropout self-distillation method for object segmentation

There is a phenomenon that better teachers cannot teach out better students in knowledge distillation due to the capacity mismatch. Especially in pixel-level object segmentation, there are some challenging pixels that are difficult for the student model to learn. Even if the student model learns from the teacher model for each pixel, the student’s performance still struggles to show significant improvement. Mimicking the learning process of human beings from easy to difficult, a dynamic dropout self-distillation method for object segmentation is proposed, which solves this problem by discarding the knowledge that the student struggles to learn. Firstly, the pixels where there is a significant difference between the teacher and student models are found according to the predicted probabilities. And these pixels are defined as difficult-to-learn pixel for the student model. Secondly, a dynamic dropout strategy is proposed to match the capability variation of the student model, which is used to discard the pixels with hard knowledge for the student model. Finally, to validate the effectiveness of the proposed method, a simple student model for object segmentation and a virtual teacher model with perfect segmentation accuracy are constructed. Experiment results on four public datasets demonstrate that, when there is a large performance gap between the teacher and student models, the proposed self-distillation method is more effective in improving the performance of the student model compared to other methods.