Segmentation Problem Research Articles

A Nighttime Driving-Scene Segmentation Method Based on Light-Enhanced Network

To solve the semantic segmentation problem of night driving-scene images, which often have low brightness, low contrast, and uneven illumination, a nighttime driving-scene segmentation method based on a light-enhanced network was proposed. Firstly, we designed a light enhancement network, which comprises two parts: a color correction module and a parameter predictor. The color correction module mitigates the impact of illumination variations on the segmentation network by adjusting the color information of the image. Meanwhile, the parameter predictor accurately predicts the parameters of the image filter through the analysis of global content, including factors such as brightness, contrast, hue, and exposure level, thereby effectively enhancing the image quality. Subsequently, the output of the light enhancement network is input into the segmentation network to obtain the final segmentation prediction. Experimental results show that the proposed method achieves mean Intersection over Union (mIoU) values of 59.4% on the Dark Zurich-test dataset, outperforming other segmentation algorithms for nighttime driving-scenes.

Improved MRF rail surface defect segmentation method based on clustering features

Aiming at the characteristics of small number and many types of rail surface defect samples, as well as the problems of unstable transfer learning effect and threshold segmentation being easily affected by environmental factors in real scenes, an improved Markov defect segmentation method with zero samples is proposed. Firstly, the collected data is processed by Gabor function to highlight the defect features and reduce the data dimension to obtain the reduced dimension feature map; Kmeans clustering is performed on the processed feature map to reduce the distribution of data and reduce the influence of reflection and shadow, and the clustering result is used as the pre-classification matrix; an improved Markov random field two-layer graph model is constructed and inferred through the reduced dimension feature map and the pre-classification matrix; the local geometric structure of the defect part is analyzed according to the eigenvalues of the classification matrix inferred by the model; finally, the defect area is marked and the defect segmentation is completed. The experimental part uses a self-sampling data set, and the final conclusion is drawn based on the comparative experiment and ablation experiment. The experimental results show that the pixel accuracy, average pixel accuracy, weighted intersection-over-union ratio, and average intersection-over-union ratio of this method on the self-sampling data set are respectively 93.6%、80.7%、89.4%、68.2% , which exceeds the accuracy of other comparative detection algorithms.

Semi-supervised segmentation of cardiac chambers from LGE-CMR using feature consistency awareness

BackgroundLate gadolinium enhancement cardiac magnetic resonance imaging (LGE-CMR) is a valuable cardiovascular imaging technique. Segmentation of cardiac chambers from LGE-CMR is a fundamental step in electrophysiological modeling and cardiovascular disease diagnosis. Deep learning methods have demonstrated extremely promising performance. However, excellent performance often depended on a large amount of finely annotated data. The purpose of this manuscript was to develop a semi-supervised segmentation method to use unlabeled data to improve model performance.MethodsThis manuscript proposed a semi-supervised network that integrates triple-consistency constraints (data-level, task-level, and feature-level) for cardiac chambers segmentation from LGE-CMR. Specifically, we designed a network that integrated segmentation and edge prediction tasks based on the mean teacher architecture. This addressed the problem of ignoring some challenging regions because of excluding low-confidence regions of previous research. We also applied a voxel-level contrastive learning strategy to achieve feature-level consistency, helping the model pay attention to the consistency between features overlooked in previous research.ResultsIn terms of the Dice, Jaccard, Average Surface Distance (ASD), and 95% Hausdorff Distance (95HD) metrics, for the atrial segmentation dataset, the proposed method achieved scores of 88.34%, 79.30%, 7.92, and 2.02 when trained with 10% labeled data, and 90.70%, 83.09%, 6.41, and 1.72 when trained with 20% labeled data. For the ventricular segmentation task, the results were 87.22%, 77.95%, 2.27, and 0.61 with 10% labeled data, and 88.99%, 80.45%, 1.87, and 0.51 with 20% labeled data, respectively.ConclusionExperiments demonstrated that our method outperforms previous semi-supervised methods, showing the potential of the proposed network for semi-supervised segmentation problems.

Core Potentials: The Consensus Segmentation Conjecture

Segmentations are partitions of an ordered set into non-overlapping intervals. The Consensus Segmentation or Segmentation Aggregation problem is a special case of the median problems with applications in time series analysis and computational biology. A wide range of dissimilarity measures for segmentations can be expressed in terms of potentials, a special type of set-functions. In this contribution, we shed more light on the properties of potentials, and how such properties affect the solutions of the Consensus Segmentation problem. In particular, we disprove a conjecture stated in 2021, and we provide further insights into the theoretical foundations of the problem.

An enhanced seagull algorithm for multi-threshold image segmentation based on Kapur entropy

Abstract To address the shortcomings of the Seagull Optimization Algorithm (SOA), such as poor distribution of individuals’ positions and low population diversity leading to susceptibility to local optima after the collision avoidance phase, an Enhanced Seagull Algorithm (ESOA) is proposed. Firstly, a chaos-guided mechanism is utilized to replace the original individual positions after collision avoidance with the mean of individual optima and current positions, effectively improving the distribution of the population. Secondly, a circular-topology neighborhood structure with random weights is introduced, allowing interaction between the current individual’s position and the information from its two adjacent optimal positions, thereby improving population clustering and enhancing diversity. Thirdly, a dynamic lens mapping strategy is employed to direct the optimal seagull individuals toward their newly generated mapped points, enhancing their ability to avoid local optima. ESOA is applied to the problem of multi-threshold image segmentation based on Kapur entropy, and experimental results demonstrate its effectiveness in improving segmentation performance.

Analysis of applying machine learning methods in rock classification problems on core samples

Relevance. Core studies in the oil and gas industry allow us to obtain some filtration and capacity characteristics of rocks, as well as to give an idea of the composition and structure of the subsoil. This information is extremely important in the early stages of field development, as it allows us to formulate a primary version of the development project, which is then refined during the course of field drilling. However, core analysis and description are an extremely labor-intensive and human-influenced works that require automation. Thus, core image research is a popular task in the oil and gas industry that requires high accuracy and care during work; especially considering the volume of images that have to be analyzed. Aim. To review and analyze existing algorithms for classifying rocks from core images based on machine learning methods; as well as to use the information obtained to formulate recommendations for the development of these algorithms. Methods. Machine learning methods, including neural networks. Results. The work analyzed existing approaches to the study of core samples. The main advantages and disadvantages of each of them were noted and, based on the findings, a plan and requirements for conducting further research on core samples using machine learning were developed. As a result, using a convolutional neural network on the U-Net architecture, the authors have trained a model to solve the problem of segmentation of core samples in daytime images and presented the results of the model operation.

Deep handwritten diagram segmentation

AbstractHandwriting is a natural way to communicate and exchange ideas, but converting handwritten diagrams to application‐specific digital formats requires skill and time. Automatic handwritten document conversion can save time, but diagrams and text require different recognition engines. Since accurate segmentation of handwritten diagrams can improve the accuracy of later diagram recognition steps, the authors propose to solve the problem of segmentation of text and non‐text elements of handwritten diagrams using deep semantic segmentation. The model, DeepDP is a flexible U‐net style architecture that can be tuned in complexity to a level appropriate for a particular dataset and diagram type. Experiments on a public hand‐drawn flowchart dataset and a business process diagram dataset show excellent performance, with a per pixel accuracy of 98.6% on the public flowchart datasets and improvement over the 99.3% text stroke accuracy and 96.6% non‐text stroke accuracy obtained by state of the art methods that use online stroke information. On the smaller offline business process diagram dataset, the method obtains a per‐pixel accuracy of 96.9%.

Edge morphology attention mechanism and optimal geometric matching connection model for vascular segmentation

Over the past decades, medical image segmentation methods have been intensively developed, but there are still a number of unsolved challenging problems in vascular image segmentation, including broken vessels, insufficient vessel branches, and missing small vessels. In order to optimize the topology and accuracy of segmented vessels, we propose a novel Edge Morphology Attention Network (EMA-Net) for the segmentation of vessel-like structures, and an Optimal Geometric Matching Connection (OGMC) model to connect the broken vessel fragments. The EMA-Net has an edge attention module that is able to improve segmented performances of edges and small branches by morphology operation extracting boundary voxels on multi-scales. The OGMC model uses the geometry concept of curve touching to filter out fragmented vessel endpoints, and then employs minimal surfaces to determine the optimal connection order between blood vessels. Finally, we adopt geodesics to repair missing vessels under a Riemannian metric. Our method achieves superior or competitive results compared to state-of-the-art methods on four datasets of 3D vascular segmentation tasks, both effectively reducing vessel broken and increasing vessel branch richness, yielding blood vessels with more precise topological structures.

A Study on Resolving Dynamic analysis using deep learning Framework Logical Methods

Deep learning technologies have gained a great deal of interest in recent months for demonstrating material removal capabilities in a variety of separation, categorization, as well as other machine vision activities. The majority of these deep networks are built employing convolution or completely convolution structures. Our suggested Dynamic and Static gestures System guided by a hand gesture high dimensional data methodology We use a snapshot of a human's thoracic spine to determine the person's depth and the size of the space around his or her hands. In this research, we provide a novel object-based deep-learning framework for semantic segmentation of extremely high-resolution satellite data. In specifically, we leverage object-based prior record by augmenting a fully convolutional neural network's training strategy with an anisotropic diffusion data pre-processing phase and an additional loss term. The goal of this restricted framework is to ensure that similar visual data is assigned to the same semantic class. Here, we employ intermediate steps based on traditional image processing techniques to aid in the resolution of the subsequent problem of classification, detection, or segmentation. Research shows that adding pre- and post-processing steps to a deep learning pipeline can boost model performance over using only the network. Recent advances in UQ algorithms used for deep learning are analysed, and their potential for use in relevance feedback is addressed. It also emphasizes basic research difficulties and directions linked with UQ. Quantifiably, man-made classes with more precise geometry, such as buildings, benefited the most from our strategy, particularly along object borders, indicating the developed approach's enormous potential. The purpose of this paper is to offer an overview of the approaches used inside deep learning frameworks to either appropriately prepare the input (pre-processing) or enhance the outcomes of the network output (post-processing), with an emphasis on digital pathology image analysis.

Spatial attention U-Net model with Harris hawks optimization for retinal blood vessel and optic disc segmentation in fundus images.

The state of the human eye's blood vessels is a crucial aspect in the diagnosis of ophthalmological illnesses. For many computer-aided diagnostic systems, precise retinal vessel segmentation is an essential job. However, it remains a difficult task due to the intricate vascular system of the eye. Although many different vascular segmentation techniques have already been presented, additional study is still required to address the problem of inadequate segmentation of thin and tiny vessels. In this work, we introduce the Spatial Attention U-Net (SAU-Net) modelwith harris hawks' optimization (HHO), a lightweight network that can be applied as a data augmentation technique to improve the efficiency of the existingannotated samples without the need of thousands of training instances for Retinal Blood Vessel and Optic Disc Segmentation. The SAU-Net-HHO implementation uses a spatially inferred attention map multiplied by the input feature map for adaptive feature enhancement. U-Net convolutional blocks have been replaced with structured dropout blocks in the proposed network to prevent overfitting. Data from both vascular extraction (DRIVE) and structured analysis of the retina (STARE) are used to evaluate SAU-Net-HHO performance. The results show that the proposed SAU-Net-HHO performs well on both datasets. Analysing the obtained results, an average of 98.5% accuracy and Specificity 96.7% was achieved for DRIVE dataset and 97.8% accuracy and specificity 94.5% for STARE dataset. The proposed method yields numerical results with average values that are on par with those of state-of-the-art methods. Visual inspection has revealed that the suggested method can segment thin and tiny vessels with greater accuracy than previous methods. It also demonstrates its potential for real-life clinical application.

FCT-Net: A dual-encoding-path network fusing atrous spatial pyramid pooling and transformer for pavement crack detection

Cracks are a typical form of road damage, and accurate detection of cracks is of great significance for road maintenance work and ensuring traffic safety. Recently, computer vision has gradually been applied in the field of crack segmentation. However, there are still some extremely challenging problems in crack segmentation, such as complex backgrounds, information loss caused by pooling and convolution operations, and insufficient fusion of global and local semantic information. In response to the above problems, this paper proposes a dual-encoding-path network with U-Net architecture called FCT-Net, by fusing channel atrous spatial pyramid pooling (CASPP) and transformer. Specifically, CASPP obtains multi-scale receptive fields by incorporating spatial and channel attention, while refining and extracting local features. Meanwhile, we introduce long-short distance attention to construct a novel transformer with the prominent characteristic of interaction between local and global attention features. In addition, a residual convolution module is designed to enhance the local features of the transformer. Furthermore, we devise a multi-scale attention weight cross fusion module to aggregate the features of the dual encoding branch, for reducing information loss during downsampling and suppress background information. Eventually, we evaluate the performance of FCT-Net by experiments on three public datasets. Extensive experimental results show that FCT-Net achieves higher F1-score and mean intersection over union (mIoU) than state-of-the-art segmentation networks on the DeepCrack537 and CrackLS315 datasets. Meanwhile, it has excellent segmentation performance for cracks in complex scenes, with the highest recall, F1-score, and mIoU respectively as 85.64%, 81.67%, and 84.05% on the CrackTree260 dataset.

An improved nutcracker optimization algorithm for discrete and continuous optimization problems: Design, comprehensive analysis, and engineering applications

This study is presented to examine the performance of a newly proposed metaheuristic algorithm within discrete and continuous search spaces. Therefore, the multithresholding image segmentation problem and parameter estimation problem of both the proton exchange membrane fuel cell (PEMFC) and photovoltaic (PV) models, which have different search spaces, are used to test and verify this algorithm. The traditional techniques could not find approximate solutions for those problems in a reasonable amount of time, so researchers have used metaheuristic algorithms to overcome those shortcomings. However, the majority of metaheuristic algorithms still suffer from slow convergence speed and stagnation into local minima problems, which makes them unsuitable for tackling these optimization problems. Therefore, this study proposes an improved nutcracker optimization algorithm (INOA) for better solving those problems in an acceptable amount of time. INOA is based on improving the performance of the standard algorithm using a newly proposed convergence improvement strategy that aims to improve the convergence speed and prevent stagnation in local minima. This algorithm is first applied to estimating the unknown parameters of the single-diode, double-diode, and triple-diode models for a PV module and a solar cell. Second, four PEMFC modules are used to further observe INOA's performance for the continuous optimization challenge. Finally, the performance of INOA is investigated for solving the multi-thresholding image segmentation problem to test its effectiveness in a discrete search space. Several test images with different threshold levels were used to validate its effectiveness, stability, and scalability. Comparison to several rival optimizers using various performance indicators, such as convergence curve, standard deviation, average fitness value, and Wilcoxon rank-sum test, demonstrates that INOA is an effective alternative for solving both discrete and continuous optimization problems. Quantitively, INOA could solve those problems better than the other rival optimizers, with improvement rates for final results ranging between 0.8355 % and 3.34 % for discrete problems and 4.97 % and 99.9 % for continuous problems.

Multi-Person Action Recognition Based on Millimeter-Wave Radar Point Cloud

Human action recognition has many application prospects in human-computer interactions, innovative furniture, healthcare, and other fields. The traditional human motion recognition methods have limitations in privacy protection, complex environments, and multi-person scenarios. Millimeter-wave radar has attracted attention due to its ultra-high resolution and all-weather operation. Many existing studies have discussed the application of millimeter-wave radar in single-person scenarios, but only some have addressed the problem of action recognition in multi-person scenarios. This paper uses a commercial millimeter-wave radar device for human action recognition in multi-person scenarios. In order to solve the problems of severe interference and complex target segmentation in multiplayer scenarios, we propose a filtering method based on millimeter-wave inter-frame differences to filter the collected human point cloud data. We then use the DBSCAN algorithm and the Hungarian algorithm to segment the target, and finally input the data into a neural network for classification. The classification accuracy of the system proposed in this paper reaches 92.2% in multi-person scenarios through experimental tests with the five actions we set.

A novel hybrid methodology integrating pixel- and object-based techniques for mapping land use and land cover from high-resolution satellite data

ABSTRACT The classification of very high-resolution satellite imagery remains a focal point in remote sensing, attracting increased attention across diverse scientific disciplines. Various classification methods, including pixel- and object-based techniques, have been proposed, and their performances and limitations have been discussed in the literature. This paper presents a hybrid method that combines the strengths of pixel- and object-based methods in image classification to minimize errors associated with the segmentation process, particularly under-segmentation errors in object-based image analysis. The core concept behind the method lies in categorizing segmented image objects as either homogeneous or heterogeneous based on their class probability. In this process, the estimated possibilities from the object-based classification model are considered, and segments are designated as homogeneous or heterogeneous using a user-defined threshold. The object-based classification model determines the class labels for homogeneous image objects, while the heterogeneous ones, containing pixels representing different land cover classes, are classified using the pixel-based model. The performance of hybrid classification models, created by varying thresholds, is analysed using high-resolution WorldView-3 and WorldView-2 imagery and compared with pixel- and object-based classification results. For the implementation of image classification methods, Canonical Correlation Forest (CCF), Extreme Gradient Boosting (XGBoost), and Categorical Boosting (CatBoost) were employed. The findings indicated that employing the suggested hybrid strategy with a threshold value selected within a specific range (e.g. between 60% and 80%) and employing a robust classification algorithm that provides class probabilities (e.g. CCF) results in a statistically significant improvement in overall accuracy compared to pixel and object-based methods, with gains of 5% and 4%, respectively. Visual analysis of the produced thematic maps revealed that the proposed method minimizes the salt-and-pepper effect associated with pixel-based classification while mitigating segmentation problems arising from object-based classification.

Bayesian image segmentation under varying blur with triplet Markov random field

Abstract In this paper, we place ourselves in the context of the Bayesian framework for image segmentation in the presence of varying blur. The proposed approach is based on Triplet Markov Random Fields (TMRF). This method takes into account, during segmentation, peculiarities of an image such as noise, blur, and texture. We present an unsupervised TMRF method, which jointly deals with the problem of segmentation, and that of depth estimation in order to process fluorescence microscopy images. In addition to the estimation of the depth maps using the Metropolis-Hasting and the Stochastic Parameter Estimation (SPE) algorithms, we also estimate the model parameters using the SPE algorithm. We compare our TMRF method to other MRF models on simulated images, and to an unsupervised method from the state of art on real fluorescence microscopy images. Our method offers improved results, especially when blur is important.

Medical image segmentation approach based on hybrid adaptive differential evolution and crayfish optimizer

Image segmentation plays a pivotal role in medical image analysis, particularly for accurately isolating tumors and lesions. Effective segmentation improves diagnostic precision and facilitates quantitative analysis, which is vital for medical professionals. However, traditional segmentation methods often struggle with multilevel thresholding due to the associated computational complexity. Therefore, determining the optimal threshold set is an NP-hard problem, highlighting the pressing need for efficient optimization strategies to overcome these challenges. This paper introduces a multi-threshold image segmentation (MTIS) method that integrates a hybrid approach combining Differential Evolution (DE) and the Crayfish Optimization Algorithm (COA), known as HADECO. Utilizing two-dimensional (2D) Kapur’s entropy and a 2D histogram, this method aims to enhance the efficiency and accuracy of subsequent image analysis and diagnosis. HADECO is a hybrid algorithm that combines DE and COA by exchanging information based on predefined rules, leveraging the strengths of both for superior optimization results. It employs Latin Hypercube Sampling (LHS) to generate a high-quality initial population. HADECO introduces an improved DE algorithm (IDE) with adaptive and dynamic adjustments to key DE parameters and new mutation strategies to enhance its search capability. In addition, it incorporates an adaptive COA (ACOA) with dynamic adjustments to the switching probability parameter, effectively balancing exploration and exploitation. To evaluate the effectiveness of HADECO, its performance is initially assessed using CEC’22 benchmark functions. HADECO is evaluated against several contemporary algorithms using the Wilcoxon signed rank test (WSRT) and the Friedman test (FT) to integrate the results. The findings highlight HADECO’s superior optimization abilities, demonstrated by its lowest average Friedman ranking of 1.08. Furthermore, the HADECO-based MTIS method is evaluated using MRI images for knee and CT scans for brain intracranial hemorrhage (ICH). Quantitative results in brain hemorrhage image segmentation show that the proposed method achieves a superior average peak signal-to-noise ratio (PSNR) and feature similarity index (FSIM) of 1.5 and 1.7 at the 6-level threshold. In knee image segmentation, it attains an average PSNR and FSIM of 1.3 and 1.2 at the 5-level threshold, demonstrating the method’s effectiveness in solving image segmentation problems.

Robust variance estimators in application to segmentation of measurement data distorted by impulsive and non-Gaussian noise

The paper algorithmizes the problem of regime change point identification for data measured in a system exhibiting impulsive behaviors. This is a fundamental challenge for annotation of measurement data, relevant, e.g., for designing data-driven autonomous systems. The contribution consists in formulation of an offline robust methodology based on the classical approach for structural break detection. The problem of data segmentation for the changing scale is solved as a process variable sensitive to the occurrence of a critical event in a reorganizing physical system. The advantage is that this approach does not require the existence of a variance of the data distribution. Efficiency has been evaluated for simulated data from two distributions and for real-world data sets measured in financial, mechanical, and medical systems. Simulation studies show that in the most challenging case, the error mean absolute in estimating the regime change is 20 times lower for the robust approach compared to the classical approach.

A Graph Multi-separator Problem for Image Segmentation

We propose a novel abstraction of the image segmentation task in the form of a combinatorial optimization problem that we call the multi-separator problem. Feasible solutions indicate for every pixel whether it belongs to a segment or a segment separator, and indicate for pairs of pixels whether or not the pixels belong to the same segment. This is in contrast to the closely related lifted multicut problem, where every pixel is associated with a segment and no pixel explicitly represents a separating structure. While the multi-separator problem is np-hard, we identify two special cases for which it can be solved efficiently. Moreover, we define two local search algorithms for the general case and demonstrate their effectiveness in segmenting simulated volume images of foam cells and filaments.

Instance segmentation of pigs in infrared images based on INPC model

Conventional segmentation methods based on visible images in intensive pig farming face various challenges. Examples include color differences between pig breeds, background interference and lighting conditions. To overcome these issues, we designed the infrared pig cascade segmentation (INPC) model for the first time on infrared images. The model uses a cascade structure. Each stage utilizes higher resolution feature maps to better preserve fine details. It also solves the problem of poor segmentation of small objects due to low resolution of infrared images. At the same time, the model’s cross-guidance strategy enhances the interaction between bounding box regression and mask prediction. This reduces errors caused by interference like feces and urine. Additionally, a progressive mask branch refines mask prediction, improving segmentation in scenarios like imaging haze or pig adhesion. To facilitate model training and evaluation, we built the first large-scale standardized infrared pig dataset. Experimental results demonstrate that INPC outperforms mainstream segmentation models in terms of average precision (AP), except for AP0.5. Specifically, INPC achieves AP0.5, AP0.75, AP0.5:0.95, AP0.5:0.95s, and AP0.5:0.95l of 97.9%, 97.1%, 88.2%, 71.5%, and 90.1% respectively. Inference for a single image on a GPU takes only 0.197 s. Some datasets are available at https://github.com/HUBUwg96/INPC.

Cross-Image Pixel Contrasting for Semantic Segmentation.

This work studies the problem of image semantic segmentation. Current approaches focus mainly on mining "local" context, i.e., dependencies between pixels within individual images, by specifically-designed, context aggregation modules (e.g., dilated convolution, neural attention) or structure-aware optimization objectives (e.g., IoU-like loss). However, they ignore "global" context of the training data, i.e., rich semantic relations between pixels across different images. Inspired by recent advance in unsupervised contrastive representation learning, we propose a pixel-wise contrastive algorithm, dubbed as PiCo, for semantic segmentation in the fully supervised learning setting. The core idea is to enforce pixel embeddings belonging to a same semantic class to be more similar than embeddings from different classes. It raises a pixel-wise metric learning paradigm for semantic segmentation, by explicitly exploring the structures of labeled pixels, which were rarely studied before. Our training algorithm is compatible with modern segmentation solutions without extra overhead during testing. We experimentally show that, with famous segmentation models (i.e., DeepLabV3, HRNet, OCRNet, SegFormer, Segmenter, MaskFormer) and backbones (i.e., MobileNet, ResNet, HRNet, MiT, ViT), our algorithm brings consistent performance improvements across diverse datasets (i.e., Cityscapes, ADE20 K, PASCAL-Context, COCO-Stuff, CamVid). We expect that this work will encourage our community to rethink the current de facto training paradigm in semantic segmentation.