Texture Segmentation Research Articles

Enhanced Wavelet Scattering Network for Image Inpainting Detection

The rapid advancement of image inpainting tools, especially those aimed at removing artifacts, has made digital image manipulation alarmingly accessible. This paper proposes several innovative ideas for detecting inpainting forgeries based on a low-level noise analysis by combining Dual-Tree Complex Wavelet Transform (DT-CWT) for feature extraction with convolutional neural networks (CNN) for forged area detection and localization, and lastly by employing an innovative combination of texture segmentation with noise variance estimations. The DT-CWT offers significant advantages due to its shift-invariance, enhancing its robustness against subtle manipulations during the inpainting process. Furthermore, its directional selectivity allows for the detection of subtle artifacts introduced by inpainting within specific frequency bands and orientations. Various neural network architectures were evaluated and proposed. Lastly, we propose a fusion detection module that combines texture analysis with noise variance estimation to give the forged area. Also, to address the limitations of existing inpainting datasets, particularly their lack of clear separation between inpainted regions and removed objects—which can inadvertently favor detection—we introduced a new dataset named the Real Inpainting Detection Dataset. Our approach was benchmarked against state-of-the-art methods and demonstrated superior performance over all cited alternatives.

Identification of wheel surface cracks under high noise environment based on texture segmentation and combination descriptor

ABSTRACT The wheel is easy to produce defects after a long time of use. Train maintenance technicians must overhaul and maintain train wheels on time and regularly to ensure the safe operation of trains. The existing manual detection has disadvantages of low efficiency and strong subjectivity. Compared with other defect forms, the identification accuracy of wheel surface crack is low due to the high noise and weak features, this paper proposes a feature enhancement and identification method for wheel surface crack based on texture segmentation and combination descriptor. Firstly, a morphology-based texture segmentation method is proposed to separate cracks and noise. Then, the crack characteristics are enhanced by filtering. Finally, a crack diagnosis method based on combination descriptor is proposed, which is evaluated from two dimensions of crack morphology and distribution characteristics. In order to verify the effectiveness of the proposed method, the images of the wheel collected from the freight car workshop were used for verification, and the detection rate reached 93.78%. The experimental results show that the proposed method is more effective in the case of weak crack features and surface dirt, and can better adapt to the identification of surface cracks on wheel in complex engineering applications.

Robust unsupervised texture segmentation for motion analysis in ultrasound images.

Ultrasound imaging has emerged as a promising cost-effective and portable non-irradiant modality for the diagnosis and follow-up of diseases. Motion analysis can be performed by segmenting anatomical structures of interest before tracking them over time. However, doing so in a robust way is challenging as ultrasound images often display a low contrast and blurry boundaries. In this paper, a robust descriptor inspired from the fractal dimension is presented to locally characterize the gray-level variations of an image. This descriptor is an adaptive grid pattern whose scale locally varies as the gray-level variations of the image. Robust features are then located based on the gray-level variations, which are more likely to be consistently tracked over time despite the presence of noise. The method was validated on three datasets: segmentation of the left ventricle on simulated echocardiography (Dice coefficient, DC), accuracy of diaphragm motion tracking for healthy subjects (mean sum of distances, MSD) and for a scoliosis patient (root mean square error, RMSE). Results show that the method segments the left ventricle accurately ( ) and robustly tracks the diaphragm motion for healthy subjects ( mm) and for the scoliosis patient ( mm). This method has the potential to segment structures of interest according to their texture in an unsupervised fashion, as well as to help analyze the deformation of tissues. Possible applications are not limited to US image. The same principle could also be applied to other medical imaging modalities such as MRI or CT scans.

A general method for mode decomposition on additive mixture: Generalized Variational Mode Decomposition and its sequentialization

Variational Mode Decomposition(VMD) method was proposed to separate non-stationary signal mixture by solving a optimization problem. This method is powerful and can reconstruct the signal components precisely when they are orthogonal(or quasi-orthogonal) in frequency domain. The crucial problem for VMD is that it requires the information of modal number before the decomposition. Also its applications are limited in 1D and 2D signal processing fields, of narrow scope.In this paper, by inheriting and developing the core idea of VMD, we build a general form for this method and extend it to the modal decomposition for common additive mixture, not only limited in signal processing. To overcome the obstacle of modal number, we sequentialize the generalized VMD method, such that the modes can be extracted one by one, without knowing the modal number a priori. After the generalization and sequentialization for the VMD, we apply them in different fields of additive case, such as texture segmentation, Gaussian Mixture Model(GMM), clustering, etc. From the experiments, we conclude that the generalized and sequentialized VMD methods can solve variety classical problems from the view of modal decomposition, which implies that our methods have higher generality and wider applicability. A raw Matlab code for this algorithm is shown in https://github.com/changwangke/SGVMD_additive_Clustering/blob/main/SGVMD_clustering.m.

Towards Robust Colour Texture Analysis with Limited Training Data

Texture analysis plays an important role in different domains of healthcare, agriculture, and industry, where multi-channel sensors are gaining more attention. This contribution presents an interpretable and efficient framework for texture classification and segmentation that exploits colour or channel information and does not require much data to produce accurate results. This makes such a framework well-suited for medical applications and resource-limited hardware. Our approach builds upon a distance-based generalized matrix learning vector quantization (GMLVQ) algorithm. We extend it with parametrized angle-based dissimilarity and introduce a special matrix format for multi-channel images. Classification accuracy evaluation of various model designs was performed on VisTex and ALOT data, and the segmentation application was demonstrated on an agricultural data set. Our extension of parametrized angle dissimilarity measure leads to better model generalization and robustness against varying lighting conditions than its Euclidean counterpart. The proposed matrix format for multichannel images enhances classification accuracy while reducing the number of parameters. Regarding segmentation, our method shows promising results, provided with a small class-imbalanced training data set. Proposed methodology achieves higher accuracy than prior work benchmarks and a small-scale CNN while maintaining a significantly lower parameter count. Notably, it is interpretable and accurate in scenarios where limited and unbalanced training data are available.

Simulation-trained machine learning models for Lorentz transmission electron microscopy

Understanding the collective behavior of complex spin textures, such as lattices of magnetic skyrmions, is of fundamental importance for exploring and controlling the emergent ordering of these spin textures and inducing phase transitions. It is also critical to understand the skyrmion–skyrmion interactions for applications such as magnetic skyrmion-enabled reservoir or neuromorphic computing. Magnetic skyrmion lattices can be studied using in situ Lorentz transmission electron microscopy (LTEM), but quantitative and statistically robust analysis of the skyrmion lattices from LTEM images can be difficult. In this work, we show that a convolutional neural network, trained on simulated data, can be applied to perform segmentation of spin textures and to extract quantitative data, such as spin texture size and location, from experimental LTEM images, which cannot be obtained manually. This includes quantitative information about skyrmion size, position, and shape, which can, in turn, be used to calculate skyrmion–skyrmion interactions and lattice ordering. We apply this approach to segmenting images of Néel skyrmion lattices so that we can accurately identify skyrmion size and deformation in both dense and sparse lattices. The model is trained using a large set of micromagnetic simulations as well as simulated LTEM images. This entirely open-source training pipeline can be applied to a wide variety of magnetic features and materials, enabling large-scale statistical studies of spin textures using LTEM.

Enhanced Detection of Artisanal Small-Scale Mining with Spectral and Textural Segmentation of Landsat Time Series

Artisanal small-scale mines (ASMs) in the Amazon Rainforest are an important cause of deforestation, forest degradation, biodiversity loss, sedimentation in rivers, and mercury emissions. Satellite image data are widely used in environmental decision-making to monitor changes in the land surface, but ASMs are difficult to map from space. ASMs are small, irregularly shaped, unevenly distributed, and confused (spectrally) with other land clearance types. To address this issue, we developed a reliable and efficient ASM detection method for the Tapajós River Basin of Brazil—an important gold mining region of the Amazon Rainforest. We enhanced detection in three key ways. First, we used the time-series segmentation (LandTrendr) Google Earth Engine (GEE) Application Programming Interface to map the pixel-wise trajectory of natural vegetation disturbance and recovery on an annual basis with a 2000 to 2019 Landsat image time series. Second, we segmented 26 textural features in addition to 5 spectral features to account for the high spatial heterogeneity in ASM pixels. Third, we trained and tested a Random Forest model to detect ASMs after eliminating irrelevant and redundant features with the Variable Selection Using Random Forests “ensemble of ensembles” technique. The out-of-bag error and overall accuracy of the final Random Forest was 3.73 and 92.6%, which are comparable to studies mapping large industrial mines with the normalized difference vegetation index (NDVI) and LandTrendr. The most important feature in our study was NDVI, followed by textural features in the near and shortwave infrared. Our work paves the way for future ASM regulation through large area monitoring from space with free and open-source GEE and operational satellites. Studies with sufficient computational resources can improve ASM monitoring with advanced sensors consisting of spectral narrow bands (Sentinel-2, Environmental Mapping and Analysis Program, PRecursore IperSpettrale della Missione Applicativa) and deep learning.

RGB‐D road segmentation based on cross‐modality feature maintenance and encouragement

AbstractDeep images can provide rich spatial structure information, which can effectively exclude the interference of illumination and road texture in road scene segmentation and make better use of the prior knowledge of road area. This paper first proposes a new cross‐modal feature maintenance and encouragement network. It includes a quantization statistics module as well as a maintenance and encouragement module for effective fusion between multimodal data. Meanwhile, for the problem that if the road segmentation is performed directly using a segmentation network, there will be a lack of supervised guidance with clear physical meaningful information and poor interpretability of learning features, this paper proposes two road segmentation models based on prior knowledge of deep image: disparity information and surface normal vector information. Then, a two‐branch neural network is used to process the colour image and the processed depth image separately, to achieve the full utilization of the complementary features of the two modalities. The experimental results on the KITTI road dataset and Cityscapes dataset show that the method in this paper has good road segmentation performance and high computational efficiency.

A Survey of Vision-Based Methods for Surface Defects’ Detection and Classification in Steel Products

In the competitive landscape of steel-strip production, ensuring the high quality of steel surfaces is paramount. Traditionally, human visual inspection has been the primary method for detecting defects, but it suffers from limitations such as reliability, cost, processing time, and accuracy. Visual inspection technologies, particularly automation techniques, have been introduced to address these shortcomings. This paper conducts a thorough survey examining vision-based methodologies related to detecting and classifying surface defects on steel products. These methodologies encompass statistical, spectral, texture segmentation based methods, and machine learning-driven approaches. Furthermore, various classification algorithms, categorized into supervised, semi-supervised, and unsupervised techniques, are discussed. Additionally, the paper outlines the future direction of research focus.

Multivariate Approach for Texture Segmentation using Probabilistic Statistical Model

The analysis of the regions of the image is of the prerogatives in the fields of medical and global systems meant for location identification. This analysis is strongly associated with partitions of regions of interest such as segmentation. For an effective strategy of analyzing the regions of interest, texture of the image plays a major concern. The texture is generally characterized using signal processing methods namely Discrete Cosine Transformation coefficients and their specific insights leading to feature vector selection. Further, to identify regions, a statistical model needs to be identified for the feature matrix vector and thus make use of Gaussian mixture model with extensions. The Expectation Maximization approach is used, and performance is assessed by experimenting with random images from the Brodatz data store domain. Performance measurements for texture segmentation that can be attributed are Global Cons. Error (GC), Prob. Rand. Index (PR) and Variation of data (VA). These are determined alongside the confusion matrix. To assess the improvement, a comparison was made with other existing models and showed better. The algorithms will be exceptionally helpful for clinical analysis and in radio navigation map systems.

Entropy Kernel Graph Cut Feature Space Enhancement with SqueezeNet Deep Neural Network for Textural Image Segmentation

Recently, image segmentation based on graph cut methods has shown remarkable performance on a set of image data. Although the kernel graph cut method provides good performance, its performance is highly dependent on the data mapping to the transformation space and image features. The entropy-based kernel graph cut method is suitable for segmentation of textured images. Nonetheless, its segmentation quality remains significantly contingent on the accuracy and richness of feature space representation and kernel centers. This paper introduces an entropy-based kernel graph cut method, which leverages the discriminative feature space extracted from SqueezeNet, a deep neural network. The fusion of SqueezeNet’s features enriches the segmentation process by capturing high-level semantic information. Moreover, the extraction of kernel centers is refined through a weighted k-means approach, contributing further to the segmentation’s precision and effectiveness. The proposed method, while exploiting the benefits of suitable computational load of graph cut methods, will be a suitable alternative for segmenting textured images. Laboratory results have been taken on a set of well-known datasets that include textured shapes in order to evaluate the efficiency of the algorithm compared to other well-known methods in the field of kernel graph cut.

QUALITY ASSESSMENT FOR MULTI-RESOLUTION SEGMENTATION AND SEGMENT-ANYTHING MODEL USING WORLDVIEW-3 IMAGERY

Abstract. Image segmentation, is of utmost importance in the disciplines of digital image processing, particularly remote sensing and computer vision, has seen an increasing demand for precise and efficient algorithms. This study focuses to conduct a comparative exploration of the segmentation capabilities of two sophisticated techniques namely, Multiresolution Segmentation (MRS) and Segment Anything Model (SAM), leveraging the high-resolution WorldView-3 (WV-3) satellite image. MRS adopts a hierarchical methodology, segmenting an image into various scales while retaining a profound understanding of its structure. Conversely, SAM employs a deep learning algorithm, prioritizing segment creation based on conceptual pixel similarity, irrespective of spatial adjacency. The WV-3 image, featuring diverse land cover elements like agricultural parcels, industrial structures, roads, red roofs, single trees, and water bodies, serves as the basis for assessing segmentation quality. Both methods are applied to the image, and their outcomes are individually evaluated against manually generated polygonal land use/cover objects. Segmentation quality metrics are employed for assessment. Results reveal MRS effectively preserves fine details and entity delineation, while SAM excels in capturing contextually similar regions. MRS outperforms SAM with a negligible discrepancy, yet SAM demonstrates superiority in the red roof object, achieving an Intersection over Union (IoU) value of 0.70 compared to MRS’s 0.49. MRS tends to generate numerous segments for an item, while SAM produces only one segment. Nevertheless, it is important to recognise that both algorithms have specific constraints in particular scenarios, such as excessive segmentation in areas with abundant texture or inadequate segmentation in areas with slight changes.

An Algorithm for Measuring the Similarity of Histograms for Texture Image Segmentation

A simple algorithm for measuring the similarity between multi-column histograms is presented. The proposed algorithm is intended for texture segmentation of images using histograms as texture features. The purpose of developing such a specialized algorithm is to more accurately determine the boundaries between neighboring texture segments. The algorithm is specially designed so that to express the similarity value as a percentage. The main peculiarity of the proposed algorithm is that when calculating the similarity value, it considers not only the corresponding histogram columns but also takes into account their neighboring components. Due to this, the algorithm more adequately evaluates the similarity of histograms. The proposed algorithm was implemented as a computer program as an integral part of the image segmentation model. The efficiency of the histogram comparison algorithm is indirectly confirmed by the texture segmentation results of the image segmentation model in image processing experiments.

Foam Segmentation in Wastewater Treatment Plants

The excessive accumulation of foam in wastewater treatment plant (WWTP) tanks can impede proper aeration, hindering the effective removal of organic matter from the water. This study proposes a novel technique to monitor in real time the presence of foams in WWTP tanks by using texture segmentation models trained with centralized and federated approaches. These models are designed to segment the foam and quantify the percentage of foam coverage across the entire tank surface. This data provides plant operators with crucial information for identifying the optimal time for foam removal. The proposed methodology is integrated into an image processing pipeline that involves acquiring images using a PTZ camera, ensuring the absence of anomalies in the captured images, and implementing a real-time communication method for event notifications to plant operators. The models exhibit noteworthy performance, achieving an 86% Dice score in foam segmentation, with comparable results obtained through both centralized and federated training. Implemented in a wastewater treatment plant, this integrated pipeline enhances operational efficiency while concurrently reducing costs.

A replication and re-analysis of a classic texture segmentation study.

A classic finding reported in Beck (1966a) is that observers tend to indicate a more natural texture break between a set of T's and tilted T's than between a set of T's and backward L's. This finding has played a prominent role in discussions about the properties of texture segmentation and in the development of computational theories of texture segmentation. Due to the small sample size of the original study, we replicated the original experiment with a larger sample. Regrettably, we discovered that the description in Beck (1966a) is insufficient to allow us to reproduce the stimuli. For our replication, we created stimuli that seem consistent with the spirit of the original study. The results of the replication study partly match Beck's original data ( ), but conclusions indicated by these results deviate from some of Beck's key conclusions. We also further explored the influential claim derived from a second experiment in Beck (1966a) that texture segmentation was not related to the perceived similarity of elements. Contrary to this claim, a re-analysis of Beck's data indicates the counterintuitive conclusion that observers tend to indicate stronger texture segmentation between regions that are rated as being more similar. In a replication of the second experiment, this relation flipped so that the replication observers indicated stronger texture segmentation for regions with lower similarity ratings. Despite being influential, we conclude that there are substantial problems with the stimuli, analyses, and conclusions in Beck (1966a).

TEXTURE-BASED SEPARATION TO REFINE BUILDING MESHES

Abstract. In this paper, we present a texture-based separation approach to refine building meshes, which aims to address the challenges of detecting and isolating different objects in an indoor scene mesh. We propose a novel segmentation model based on the materials of the different parts of the scene. The proposed approach uses factorization-based texture segmentation to separate the different materials in the meshes and detect the edges on the segmented texture. To prevent large faces containing multiple materials from being segmented wrongly, the mesh is cut along the texture boundaries and new faces with a single material are created. Finally, we segment the new faces based on the material index and neighbouring faces using a region-growing algorithm. We evaluate the proposed approach on the Matterport indoor dataset, which shows that our approach performs well on detecting boundaries between distinct materials, but over-segments on complex shapes. Our proposed method improves the segmentation of large flat surfaces like posters or rugs.

Evaluation of land roughness and weather effects on paddy field using cameras mounted on drone: A comprehensive analysis from early to mid-growth stages

This paper proposes a potential solution to the traditional method of rice field preparation and management by utilizing unmanned aerial vehicles (UAVs) equipped with RGB and spectral cameras. Farmers typically rely on their experience to deal with bad weather and prepare rice fields in the beginning of the season, resulting in time-consuming and inefficient processes. The proposed method uses UAVs to evaluate the rice fields from the start to the middle stages of the season. The UAV flight follows a pre-determined trajectory autonomously, capturing successive images using the attached cameras. In the soil preparation stage, Harralick texture segmentation is used to analyze the roughness of paddy fields’ surfaces (low land, normal land and high land). In this approach, image is segmented first, then areas belonging the same clusters are extracted, manually verified and labelled. These labelled images are used as training data for the proposed model. The result shows that roughness of field surface can be observed clearly with high accuracy at over 90%, with random forest method is at over 97%. In addition to this, the effects of rain and flooding are also considered during the reproductive stages. The results show that the proposed method achieves high accuracy in evaluating rice fields, which can significantly increase efficiency and productivity compared to traditional methods. The study highlights the potential of UAV-based monitoring to provide a cost-effective and efficient way to manage rice fields and improve crop yields, leading to higher food security and sustainable agriculture.

Повышение эффективности текстурной сегментации лесного полога по изображениям сверхвысокого пространственного разрешения

The issue of increasing the accuracy of tree canopy segmentation structures in the processing of very high spatial resolution satellite images by joint use of textural features extracted by different methods is considered. The TTSPCA method (Total Texture Statistics Principal Component Analysis) is proposed and its effectiveness is shown for a number of test cases. Estimates of TTSPCA effectiveness are obtained for the forest stands segmentation and growth types. The panchromatic Worldview-2 image of the test area (Bronnitskoye forestry, Moscow region) obtained in summer was used as remote data. To perform texture segmentation, in addition to TTSPCA, several standard second-order statistical methods and a spectral (energy) method based on the wavelet transform were also used. It is shown that almost all considered statistical and spectral methods provide forest stand segmentation with errors not exceeding 3–4 %. It has been established that the TTSPCA method makes it possible to reduce the probability of errors in the forest inventory zone, as well as to identify sections with a predominance of natural and artificial plantations with an accuracy of over 85 %. The results obtained can be further recommended for use in order to improve the system we are developing for joint spectral-textural processing of satellite images with different spatial resolutions.

Measuring uncertainty in human visual segmentation.

Segmenting visual stimuli into distinct groups of features and visual objects is central to visual function. Classical psychophysical methods have helped uncover many rules of human perceptual segmentation, and recent progress in machine learning has produced successful algorithms. Yet, the computational logic of human segmentation remains unclear, partially because we lack well-controlled paradigms to measure perceptual segmentation maps and compare models quantitatively. Here we propose a new, integrated approach: given an image, we measure multiple pixel-based same-different judgments and perform model-based reconstruction of the underlying segmentation map. The reconstruction is robust to several experimental manipulations and captures the variability of individual participants. We demonstrate the validity of the approach on human segmentation of natural images and composite textures. We show that image uncertainty affects measured human variability, and it influences how participants weigh different visual features. Because any putative segmentation algorithm can be inserted to perform the reconstruction, our paradigm affords quantitative tests of theories of perception as well as new benchmarks for segmentation algorithms.

Texture and pixel intensity characterization-based image segmentation with morphology and watershed techniques

<p>Image segmentation is an image processing technique that concentrated on finding and locating the parts of an image such as objects and boundaries. The purpose of locating these parts is for use in further processing analysis of an image such as recognition tasks, and content-based image retrieval. This paper introduces the segmentation procedure using a proposed template of features with watershed or morphology operations. Features template based on segmentation process conveys pixels’ intensities property perceived by the threshold value of histogram representation and texture feature where the regions are characterized by their texture content using standard deviation (SD) filtering. Wiener filter and histogram equalization (HE) techniques are used as preprocessing operations to enhance the image quality. The edge detector operator is hybridized to boost the segmentation process. Some statistical metrics are used for assessing and analyzing the performance of the stages in the proposed work. As a result, this proposed template of features achieved more performance with watershed and morphology segmentation.</p>