Background Texture Research Articles

A Classification and Segmentation Model for Diamond Abrasive Grains Based on Improved Swin-Unet-SAM

The detection of abrasive grain images in diamond tools serves as the foundation for assessing the overall condition of the tools, encompassing crucial aspects of diamond abrasive grains like the quantity, size, morphology, and distribution. Given the intricate background textures and reflective characteristics exhibited by diamond images, diamond detection and segmentation pose a significant challenge. Recently, numerous defect detection methods based on machine learning and deep learning have emerged. However, several issues persist, such as detection accuracy and the interference caused by intricate background textures. The present work demonstrates an efficient classification and segmentation network algorithm that combines Swin-Unet with SAM (Segment Anything Model) to alleviate the existing problems. Specifically, four embedding structures were devised to bridge the two models for iterative training. The transformer blocks within the Swin-Unet model were enhanced to facilitate classification and coarse segmentation, and the mask structure in SAM was refined to enable fine segmentation. The experimental results show that under a small sample dataset with complex background textures, the average index values of ACC (accuracy), SE (Sensitivity), and DSC (Dice Similarity Coefficient) for the classification and segmentation of diamond abrasive grains reached 98.7%, 92.5%, and 85.9%, respectively. Compared with the model before improvement, its ACC, SE and DSC increased by 1.2%, 15.9%, and 7.6%, respectively. The test results, based on four different datasets, consistently indicated that this model has excellent segmentation performance and robustness and has great application potential in the industrial field.

PBNet: Combining Transformer and CNN in Passport Background Texture Printing Image Classification

Passport background texture classification has always been an important task in border checks. Current manual methods struggle to achieve satisfactory results in terms of consistency and stability for weakly textured background images. For this reason, this study designs and develops a CNN and Transformer complementary network (PBNet) for passport background texture image classification. We first design two encoders by Transformer and CNN to produce complementary features in the Transformer and CNN domains, respectively. Then, we cross-wisely concatenate these complementary features to propose a feature enhancement module (FEM) for effectively blending them. In addition, we introduce focal loss to relieve the overfitting problem caused by data imbalance. Experimental results show that our PBNet significantly surpasses the state-of-the-art image segmentation models based on CNNs, Transformers, and even Transformer and CNN combined models designed for passport background texture image classification.

Polycrystalline silicon photovoltaic cell defects detection based on global context information and multi-scale feature fusion in electroluminescence images

In photovoltaic (PV) cell inspection, electroluminescence (EL) imaging provides high spatial resolution for detecting various types of defects. The recent integration of EL imaging with deep learning models has enhanced the recognition of defects in PV cells. However, the high surface impurity content in polycrystalline silicon PV cells presents a challenge. Defect features can be similar to complex background textures, making highlighting features for small target defects difficult and leading to potential misclassification as background or other classes. To address these challenges, we propose a novel deep convolutional neural network (CNN) model for effectively identifying small target defects in polycrystalline PV cells. We first utilize a global context information (GCI) block to improve CNN’s modeling of global information, aiding in distinguishing PV cell defects with similar local details. Moreover, we design a channel weight feature pyramid (CWFP) that dynamically adjusts the channel weights of extracted features. We further achieve multi-scale feature fusion through the pyramid structure to enhance the resolution capability for small targets. The proposed model was evaluated on a publicly available dataset of 8 defect classes in polycrystalline PV cells, achieving an accuracy of 96.36 %. The experimental results show that the proposed model outperforms existing deep learning models in detecting small target defects with higher precision.

Investigating the Influence of Wavelength-Specific Textured Backgrounds in Background Oriented Schlieren (BOS) Imaging

This study investigates the influence of wavelength-specific textured backgrounds on the effectiveness of Background-Oriented Schlieren (BOS) imaging, focusing on wavelengths from 400 nm to 670 nm at intervals of 30 nm intervals and multiple captured recordings for each background wavelength interval. By analyzing the signal-to-noise ratio (SNR) computationally, and the image gradient magnitude, we aimed to determine the optimal wavelengths for capturing turbulence and determine the effectiveness of colored backgrounds in natural external environments for schlieren. The SNR, calculating the ratio of mean signal intensity to noise standard deviation, revealed the highest value at 550 nm (SNR = 22.8), indicating maximized clarity. Similarly, image gradient magnitude, computed using the Sobel operator to assess spatial intensity changes, peaked at 550 nm (G=52.3), confirming effective turbulence visualization. Our findings align with the Bayer color filter trend, suggesting that the green spectrum is particularly advantageous for BOS imaging. Deviations at 490 nm and 580 nm, characterized by lower SNR and gradient magnitude, could be attributed to atmospheric scattering, refractive index overlap, or slight digital video capture differences., highlighting environmental factors that can influence imaging performance and value variation. These insights emphasize the importance of wavelength selection and background design in real-world BOS applications, suggesting that while 550 nm provides optimal results, further refinement may enhance the effectiveness of other wavelengths.

Factors Affecting Stimulus Duration Threshold for Depth Discrimination of Asynchronous Targets in the Intermediate Distance Range.

Binocular depth discrimination in the near distance range (< 2 m) improves with stimulus duration. However, whether the same response-pattern holds in the intermediate distance range (approximately 2-25 m) remains unknown because the spatial coding mechanisms are thought to be different. We used the two-interval forced choice procedure to measure absolute depth discrimination of paired asynchronous targets (3, 6, or 16 arc min). The paired targets (0.2 degrees) were located over a distance and height range, respectively, of 4.5 to 7.0m and 0.15 to 0.7 m. Experiment 1 estimated duration thresholds for binocular depth discrimination at varying target durations (40-1610 ms), in the presence of a 2 × 6 array of parallel texture-elements spanning 1.5 × 5.83 m on the floor. The texture-elements provided a visible background in the light-tight room (9 × 3 m). Experiment 2 used a similar setup to control for viewing conditions: binocular versus monocular and with versus without texture background. Experiment 3 compared binocular depth discrimination between brief (40, 80, and 125 ms) and continuous texture background presentation. Stimulus duration threshold for depth discrimination decreased with increasing disparity in experiment 1. Experiment 2 revealed depth discrimination performance with texture background was near chance level with monocular viewing. Performance with binocular viewing degraded without texture background. Experiment 3 showed continuous texture background presentation enhances binocular depth discrimination. Absolute depth discrimination improves with target duration, binocular viewing, and texture background. Performance further improved with longer background duration underscoring the role of ground surface representation in spatial coding.

Learning graph-Fourier spectra of textured surface images for defect localization

In the realm of industrial manufacturing, product inspection remains a significant bottleneck, with only a small fraction of manufactured items undergoing inspection for surface defects. Advances in imaging systems and AI can allow automated full inspection of manufactured surfaces. However, even the most contemporary imaging and machine learning methods perform poorly for detecting defects in images with highly textured backgrounds, that stem from diverse manufacturing processes. This paper introduces an approach based on graph Fourier analysis to automatically identify defective images, as well as crucial graph Fourier coefficients that inform the defects in the images. The approach thereby facilitates precise localization and characterization of defects, amidst highly textured backgrounds. A convolutional neural network model (1D-CNN) was trained with the coefficients of the graph Fourier transform of the images as the input to identify, with classification accuracy of 99.4%, if the image contains a defect. An explainable AI method using SHAP (SHapley Additive exPlanations) was used to further analyze the trained 1D-CNN model to discern important spectral coefficients for each image. This approach sheds light on the crucial contribution of low-frequency graph eigen waveforms to precisely localize surface defects in images, thereby advancing the realization of zero-defect manufacturing.

Multi-scale feature reconstruction network for industrial anomaly detection

Unsupervised anomaly detection techniques, which operate without prior knowledge of anomalies, have garnered significant attention in industrial inspection due to their adaptability and generalization. Therefore, knowledge-based computer vision techniques have been broadly applied to identify unusual image patterns. However, real-time industrial applications present challenges such as limited anomalous samples, inadequate defect knowledge, and complex background textures. These factors lead to difficulties in accurately identifying defect regions, and conventional auto-encoder networks often struggle to overcome these issues.To address these limitations, we propose a multi-scale feature reconstruction (MSFR) network specifically designed for domain shift scenarios. Our approach employs a pyramidal vision transformer network (PVTN) to reconstruct multi-scale feature maps, capturing discriminative features at various scales. Additionally, a pre-trained module extracts multi-level features at the same scale, and a dedicated feature matching module enhances accuracy by improving the alignment probability between features. The MSFR strategy surpasses conventional auto-encoders by filtering pixel-level information at multiple depths.Empirical evaluations were conducted using benchmark datasets such as MVTec AD and AeBAD-S. Furthermore, an extensive ablation study demonstrates the effectiveness and viability of the proposed MSFR approach for industrial anomaly detection tasks. The experimental results show that the proposed model significantly outperforms recent approaches, making it highly suitable for real-world industrial applications, particularly in manufacturing.

Asymmetric Network Combining CNN and Transformer for Building Extraction from Remote Sensing Images.

The accurate extraction of buildings from remote sensing images is crucial in fields such as 3D urban planning, disaster detection, and military reconnaissance. In recent years, models based on Transformer have performed well in global information processing and contextual relationship modeling, but suffer from high computational costs and insufficient ability to capture local information. In contrast, convolutional neural networks (CNNs) are very effective in extracting local features, but have a limited ability to process global information. In this paper, an asymmetric network (CTANet), which combines the advantages of CNN and Transformer, is proposed to achieve efficient extraction of buildings. Specifically, CTANet employs ConvNeXt as an encoder to extract features and combines it with an efficient bilateral hybrid attention transformer (BHAFormer) which is designed as a decoder. The BHAFormer establishes global dependencies from both texture edge features and background information perspectives to extract buildings more accurately while maintaining a low computational cost. Additionally, the multiscale mixed attention mechanism module (MSM-AMM) is introduced to learn the multiscale semantic information and channel representations of the encoder features to reduce noise interference and compensate for the loss of information in the downsampling process. Experimental results show that the proposed model achieves the best F1-score (86.7%, 95.74%, and 90.52%) and IoU (76.52%, 91.84%, and 82.68%) compared to other state-of-the-art methods on the Massachusetts building dataset, the WHU building dataset, and the Inria aerial image labeling dataset.

MT-U2Net: Lightweight detection network for high-precision magnetic tile surface defect localization

Automated surface defect localization is closely related to product quality control. Although many image segmentation models have achieved excellent results in industrial defect localization, the wide-ranging defects in magnetic tile materials and the similarity between textures and background textures make it difficult to balance detection model performance and model size. Therefore, we propose a new magnetic tile defect localization model called MT-U2Net. The model consists of three parts: Dres-Ublock, U2 structure, and attention mechanism. Dres-Ublock is used to enhance the model's feature extraction capability and reduce its size; the U2-Net structure is utilized to learn cross-scale defect information; the attention mechanism is employed to eliminate redundant information. MT-U2Net improves detection performance across various defect scales. In the test set, MT-U2Net achieves an F1 Score of 0.840 and an MAE of 0.024 and compresses the model size to 3.39 M. Extensive experimental results show that MT-U2Net outperforms existing state-of-the-art algorithms in magnetic tile defect detection performance and model size.

Fabric Defect Detection Based on Improved Lightweight YOLOv8n

In response to the challenges posed by complex background textures and limited hardware resources in fabric defect detection, this study proposes a lightweight fabric defect detection algorithm based on an improved GSL-YOLOv8n model. Firstly, to reduce the parameter count and complexity of the YOLOv8n network, the GhostNet concept is used to construct the C2fGhost module, replacing the conventional convolution layers in the YOLOv8n structure with Ghost convolutions. Secondly, the SimAM parameter-free attention mechanism is embedded at the end of the backbone network to eliminate redundant background, enhance semantic information for small targets, and improve the network’s feature extraction capability. Lastly, a lightweight shared convolution detection head is designed, employing the scale layer to adjust features, ensuring the lightweight nature of the model while minimizing precision loss. Compared to the original YOLOv8n model, the improved GSL-YOLOv8n algorithm increases the mAP@0.5 by 0.60% to 98.29% and reduces model size, computational load, and parameter count by 66.7%, 58.0%, and 67.4%, respectively, meeting the application requirements for fabric defect detection in textile industry production.

Enhanced Scratch Detection for Textured Materials Based on Optimized Photometric Stereo Vision and Fast Fourier Transform–Gabor Filtering

In the process of scratch defect detection in textured materials, there are often problems of low efficiency in traditional manual detection, large errors in machine vision, and difficulty in distinguishing defective scratches from the background texture. In order to solve these problems, we developed an enhanced scratch defect detection system for textured materials based on optimized photometric stereo vision and FFT-Gabor filtering. We designed and optimized a novel hemispherical image acquisition device that allows for selective lighting angles. This device integrates images captured under multiple light sources to obtain richer surface gradient information for textured materials, overcoming issues caused by high reflections or dark shadows under a single light source angle. At the same time, for the textured material, scratches and a textured background are difficult to distinguish; therefore, we introduced a Gabor filter-based convolution kernel, leveraging the fast Fourier transform (FFT), to perform convolution operations and spatial domain phase subtraction. This process effectively enhances the defect information while suppressing the textured background. The effectiveness and superiority of the proposed method were validated through material applicability experiments and comparative method evaluations using a variety of textured material samples. The results demonstrated a stable scratch capture success rate of 100% and a recognition detection success rate of 98.43% ± 1.0%.

Structure Embedded Nucleus Classification for Histopathology Images.

Nuclei classification provides valuable information for histopathology image analysis. However, the large variations in the appearance of different nuclei types cause difficulties in identifying nuclei. Most neural network based methods are affected by the local receptive field of convolutions, and pay less attention to the spatial distribution of nuclei or the irregular contour shape of a nucleus. In this paper, we first propose a novel polygon-structure feature learning mechanism that transforms a nucleus contour into a sequence of points sampled in order, and employ a recurrent neural network that aggregates the sequential change in distance between key points to obtain learnable shape features. Next, we convert a histopathology image into a graph structure with nuclei as nodes, and build a graph neural network to embed the spatial distribution of nuclei into their representations. To capture the correlations between the categories of nuclei and their surrounding tissue patterns, we further introduce edge features that are defined as the background textures between adjacent nuclei. Lastly, we integrate both polygon and graph structure learning mechanisms into a whole framework that can extract intra and inter-nucleus structural characteristics for nuclei classification. Experimental results show that the proposed framework achieves significant improvements compared to the previous methods. Code and data are made available via https://github.com/lhaof/SENC.

Infrared and visible image fusion based on semi-global weighted least squares and guided edge-aware filters

Most of the existing image fusion methods are dedicated to extracting the respective private features in the source image to reconstruct to get a fused image that contains rich information. However, its neglect of the enhancement of texture details in the source image as well as the preservation of background structures, which makes the fusion result look less impressive. To this end, an infrared and visible image fusion method based on a semi-global weighted least squares (SGWLS) method and guided edge-aware filter (GEAF) is proposed. First, a SGWLS method is applied to decompose the source image into base and detail layers containing different scale information. Then, the detail layer are texture-enhanced by using the scale coefficients. Due to the lack of edge features in the original base layer, for this reason, a guided edge-aware filter is designed, in which the enhanced detail layer was used as the guidance image as a way to perform a quadratic feature enhancement fusion for the base layer in order to retain richer gradient information. Finally, reconstruction of the fused detail and base layers by using inverse transformation to obtain the fusion result. Compared with nine state-of-the-art fusion algorithms, a large number of experimental results demonstrate the superior performance of the proposed method in highlighting texture details and background features, as well as the obvious advantages in SF and AG metrics.

Mask-DerainGAN: Learning to remove rain streaks by learning to generate rainy images

Image deraining with unpaired data has been a challenging problem. Previous methods suffer from either the color distortion artifacts, due to the pixel-level cycle consistency loss, or the time-consuming training process. To address these problems, in this paper, we propose a novel method for rain removal based on using unpaired data. First, we obtain a rain streak mask from the derained result, which serves as a guidance for generating rainy images. Both the mask and the rain-free image are then fed into the proposed generator to obtain a high-quality rainy image, which implicitly helps improve the rain removal performance. In this way, the proposed learning framework simultaneously learns rain removal and rain generation in order to produce high-quality rain-free images and rainy images. Second, we propose a contrastive learning generator to preserve background texture details and ensure semantic consistency between the generated rain-free image and the original input. Experimental results demonstrate that our method surpasses most state-of-the-art unsupervised methods on multiple benchmark synthetic and real datasets.

PaveSAM – segment anything for pavement distress

Automated pavement monitoring using computer vision can analyse pavement conditions more efficiently and accurately than manual methods. Accurate segmentation is essential for quantifying the severity and extent of pavement defects and consequently, the overall condition index used for prioritising maintenance activities. Deep-learning-based segmentation models are however, often supervised and require pixel-level annotations, which can be costly and time-consuming. While the recent evolution of zero-shot segmentation models can generate pixel-wise labels for unseen classes without any training data, they struggle with irregularities of cracks and textured pavement backgrounds. This research proposes a zero-shot segmentation model, PaveSAM, which can segment pavement distresses using bounding box prompts. By retraining SAM's mask decoder with just 180 images, pavement distress segmentation is revolutionised, enabling efficient distress segmentation using boundingbox prompts, a capability not found in current segmentation models. This drastically reduces labelling costs and establishing the pioneering use of SAM in pavement distress segmentation.

Multi-Scale Marine Object Detection in Side-Scan Sonar Images Based on BES-YOLO.

Aiming at the problem of low accuracy of multi-scale seafloor target detection in side-scan sonar images with high noise and complex background texture, a model for multi-scale target detection using the BES-YOLO network is proposed. First, an efficient multi-scale attention (EMA) mechanism is used in the backbone of the YOLOv8 network, and a bi-directional feature pyramid network (Bifpn) is introduced to merge the information of different scales, finally, a Shape_IoU loss function is introduced to continuously optimize the model and improve its accuracy. Before training, the dataset is preprocessed using 2D discrete wavelet decomposition and reconstruction to enhance the robustness of the network. The experimental results show that 92.4% of the mean average accuracy at IoU of 0.5 (mAP@0.5) and 67.7% of the mean average accuracy at IoU of 0.5 to 0.95 (mAP@0.5:0.95) are achieved using the BES-YOLO network, which is an increase of 5.3% and 4.4% compared to the YOLOv8n model. The research results can effectively improve the detection accuracy and efficiency of multi-scale targets in side-scan sonar images, which can be applied to AUVs and other underwater platforms to implement intelligent detection of undersea targets.

PLM-Res-U-Net: A light weight binarization model for enhancement of multi-textured palm leaf manuscript images

This paper proposes a deep semantic binarization model, PLM-Res-U-Net, for enhancing palm-leaf manuscripts. PLM-Res-U-Net is a lightweight model comprising encoding and decoding blocks with skip connections. The model enhances the palm leaf manuscript by efficiently retaining the text strokes by removing the degradations such as uneven illumination, aging marks, brittleness, and background discolorations. Two datasets of palm leaf manuscript collections with multiple degradation patterns and diverse textured backgrounds are used for experimentation. PLM-Res-U-Net is trained from scratch with 50 epochs with a learning rate of1e−8 with three sampling strategies. The performance of state-of-the-art deep learning models ResUnet, Pspnet, U-Net++, and Segnet are also evaluated along with two diverse benchmark datasets. Analysis shows that results obtained by the proposed PLM-Res-U-Net prove generalizability and computational efficacy with a dice score of 0.986. Additionally, PLM-Res-U-Net successfully preserves the edge strokes of the text compared with state-of-the-art models.

RADDA-Net: Residual attention-based dual discriminator adversarial network for surface defect detection

Surface defect detection is a critical yet challenging task in the product production process, mainly because of the complex background texture on the material surface, the large intra-class defect differences, and the weak texture defect regions. To address this problem, a Residual Attention-based Dual Discriminator Adversarial Network (RADDA-Net) is developed to accurately detect various types of texture defect regions. Specifically, a residual network-based feature extraction module integrating multi-channel attention mechanism and multi-scale convolutional layers is designed to improve the detection ability of abnormal defects. Besides, to deeply excavate material surface texture information, the pixel attention-based up-sampling blocks are also incorporated into RADDA-Net to enhance the extraction of shallow features. To further improve the detection performance of RADDA-Net, a dual discriminant adversarial training strategy and a comprehensive measurement method are introduced to assist the developed network to identify defects from both macroscopic structure and microscopic edges. Finally, the six evaluation indexes of our proposed method outperformed other state-of-the-art methods on the four defect datasets, among which the highest detection precision, recall, and F-measure are 0.926, 0.916, 0.920, respectively. Therefore, the experimental results consistently verify the detection accuracy and effectiveness of RADDA-Net.

The transmission line foreign body detection algorithm based on weighted spatial attention.

The secure operation of electric power transmission lines is essential for the economy and society. However, external factors such as plastic film and kites can cause damage to the lines, potentially leading to power outages. Traditional detection methods are inefficient, and the accuracy of automated systems is limited in complex background environments. This paper introduces a Weighted Spatial Attention (WSA) network model to address the low accuracy in identifying extraneous materials within electrical transmission infrastructure due to background texture occlusion. Initially, in the model preprocessing stage, color space conversion, image enhancement, and improved Large Selective Kernel Network (LSKNet) technology are utilized to enhance the model's proficiency in detecting foreign objects in intricate surroundings. Subsequently, in the feature extraction stage, the model adopts the dynamic sparse BiLevel Spatial Attention Module (BSAM) structure proposed in this paper to accurately capture and identify the characteristic information of foreign objects in power lines. In the feature pyramid stage, by replacing the feature pyramid network structure and allocating reasonable weights to the Bidirectional Feature Pyramid Network (BiFPN), the feature fusion results are optimized, ensuring that the semantic information of foreign objects in the power line output by the network is effectively identified and processed. The experimental outcomes reveal that the test recognition accuracy of the proposed WSA model on the PL (power line) dataset has improved by three percentage points compared to that of the YOLOv8 model, reaching 97.6%. This enhancement demonstrates the WSA model's superior capability in detecting foreign objects on power lines, even in complex environmental backgrounds. The integration of advanced image preprocessing techniques, the dynamic sparse BSAM structure, and the BiFPN has proven effective in improving detection accuracy and has the potential to transform the approach to monitoring and maintaining power transmission infrastructure.

Automated detection of pavement distress based on enhanced YOLOv8 and synthetic data with textured background modeling

Regular inspections of pavement distress are essential for accident prevention, and deep learning based algorithms have been developed to ensure high accuracy of inspections. However, the lack of available data is a critical challenge for existing algorithms. To address this issue, an elaborate image synthesis strategy is proposed. By combining textured background modeling for real pavement and Unreal Engine-based distress block stitching technique, high-resolution virtual images that are indistinguishable from real images are generated, including five types of pavement distresses. In addition, an enhanced YOLOv8 network utilizing synthetic data is designed in this paper. The enhanced YOLOv8 network is embedded with the Squeeze and Excitation attention module, and the Swin Transformer module, which are designed to distinguish different types of pavement distress accurately and suppress the interference of complex backgrounds (noise, shadow, blur). The results show that the performance of the algorithms trained based on appropriately ratios of synthetic data (2:1 ratio of virtual to real) improves by more than 10% compared with no virtual data. The enhanced YOLOv8 network achieves a Mean Average Precision (MAP) of 94.8% for transverse cracks, longitudinal cracks, cross-cracks, alligator cracks, and potholes, which is better than seven existing object detection models. The proposed image synthesis method can contribute to improving the accuracy and reliability of pavement inspection and alleviate the reliance on large number of distress samples collection.