Texture Fusion Research Articles

Frequency‐Aware Facial Image Shadow Removal through Skin Color and Texture Learning

AbstractExisting facial image shadow removal methods predominantly rely on pre‐extracted facial features. However, these methods often fail to capitalize on the full potential of these features, resorting to simplified utilization. Furthermore, they tend to overlook the importance of low‐frequency information during the extraction of prior features, which can be easily compromised by noises. In our work, we propose a frequency‐aware shadow removal network (FSRNet) for facial image shadow removal, which utilizes the skin color and texture information in the face to help recover illumination in shadow regions. Our FSRNet uses a frequency‐domain image decomposition network to extract the low‐frequency skin color map and high‐frequency texture map from the face images, and applies a color‐texture guided shadow removal network to produce final shadow removal result. Concretely, the designed fourier sparse attention block (FSABlock) can transform images from the spatial domain to the frequency domain and help the network focus on the key information. We also introduce a skin color fusion module (CFModule) and a texture fusion module (TFModule) to enhance the understanding and utilization of color and texture features, promoting high‐quality result without color distortion and detail blurring. Extensive experiments demonstrate the superiority of the proposed method. The code is available at https://github.com/laoxie521/FSRNet.

Supervised learning-based artificial senses for non-destructive fish quality classification

Human sensory techniques are inadequate for automating fish quality monitoring and maintaining controlled storage conditions throughout the supply chain. The dynamic monitoring of a single quality index cannot anticipate explicit freshness losses, which remarkably drops consumer acceptability. For the first time, a complete artificial sensory system is designed for the early detection of fish quality prediction. At non-isothermal storages, the rainbow trout quality is monitored by the gas sensors, texturometer, pH meter, camera, and TVB-N analysis. After data preprocessing, correlation analysis identifies the key parameters such as trimethylamine, ammonia, carbon dioxide, hardness, and adhesiveness to input into a back-propagation neural network. Using gas and textural key parameters, around 99 % prediction accuracy is achieved, precisely classifying fresh and spoiled classes. The regression analysis identifies a few gaps due to fewer datasets for model training, which can be reduced using few-shot learning techniques in the future. However, the multiparametric fusion of texture with gases enables early freshness loss detection and shows the capacity to automate the food supply chain completely.

Rapid and nondestructive identification of rice storage year using hyperspectral technology

Rice is the main staple food for more than half of the world's population. Consumption of long-stored rice will have adverse effects on the human body. Here, we proposed the near-infrared (NIR) hyperspectral imaging (HSI) technique to distinguish rice from different storage years. Multiplicative Scatter Correction (MSC), Standard Normalize Variate (SNV) and 1st Derivative (1st) were used for the pretreatment of HSI data. In order to reduce dimensional spectral features, Principal Component Analysis (PCA) and t-distributed Stochastic Neighbor Embedding (tSNE) were used for data visualization. Besides, spectral characteristic wavelengths were extracted by competitive adaptive reweighted sampling (CARS) and least absolute shrinkage and selection operator (Lasso). Simultaneously, the textural features of rice were analyzed by Local Binary Pattern (LBP), Gray-Level Co-occurrence Matrix (GLCM) and Tamura algorithms. In order to realize feature fusion of spectra and texture, Support Vector Machine (SVM) model was optimized using the Whale optimization algorithm (WOA) and Extreme Gradient Boosting (XGBoost) model were established based on spectral features and textural features. Compared to other models, feature fusion model showed an excellent result with an accuracy of 98.89%. Experimental results suggested that HSI technology can be served as an effective method for rice detection.

Flotation froth image enhancement based on region decomposition and guided filtering

Extraction of information from froth images is important for automatic control of froth flotation. However, images captured by cameras often suffer from severe uneven lighting, which significantly reduces the quality of froth images. Low-quality images hinder the accurate extraction of froth information, thereby affecting the control of the froth flotation system. Hence, we propose an image enhancement method based on region decomposition and guided filtering to improve the quality of images. Initially, we separate the image into regions with sufficient and insufficient illumination based on reflectance. In regions with insufficient illumination, the guided filter is applied to direct pixels to acquire information from brighter points in their neighborhood. Conversely, in other regions, we regulate the magnitude of pixel variations to prevent overexposure. Finally, a detail enhancement method is proposed based on a multi-scale Gaussian pyramid and texture fusion to improve clarity and naturalness. The experiments show that the method we proposed surpasses several state-of-the-art algorithms on public datasets. In the field of flotation, our method effectively enhances the image quality. Compared to other enhancement methods under the same segmentation strategy, our method significantly improves segmentation accuracy, demonstrating its strong practical value. In addition, our method also shows advantages in terms of computational speed.

A CNN model for early detection of pepper Phytophthora blight using multispectral imaging, integrating spectral and textural information

BackgroundPepper Phytophthora blight is a devastating disease during the growth process of peppers, significantly affecting their yield and quality. Accurate, rapid, and non-destructive early detection of pepper Phytophthora blight is of great importance for pepper production management. This study investigated the possibility of using multispectral imaging combined with machine learning to detect Phytophthora blight in peppers. Peppers were divided into two groups: one group was inoculated with Phytophthora blight, and the other was left untreated as a control. Multispectral images were collected at 0-h samples before inoculation and at 48, 60, 72, and 84 h after inoculation. The supporting software of the multispectral imaging system was used to extract spectral features from 19 wavelengths, and textural features were extracted using a gray-level co-occurrence matrix (GLCM) and a local binary pattern (LBP). The principal component analysis (PCA), successive projection algorithm (SPA), and genetic algorithm (GA) were used for feature selection from the extracted spectral and textural features. Two classification models were established based on effective single spectral features and significant spectral textural fusion features: a partial least squares discriminant analysis (PLS_DA) and one-dimensional convolutional neural network (1D-CNN). A two-dimensional convolutional neural network (2D-CNN) was constructed based on five principal component (PC) coefficients extracted from the spectral data using PCA, weighted, and summed with 19-channel multispectral images to create new PC images.ResultsThe results indicated that the models using PCA for feature selection exhibit relatively stable classification performance. The accuracy of PLS-DA and 1D-CNN based on single spectral features is 82.6% and 83.3%, respectively, at the 48h mark. In contrast, the accuracy of PLS-DA and 1D-CNN based on spectral texture fusion reached 85.9% and 91.3%, respectively, at the same 48h mark. The accuracy of the 2D-CNN based on 5 PC images is 82%.ConclusionsThe research indicates that Phytophthora blight infection can be detected 48 h after inoculation (36 h before visible symptoms). This study provides an effective method for the early detection of Phytophthora blight in peppers.

Building detection in SAR images based on fusion of classic and deep learning features

ABSTRACT Because of high-resolution imaging in day and night and any weather condition, Synthetic Aperture Radar (SAR) has various applications in remote sensing. Building detection is one of the important utilization of SAR images. With analysing SAR images information, a new framework based on fusion of statistical, texture, and semantic features is proposed. At first, in order to reduce the speckle noise effects on the classification results, the SAR image is segmented into superpixels which the classification process is performed for these segments. Then, the statistical Fisher and Haralick texture features are extracted and the best features are selected. By applying an adapted VGGNet, the third type of features are extracted. Using training samples, the features achieving the highest classification accuracy are selected and applied to the classifier. Then, the classification output is improved by decreasing the splitting of the large-size buildings and false alarms using morphological operations and contour fitting process. The performance of the proposed framework is evaluated with two different types of TerraSAR-X images of urban areas. Experiment results show that the proposed method has superior results than the similar works. Also, the proposed method has considerable small false alarm rate.

FabricGAN: an enhanced generative adversarial network for data augmentation and improved fabric defect detection

When deep learning is applied to intelligent textile defect detection, the insufficient training data may result in low accuracy and poor adaptability of varying defect types of the trained defect model. To address the above problem, an enhanced generative adversarial network for data augmentation and improved fabric defect detection was proposed. Firstly, the dataset is preprocessed to generate defect localization maps, which are combined with non-defective fabric images and input into the network for training, which helps to better extract defect features. In addition, by utilizing a Double U-Net network, the fusion of defects and textures is enhanced. Next, random noise and the multi-head attention mechanism are introduced to improve the model’s generalization ability and enhance the realism and diversity of the generated images. Finally, we merge the newly generated defect image data with the original defect data to realize the data enhancement. Comparison experiments were performed using the YOLOv3 object detection model on the training data before and after data enhancement. The experimental results show a significant accuracy improvement for five defect types – float, line, knot, hole, and stain – increasing from 41%, 44%, 38%, 42%, and 41% to 78%, 76%, 72%, 67%, and 64%, respectively.

RGB road scene material segmentation

We introduce RGB road scene material segmentation, i.e., per-pixel segmentation of materials in real-world driving views with pure RGB images, as a novel computer vision task by building a benchmark dataset and by deriving a new method. Our dataset, KITTI-Materials, is based on the well-established KITTI dataset and consists of 1000 frames covering 24 different road scenes of urban/suburban landscapes, carefully annotated with one of 20 material categories for every pixel. It is the first dataset for RGB material segmentation in real driving scenes. Through careful analysis of KITTI-Materials, we identify the extraction and fusion of texture and image context as the key to accurate modeling of road scene material appearance. For this, we introduce Road scene Material Segmentation Network (RMSNet) as a baseline method for this challenging task. RMSNet encodes multi-scale hierarchical features with efficient Transformer layers. We construct the decoder of RMSNet based on a novel efficient self-attention model, which we refer to as SAMixer which adaptively fuses texture and context cues across multiple feature levels. Extensive experiments on KITTI-Materials validate the effectiveness of our RMSNet. We believe our work lays a solid foundation for further studies on RGB road scene material segmentation.

Highlighting Water Stress in Apple Seedlings Using HSI Texture with Machine Learning Technique

Apples are known for their nutrition and economic value. Accurate and rapid diagnosis of water status in apple seedlings on an individual rootstock basis is a prerequisite for precision water management. This study presents a rapid and non-destructive approach for estimating water content in apple seedlings at leaf levels. A PIKA L system collects hyperspectral images (400-1000nm) of apple leaves. Our research extracts spatial information, gray-level co-occurrence matrix (GLCM), from feature wavelength images of hypercubes. Machine learning methods are applied to these spatial feature matrixs to identify apple leaves under different water stresses. In addition, differences in spectral responses were analysed using machine learning techniques for sorting apple seedlings with varying water treatments (dry, normal, and overwatering). Also, we measure chlorophyll to determine the relationship between hyperspectral characteristics and physiological changes. The achievements of the research indicate that the fusion of texture and hyperspectral imaging coupled with machine learning techniques is promising and presents a powerful potential to determine the water stress in the leaves of apple seedlings.

Clarification of Water Stress in Apple Seedlings Using HSI Texture with Machine Learning Technique

Apples are known for their nutrition and economic value. Accurate and rapid diagnosis of water status in apple seedlings on an individual rootstock basis is a prerequisite for precision water management. This study presents a rapid and non-destructive approach for estimating water content in apple seedlings at leaf levels. A PIKA L system collects hyperspectral images(400-1000nm) of apple leaves. To the author's knowledge, no prior work was conducted using the spectral-texture approach in plant water stress. Our research extracts spatial information, gray-level co-occurrence matrix (GLCM), from feature wavelength images of hypercubes. Machine learning methods are applied to these spatial feature matrixs to identify apple leaves under different water stresses. In addition, differences in spectral responses were analysed using machine learning techniques for sorting apple seedlings with varying water treatments (dry, normal, and overwatering). Also, we measure chlorophyll to determine the relationship between hyperspectral characteristics and physiological changes. The achievements of the research indicate that the fusion of texture and hyperspectral imaging coupled with machine learning techniques is promising and presents a powerful potential to determine the water stress in the leaves of apple seedlings.

Enhancing the Visual Effectiveness of Overexposed and Underexposed Images in Power Marketing Field Operations Using Gray Scale Logarithmic Transformation and Histogram Equalization

Abstract In this paper, we propose an adaptive gamma transform that adjusts the local values of bright and dark parts to enhance the effect of low-illumination images, thereby improving the light component. We then apply diff texture enhancement to enhance the contrast of images processed by the Retinex algorithm, thereby optimizing the perception of overexposed and underexposed imagery. Analyze the effect of image brightness enhancement based on a nonlinear transformation combined with the LOL dataset. Use PSNR and SSIM image quality evaluation criteria to analyze the visual effect of improving low-illumination images based on Retinex theory. Create a dataset of power marketing field operation inspection images and examine the effects of overexposure and underexposure image processing on four types of images: high-voltage towers, transmission lines, high-voltage fixtures, and high-voltage wireframes, using the low-light image texture fusion algorithm based on Retinex theory. Overall, this paper’s algorithm and the three DeblurGAN and DMCNN models achieve the effect of deblurring overexposed and underexposed power marketing field operation inspection images. From the local details, the model in this paper has a better effect on the de-exposure of the image, which can provide effective help for the electric power staff to understand the situation of the electric power marketing operation site and has strong practicality.

SkinFormer: Learning Statistical Texture Representation With Transformer for Skin Lesion Segmentation.

Accurate skin lesion segmentation from dermoscopic images is of great importance for skin cancer diagnosis. However, automatic segmentation of melanoma remains a challenging task because it is difficult to incorporate useful texture representations into the learning process. Texture representations are not only related to the local structural information learned by CNN, but also include the global statistical texture information of the input image. In this paper, we propose a transFormer network (SkinFormer) that efficiently extracts and fuses statistical texture representation for Skin lesion segmentation. Specifically, to quantify the statistical texture of input features, a Kurtosis-guided Statistical Counting Operator is designed. We propose Statistical Texture Fusion Transformer and Statistical Texture Enhance Transformer with the help of Kurtosis-guided Statistical Counting Operator by utilizing the transformer's global attention mechanism. The former fuses structural texture information and statistical texture information, and the latter enhances the statistical texture of multi-scale features. Extensive experiments on three publicly available skin lesion datasets validate that our SkinFormer outperforms other SOAT methods, and our method achieves 93.2% Dice score on ISIC 2018. It can be easy to extend SkinFormer to segment 3D images in the future.

Limb-Aware Virtual Try-On Network With Progressive Clothing Warping

Image-based virtual try-on aims to transfer an in-shop clothing image to a person image. Most existing methods adopt a single global deformation to perform clothing warping directly, which lacks fine-grained modeling of in-shop clothing and leads to distorted clothing appearance. In addition, existing methods usually fail to generate limb details well because they are limited by the used clothing-agnostic person representation without referring to the limb textures of the person image. To address these problems, we propose Limb-aware Virtual Try-on Network named PL-VTON, which performs fine-grained clothing warping progressively and generates high-quality try-on results with realistic limb details. Specifically, we present Progressive Clothing Warping (PCW) that explicitly models the location and size of in-shop clothing and utilizes a two-stage alignment strategy to progressively align the in-shop clothing with the human body. Moreover, a novel gravity-aware loss that considers the fit of the person wearing clothing is adopted to better handle the clothing edges. Then, we design Person Parsing Estimator (PPE) with a non-limb target parsing map to semantically divide the person into various regions, which provides structural constraints on the human body and therefore alleviates texture bleeding between clothing and body regions. Finally, we introduce Limb-aware Texture Fusion (LTF) that focuses on generating realistic details in limb regions, where a coarse try-on result is first generated by fusing the warped clothing image with the person image, then limb textures are further fused with the coarse result under limb-aware guidance to refine limb details. Extensive experiments demonstrate that our PL-VTON outperforms the state-of-the-art methods both qualitatively and quantitatively.

IFICI: Infrared and visible image fusion based on interactive compensation illumination

The goal of infrared and visible image fusion is to generate an informative image that preserves the complementary information of the two types of source images, such as texture details and infrared targets. Existing methods have designed a variety of means to achieve the fusion of image texture and target complementary information. But they ignore the fact that the two types of source images have the complementarity of illumination, especially the infrared image can mainly supplement the information of the low-light visible image. An image fusion method using infrared and visible light to supplement light interactively is proposed. It performs intrinsic image decomposition on the input infrared and visible light images respectively and then fuses the illumination components and material components of the two types of images respectively. Further, to better meet the needs of image fusion, an improved variational model for the decomposition of image intrinsics is proposed. Experimental results demonstrate the effectiveness of the proposed model with its algorithm. Compared with the state-of-the-art methods, the significant advantage of the proposed method is that it has better fusion accuracy and visual perception at the same time, and the comprehensive metric evaluation is the best (9.2% performance improvement). The code is available at https://github.com/skyworkds/IFICI.

Land Cover Mapping and Change Analysis Using Optimized Random Forest Classifier Incorporating Fusion of Texture and Gabor Features

Land Cover Mapping and Change Analysis Using Optimized Random Forest Classifier Incorporating Fusion of Texture and Gabor Features

Self-supervised feature matched virtual try-on

Abstract Virtual try-on is a technology that enables users to preview the effect of wearing a target garment without wearing the actual garment. However, existing image-based virtual try-on methods often require additional human parsing or segmentation operations to generate intermediate representations required for garment deformation and texture fusion. These operations not only increase the computational complexity and memory consumption, but also limit the real-time and portability of virtual try-on. Additionally, inaccurate parsing results can lead to misleading final generated images. To overcome these challenges, we propose a self-supervised feature matched virtual try-on network, which can directly generate high-quality try-on results from human body images and target clothing images without any additional input. Specifically, we design an optical flow warp module, which focuses on the optical flow changes between the person image and the clothing image to achieve accurate clothing alignment and deformation. Furthermore, a feature refine warp module is designed to enhance the features of the extracted optical flow information and the original character segmentation and analysis operations, reducing the influence of background clutter features on the content, and ensuring that the wrinkles and deformation of the replacement clothes are close to the original clothes. The feature match module is developed to calculate the feature matching loss of the converted clothing and the generated results of the teacher network and the student network, and the corresponding knowledge is distilled and passed to the student network to assist in self-supervised training. We conduct experiments on the VITON dataset and show that our model can generate high quality and high resolution, and our proposed method outperforms the state-of-the-art virtual try-on methods both qualitatively and quantitatively.

Conv-ViT: A Convolution and Vision Transformer-Based Hybrid Feature Extraction Method for Retinal Disease Detection.

The current advancement towards retinal disease detection mainly focused on distinct feature extraction using either a convolutional neural network (CNN) or a transformer-based end-to-end deep learning (DL) model. The individual end-to-end DL models are capable of only processing texture or shape-based information for performing detection tasks. However, extraction of only texture- or shape-based features does not provide the model robustness needed to classify different types of retinal diseases. Therefore, concerning these two features, this paper developed a fusion model called 'Conv-ViT' to detect retinal diseases from foveal cut optical coherence tomography (OCT) images. The transfer learning-based CNN models, such as Inception-V3 and ResNet-50, are utilized to process texture information by calculating the correlation of the nearby pixel. Additionally, the vision transformer model is fused to process shape-based features by determining the correlation between long-distance pixels. The hybridization of these three models results in shape-based texture feature learning during the classification of retinal diseases into its four classes, including choroidal neovascularization (CNV), diabetic macular edema (DME), DRUSEN, and NORMAL. The weighted average classification accuracy, precision, recall, and F1 score of the model are found to be approximately 94%. The results indicate that the fusion of both texture and shape features assisted the proposed Conv-ViT model to outperform the state-of-the-art retinal disease classification models.

Research on rice leaf area index estimation based on fusion of texture and spectral information

Leaf Area Index (LAI) is one of the indicators used to measure the growth status of rice fields. Rapid, accurate, and large-scale monitoring of LAI plays an important role in ensuring stable grain yield increase. In recent years, the spectral saturation problem and the parameter adjustment problem of machine learning algorithms have become the main limitations to improve the accuracy of LAI estimation. High-resolution Unmanned Aerial Vehicles (UAVs) images contain not only rich spectral information, but also texture information reflecting the crop canopy structure. Therefore, in order to fully understand the role of spectral information and texture information fusion in rice LAI estimation, this study used the hyperspectral sensor carried by the UAVs to obtain the spectral images of rice canopy of different varieties and different growth stages. Rice canopy reflectance and 8 basic texture features based on Gray-level Co-occurrence Matrix (GLCM) were extracted from hyperspectral images to calculate vegetation indexs (VIs) and combined texture features. Normalized difference texture index (NDTI), Non-linear texture index (NLTI), Enhanced vegetation texture index (EVTI), and Modified triangular texture index (MTTI) were calculated using two and three GLMC-based texture features to explore the effect of combinations of different basic texture features on LAI sensitivity. Two rice LAI estimation models were developed for single spectral indicators and combined with texture indicators, respectively. The results show that: (1) After preprocessing and feature band screening, the optimal spectral band, vegetation index, and trilateral parameters were obtained. When the combined spectral parameters (SP) of the three were used as the only input to the model, R2 showed an increasing trend throughout the growth period. The best results were achieved using the support vector regression (SVR) combined with the pelican optimization algorithm (POA) in the pre jointing stage: R2= 0.839, RMSE = 0.107, and MAPE = 7.02%. (2) When texture information based on hyperspectral images was incorporated into the model input, the results showed that the models based on spectral indicators combined with texture measurements were all superior to those using spectral indicators alone, with the best model having a coefficient of determination: R2 = 0.917, RMSE = 0.078, and MAPE = 4.19%, which has promising applications in crop growth index detection. This technology can quickly and effectively monitor the growth status of crops in the field, providing a theoretical basis for estimating crop yield in the later stage.

A method of face texture fusion based on visibility weight

In the reconstruction process from 2D images to 3D face models, texture completion still suffers from pixel blurring and color inconsistency when face images are under different perspectives. In this paper, we propose a method based on visibility weights for face texture fusion. Meanwhile, for the complex geometric structure of the ear region where the traditional texture mapping algorithm is inapplicable, a skin color probability method with Gaussian model is used for pixel completion, and jointly optimized with the texture fusion band. Finally, we generate a complete and high-fidelity face texture model. The simulation experiment shows that the novel face texture fusion and completion method generates the perfect texture under multiple viewpoints. Our face texture model outperforms state-of-the-art techniques under the same rendering conditions.

V-PCC Projection Based Blind Point Cloud Quality Assessment for Compression Distortion

In immersive media communication with point cloud (PC), PC compression will inevitably cause distortion, which may affect quality of users’ visual experience. To monitor the visual quality of PCs, point cloud quality assessment (PCQA) metrics are highly desired. Inspired by the video-based point cloud compression (V-PCC) standard to project a PC to planes and quantify the corresponding projection maps, a new blind PCQA metric based on V-PCC texture projection map and geometry projection map of PC is proposed for evaluating PC with compression distortion. Specifically, considering the combination of texture and geometry features of the projection maps to characterize distorted PC, a dual-stream convolutional network is designed from the perspective of global and local feature description to extract texture and geometry features of the distorted PC. The network mainly consists of two modules, that is, global texture and geometry fusion (GTGF) module and local geometry weighted local texture (LGWLT) module. The GTGF module is designed to realize the fusion of global texture and geometry features, which can describe PC's combination effect of texture and geometry distortions. Aiming at the relationship between texture and geometry distortions, the LGWLT module is designed to extract the features of local texture distortion caused by local geometry distortion. Furthermore, visual perception is combined to take distortions of the PC's salient regions into account in the process of PCQA. The experimental results on three PC databases show that the proposed PCQA metric is more consistent with visual perception, thus has greater potential than the representative PCQA metrics.