Rich Texture Research Articles

Physical prior-guided deep fusion network with shading cues for shape from polarization

Shape from polarization (SfP) is a powerful passive three-dimensional imaging technique that enables the reconstruction of surface normal with dense textural details. However, existing deep learning-based SfP methods only focus on the polarization prior, which makes it difficult to accurately reconstruct targets with rich texture details under complicated scenes. Aiming to improve the reconstruction accuracy, we utilize the surface normal estimated from shading cues and the innovatively proposed specular confidence as shading prior to provide additional feature information. Furthermore, to efficiently combine the polarization and shading priors, a novel deep fusion network named SfPSNet is proposed for the information extraction and the reconstruction of surface normal. SfPSNet is implemented based on a dual-branch architecture to handle different physical priors. A feature correction module is specifically designed to mutually rectify the defects in channel-wise and spatial-wise dimensions, respectively. In addition, a feature fusion module is proposed to fuse the feature maps of polarization and shading priors based on an efficient cross-attention mechanism. Our experimental results show that the fusion of polarization and shading priors can significantly improve the reconstruction quality of surface normal, especially for objects or scenes illuminated by complex lighting sources. As a result, SfPSNet shows state-of-the-art performance compared with existing deep learning-based SfP methods benefiting from its efficiency in extracting and fusing information from different priors.

Illumination enhancement discriminator and compensation attention based low-light visible and infrared image fusion

Infrared and visible image fusion is an important image enhancement technology, aiming to generate high-quality fused images with prominent targets and rich textures in extreme environments. However, most existing image fusion methods are designed for infrared and visible images under normal lighting. At night, due to severe degradation of visible images, existing fusion methods have deficiencies in texture details and visual perception, which affects subsequent visual applications. To this end, this paper proposes a three-discriminator infrared and visible image fusion method based on GAN network. Specifically, this method adds an illumination enhancement discriminator based on the GAN-based dual discriminator fusion network. The input of this discriminator is the fused image generated by the generator and the low-light enhanced visible light image. By fighting in the third discriminator, it is ensured that the fused image output by the generator achieves the expected effect on the brightness information. In addition, this method also proposes a compensation attention module to convey the multi-scale features extracted by the feature extraction network and ensure that the fused image contains important detailed texture information. Compared with other fusion methods on public data sets such as MSRS, M3FD, Roadscence and TNO, the fusion results of this paper perform better in both quantitative measurement and qualitative effects. It also performs better in enhancing the brightness information.

DGGNets: Deep Gradient-Guidance Networks for Speckle Noise Reduction

Speckle noise is a granular interference that degrades image quality in coherent imaging systems, including underwater sonar, Synthetic Aperture Radar (SAR), and medical ultrasound. This study aims to enhance speckle noise reduction through advanced deep learning techniques. We introduce the Deep Gradient-Guidance Network (DGGNet), which features an architecture comprising one encoder and two decoders—one dedicated to image recovery and the other to gradient preservation. Our approach integrates a gradient map and fractional-order total variation into the loss function to guide training. The gradient map provides structural guidance for edge preservation and directs the denoising branch to focus on sharp regions, thereby preventing over-smoothing. The fractional-order total variation mitigates detail ambiguity and excessive smoothing, ensuring rich textures and detailed information are retained. Extensive experiments yield an average Peak Signal-to-Noise Ratio (PSNR) of 31.52 dB and a Structural Similarity Index (SSIM) of 0.863 across various benchmark datasets, including McMaster, Kodak24, BSD68, Set12, and Urban100. DGGNet outperforms existing methods, such as RIDNet, which achieved a PSNR of 31.42 dB and an SSIM of 0.853, thereby establishing new benchmarks in speckle noise reduction.

Knowledge-embedded multi-layer collaborative adaptive fusion network: Addressing challenges in foggy conditions and complex imaging

Infrared and visible image fusion aims at generating high-quality images that serve both human and machine visual perception under extreme imaging conditions. However, current fusion methods primarily rely on datasets comprising infrared and visible images captured under clear weather conditions. When applied to real-world scenarios, image fusion tasks inevitably encounter challenges posed by adverse weather conditions such as heavy fog, resulting in difficulties in obtaining effective information and inferior visual perception. To address these challenges, this paper proposes a Mean Teacher-based Self-supervised Image Restoration and multimodal Image Fusion joint learning network (SIRIFN), which enhances the robustness of the fusion network in adverse weather conditions by employing deep supervision from a guiding network to the learning network. Furthermore, to enhance the network’s information extraction and integration capabilities, our Multi-level Feature Collaborative adaptive Reconstruction Network (MFCRNet) is introduced, which adopts a multi-branch, multi-scale design, with differentiated processing strategies for different features. This approach preserves rich texture information while maintaining semantic consistency from the source images. Extensive experiments demonstrate that SIRIFN outperforms current state-of-the-art algorithms in both visual quality and quantitative evaluation. Specifically, the joint implementation of image restoration and multimodal fusion provides more effective information for visual tasks under extreme weather conditions, thereby facilitating downstream visual tasks.

A Practical Contrastive Learning Framework for Single-Image Super-Resolution.

Contrastive learning has achieved remarkable success on various high-level tasks, but there are fewer contrastive learning-based methods proposed for low-level tasks. It is challenging to adopt vanilla contrastive learning technologies proposed for high-level visual tasks to low-level image restoration problems straightly. Because the acquired high-level global visual representations are insufficient for low-level tasks requiring rich texture and context information. In this article, we investigate the contrastive learning-based single-image super-resolution (SISR) from two perspectives: positive and negative sample construction and feature embedding. The existing methods take naive sample construction approaches (e.g., considering the low-quality input as a negative sample and the ground truth as a positive sample) and adopt a prior model (e.g., pretrained very deep convolutional networks proposed by visual geometry group (VGG) model) to obtain the feature embedding. To this end, we propose a practical contrastive learning framework for SISR (PCL-SR). We involve the generation of many informative positive and hard negative samples in frequency space. Instead of utilizing an additional pretrained network, we design a simple but effective embedding network inherited from the discriminator network, which is more task-friendly. Compared with the existing benchmark methods, we retrain them by our proposed PCL-SR framework and achieve superior performance. Extensive experiments have been conducted to show the effectiveness and technical contributions of our proposed PCL-SR thorough ablation studies. The code and resulting models will be released via https://github.com/Aitical/PCL-SISR.

SPFusion: A multi-task semantic perception infrared and visible light fusion method with quality assessment

The fusion of infrared and visible light aims to combine the advantages of infrared highlighting thermal targets and the rich textures of visible light, to produce a fused image that is both texture-rich and has prominent targets. However, under extreme conditions such as heavy fog and overexposure, directly fusing low-quality visible light images with infrared images may retain excessive redundant information in these areas. Therefore, this paper proposes a method for achieving adaptive fusion under extreme conditions, called SPFusion. Specifically, the network is guided by a quality assessment module and advanced visual tasks such as segmentation, classification, and reconstruction, to learn an adaptive balance point for rationally distributing infrared and visible light information. Additionally, a progressive fusion strategy and a modality complementarity module are employed to fully integrate information from each modality. Consequently, the network can autonomously allocate visible and infrared information based on image quality and semantics. Extensive experiments have demonstrated that SPFusion achieves state-of-the-art (SOTA) performance on the TNO, MSRS, LLVIP, and RoadScene datasets, and enhances the performance of advanced visual tasks such as object detection. The code is available as open source at linshenj/SPFusion (github.com).

HDN-Net: A Hybrid Deep Neural Network to Improve Iris Recognition in Unconstrained Environments with Eyeglasses

Recently, the use of iris recognition technology for biometric authentication has gained widespread acceptance due to the rich texture of the iris region, which provides a reliable standard for recognising individuals, as well as the non-intrusive nature of this method. However, the presence of eyeglasses poses a significant challenge to the accuracy of such systems. In unrestricted environments, current iris recognition techniques cannot effectively extract distinguishing features of the iris. Eyeglasses introduce scratches, specular reflections, dirt, blurriness, and other noise factors over the image of the iris, resulting in low recognition accuracy. To tackle this challenge, researchers have proposed the HDN-Net architecture. This architecture employs a multi-CNN model to combine the features of both the right and left iris images, extracting more distinguishing features to improve the accuracy of the classification task in the presence of challenges caused by eyeglasses. Experiment results show that the proposed iris recognition system achieves more promising performance compared to previous methods used in this field. The overall performance of our suggested HDN-Net method on the UBIRIS.V2 and CASIA-Iris.V4-1000 databases achieved 97.89% and 98.79% accuracy, respectively. Thus, the proposed HDN-Net method consistently outperforms other traditional and deep learning approaches and has the possibility to improve the accuracy of iris recognition systems (IRS) in real-world scenarios.

Enhanced Visual SLAM for Collision-Free Driving with Lightweight Autonomous Cars.

The paper presents a vision-based obstacle avoidance strategy for lightweight self-driving cars that can be run on a CPU-only device using a single RGB-D camera. The method consists of two steps: visual perception and path planning. The visual perception part uses ORBSLAM3 enhanced with optical flow to estimate the car's poses and extract rich texture information from the scene. In the path planning phase, the proposed method employs a method combining a control Lyapunov function and control barrier function in the form of a quadratic program (CLF-CBF-QP) together with an obstacle shape reconstruction process (SRP) to plan safe and stable trajectories. To validate the performance and robustness of the proposed method, simulation experiments were conducted with a car in various complex indoor environments using the Gazebo simulation environment. The proposed method can effectively avoid obstacles in the scenes. The proposed algorithm outperforms benchmark algorithms in achieving more stable and shorter trajectories across multiple simulated scenes.

PTPFusion: A progressive infrared and visible image fusion network based on texture preserving

Infrared and visible image fusion aims to provide a more comprehensive image for downstream tasks by highlighting the main target and maintaining rich texture information. Image fusion methods based on deep learning suffer from insufficient multimodal information extraction and texture loss. In this paper, we propose a texture-preserving progressive fusion network (PTPFusion) to extract complementary information from multimodal images to solve these issues. To reduce image texture loss, we design multiple consecutive texture-preserving blocks (TPB) to enhance fused texture. The TPB can enhance the features by using a parallel architecture consisting of a residual block and derivative operators. In addition, a novel cross-channel attention (CCA) fusion module is developed to obtain complementary information by modeling global feature interactions via cross-queries mechanism, followed by information fusion to highlight the feature of the salient target. To avoid information loss, the extracted features at different stages are merged as the output of TPB. Finally, the fused image will be generated by the decoder. Extensive experiments on three datasets show that our proposed fusion algorithm is better than existing state-of-the-art methods.

ACDF-YOLO: Attentive and Cross-Differential Fusion Network for Multimodal Remote Sensing Object Detection

Object detection in remote sensing images has received significant attention for a wide range of applications. However, traditional unimodal remote sensing images, whether based on visible light or infrared, have limitations that cannot be ignored. Visible light images are susceptible to ambient lighting conditions, and their detection accuracy can be greatly reduced. Infrared images often lack rich texture information, resulting in a high false-detection rate during target identification and classification. To address these challenges, we propose a novel multimodal fusion network detection model, named ACDF-YOLO, basedon the lightweight and efficient YOLOv5 structure, which aims to amalgamate synergistic data from both visible and infrared imagery, thereby enhancing the efficiency of target identification in remote sensing imagery. Firstly, a novel efficient shuffle attention module is designed to assist in extracting the features of various modalities. Secondly, deeper multimodal information fusion is achieved by introducing a new cross-modal difference module to fuse the features that have been acquired. Finally, we combine the two modules mentioned above in an effective manner to achieve ACDF. The ACDF not only enhances the characterization ability for the fused features but also further refines the capture and reinforcement of important channel features. Experimental validation was performed using several publicly available multimodal real-world and remote sensing datasets. Compared with other advanced unimodal and multimodal methods, ACDF-YOLO separately achieved a 95.87% and 78.10% mAP0.5 on the LLVIP and VEDAI datasets, demonstrating that the deep fusion of different modal information can effectively improve the accuracy of object detection.

MEEAFusion: Multi-Scale Edge Enhancement and Joint Attention Mechanism Based Infrared and Visible Image Fusion.

Infrared and visible image fusion can integrate rich edge details and salient infrared targets, resulting in high-quality images suitable for advanced tasks. However, most available algorithms struggle to fully extract detailed features and overlook the interaction of complementary features across different modal images during the feature fusion process. To address this gap, this study presents a novel fusion method based on multi-scale edge enhancement and a joint attention mechanism (MEEAFusion). Initially, convolution kernels of varying scales were utilized to obtain shallow features with multiple receptive fields unique to the source image. Subsequently, a multi-scale gradient residual block (MGRB) was developed to capture the high-level semantic information and low-level edge texture information of the image, enhancing the representation of fine-grained features. Then, the complementary feature between infrared and visible images was defined, and a cross-transfer attention fusion block (CAFB) was devised with joint spatial attention and channel attention to refine the critical supplemental information. This allowed the network to obtain fused features that were rich in both common and complementary information, thus realizing feature interaction and pre-fusion. Lastly, the features were reconstructed to obtain the fused image. Extensive experiments on three benchmark datasets demonstrated that the MEEAFusion proposed in this research has considerable strengths in terms of rich texture details, significant infrared targets, and distinct edge contours, and it achieves superior fusion performance.

Infrared and Harsh Light Visible Image Fusion Using an Environmental Light Perception Network.

The complementary combination of emphasizing target objects in infrared images and rich texture details in visible images can effectively enhance the information entropy of fused images, thereby providing substantial assistance for downstream composite high-level vision tasks, such as nighttime vehicle intelligent driving. However, mainstream fusion algorithms lack specific research on the contradiction between the low information entropy and high pixel intensity of visible images under harsh light nighttime road environments. As a result, fusion algorithms that perform well in normal conditions can only produce low information entropy fusion images similar to the information distribution of visible images under harsh light interference. In response to these problems, we designed an image fusion network resilient to harsh light environment interference, incorporating entropy and information theory principles to enhance robustness and information retention. Specifically, an edge feature extraction module was designed to extract key edge features of salient targets to optimize fusion information entropy. Additionally, a harsh light environment aware (HLEA) module was proposed to avoid the decrease in fusion image quality caused by the contradiction between low information entropy and high pixel intensity based on the information distribution characteristics of harsh light visible images. Finally, an edge-guided hierarchical fusion (EGHF) module was designed to achieve robust feature fusion, minimizing irrelevant noise entropy and maximizing useful information entropy. Extensive experiments demonstrate that, compared to other advanced algorithms, the method proposed fusion results contain more useful information and have significant advantages in high-level vision tasks under harsh nighttime lighting conditions.

PSDF: Prior-Driven Neural Implicit Surface Learning for Multi-view Reconstruction.

Surface reconstruction has traditionally relied on the Multi-View Stereo (MVS)-based pipeline, which often suffers from noisy and incomplete geometry. This is due to that although MVS has been proven to be an effective way to recover the geometry of the scenes, especially for locally detailed areas with rich textures, it struggles to deal with areas with low texture and large variations of illumination where the photometric consistency is unreliable. Recently, Neural Implicit Surface Reconstruction (NISR) combines surface rendering and volume rendering techniques and bypasses the MVS as an intermediate step, which has emerged as a promising alternative to overcome the limitations of traditional pipelines. While NISR has shown impressive results on simple scenes, it remains challenging to recover delicate geometry from uncontrolled real-world scenes which is caused by its underconstrained optimization. To this end, the framework PSDF is proposed which resorts to external geometric priors from a pretrained MVS network and internal geometric priors inherent in the NISR model to facilitate high-quality neural implicit surface learning. Specifically, the visibility-aware feature consistency loss and depth prior-assisted sampling based on external geometric priors are introduced. These proposals provide powerfully geometric consistency constraints and aid in locating surface intersection points, thereby significantly improving the accuracy and delicate reconstruction of NISR. Meanwhile, the internal prior-guided importance rendering is presented to enhance the fidelity of the reconstructed surface mesh by mitigating the biased rendering issue in NISR. Extensive experiments on Tanks and Temples datasets show that PSDF achieves state-of-the-art performance on complex uncontrolled scenes.

Texture feature-aware consistency for semi-supervised honeycomb lung lesion segmentation

Accurate segmentation of honeycomb lung lesions from lung CT images is crucial for the clinical diagnosis of a wide range of lung diseases. Existing semi-supervised segmentation methods ignore the rich texture feature information in honeycomb lung CT images, resulting in inaccurate predictions in the lesion region. Therefore, this paper proposes a semi-supervised honeycomb lung lesion segmentation framework using texture feature-aware consistency. The framework consists of a shared encoder, a main decoder, and three auxiliary decoders. The different outputs produced by the decoders are used to compute the Grey Level Co-occurrence Matrix and extract texture features, respectively, and further to compute the loss of texture feature-aware consistency. In pseudo-labeling supervision, the uncertainty of each output is computed to bootstrap the pseudo-labeling supervision loss. For the small lesion recognition omission problem, a pseudo-label refinement module is constructed to reuse the multi-scale features of the encoder, focusing on small-scale lesions, and refining the pseudo-labels. Experimental results show that the proposed method outperforms other methods in the honeycomb lung segmentation task, achieving a Dice coefficient of 77.62 using 20% labeled data. Additionally, experiments on publicly available datasets further demonstrate the generalizability of the proposed method. The code is available at https://github.com/Oran9er/TFC-Net-main.

An infrared and visible image fusion using knowledge measures for intuitionistic fuzzy sets and Swin Transformer

The objectives of infrared and visible image fusion are to generate a single image that includes significant objects and rich texture information. However, the current deep-learning methods ignore uncertainty in the decision-making process during the fusion phase. To address this issue, we propose a novel infrared and visible image fusion method using a Swin Transformer and knowledge measures of intuitionistic fuzzy sets (IFSs) named SWKIF-Fusion. This model employs a Swin Transformer-based pre-trained module for feature extraction, which is the most effective module for modeling long-range dependencies. The fusion process of SWKIF-Fusion integrates the proposed knowledge measure of IFSs. IFSs inherently possess a high capability to handle uncertainty, and the knowledge measure of IFSs provides uncertainty quantification. This integration in the fusion phase mitigates the uncertainty in the decision-making process. This fusion of IFSs, knowledge measures, and the Swin Transformer-based deep learning model enhances the overall performance, as demonstrated through experiments on the TNO, Roadscene, OTCBVS, M3FD, and MSRS datasets. This study also fills the gap in developing knowledge measures for IFSs by proposing novel general construction methods. It presents a theoretically sound framework for knowledge measures of IFSs using well-established mathematical concepts such as t-norms, t-conorms, automorphisms, and aggregation operators.

High-resolution enhanced cross-subspace fusion network for light field image superresolution

Light field (LF) images offer abundant spatial and angular information, therefore, the combination of which is beneficial in the performance of LF image superresolution (LF image SR). Currently, existing methods often decompose the 4D LF data into low-dimensional subspaces for individual feature extraction and fusion for LF image SR. However, the performance of these methods is restricted because of lacking effective correlations between subspaces and missing out on crucial complementary information for capturing rich texture details. To address this, we propose a cross-subspace fusion network for LF spatial SR (i.e., CSFNet). Specifically, we design the progressive cross-subspace fusion module (PCSFM), which can progressively establish cross-subspace correlations based on a cross-attention mechanism to comprehensively enrich LF information. Additionally, we propose a high-resolution adaptive enhancement group (HR-AEG), which preserves the texture and edge details in the high resolution feature domain by employing a multibranch enhancement method and an adaptive weight strategy. The experimental results demonstrate that our approach achieves highly competitive performance on multiple LF datasets compared to state-of-the-art (SOTA) methods.

Lights off the Image: Highlight Suppression for Single Texture-Rich Images in Optical Inspection Based on Wavelet Transform and Fusion Strategy

A wavelet-transform-based highlight suppression method is presented, aiming at suppressing the highlights of single image with complex texture. The strategy involves the rough extraction of specular information, followed by extracting the high-frequency information in specular information based on multi-level wavelet transform to enhance the texture information in the original images by fusion strategy, and fusing with the same-level specular information to achieve the highlight suppression image. The experimental results demonstrate that the proposed method effectively removed large-area highlights while preserving texture details, and demonstrated the authenticity of the highlight estimation and the ‘lights off’ effect in the highlight-suppressed images. Overall, the method offers a feasibility for addressing the challenges of highlight suppression for visual detection image with rich texture and large-area highlights.

Chemosensory analysis of emotional wines: Merging of explicit and implicit methods to measure emotions aroused by red wines

Wine is a cultural product capable of arousing emotions. Measuring emotions and figuring out how much they could influence preferences or purchase decisions of consumers is a new trend in sensory and consumer research. However, the complexity of feelings makes the measure of emotions extremely challenging. Thus, a comprehensive understanding of emotions related to sensory stimuli in wine tasting is still missing. The purpose of the study is to evaluate the emotional power of tasting red wines using a multidisciplinary approach, combining sensory analysis performed by trained panelists, implicit and explicit measurements of emotions and chemical analysis of the wines tasted. Various red wines, renowned for their high polyphenol content and expected to exhibit rich texture, mouthfeel, and aging potential, have been utilized to this extent. The results obtained showed that the emotions measured were well-correlated with quantitative and hedonic attributes obtained using classic sensory analysis. Some aromatic molecules can be identified as markers capable of eliciting positive and negative emotional reactions. While increasing literature in the topic is recently available, our study appears to be the first highlighting the presence of autonomic nervous system (ANS) differences verified by means of electrocardiogram (ECG) features, related to explicit and complete sensory patterns, in response to sensory stimuli related to emotional wine, with higher sympathetic values at extrema and vagal increase in the presence of neutral sensory compounds.

Cosine-enhanced tuna swarm optimized exponential entropy segmentation method for sand grain microscopic images

The feature extraction of sand grain size, color and texture is a necessary step to identify clastic components. The sand grain features are complex and various, which brings difficulties to geological identification. Aiming at these images, a cosine-enhanced tuna swarm optimized exponential entropy segmentation method is proposed, which can effectively preserve the texture features of various sand grains. Firstly, for the tuna swarm optimization (TSO) algorithm, three improvement strategies are proposed: cosine spiral movement, cosine parabolic movement and Gauss-Cauchy mutation, which improve the TSO's global and local search. This algorithm is called cosine-enhanced TSO (CETSO). Benchmark function experiments showed that the convergence accuracy and stability of CETSO are greatly improved, and the convergence speed is also slightly increased. Secondly, CETSO optimized the exponential entropy to automatically determine the segmentation thresholds, and the feasibility of the method is verified by taking the information content of the segmented image as the standard. Finally, segmentation experiments were carried out on the Yarlung Zangbo River sand microscopic image dataset, and the results show that the method has high segmentation accuracy and stability for images with high contrast, rich texture, or significant differences in the size of sand debris. Compared with TSO, the CETSO optimized exponential entropy segmented images achieved an improvement of 21% and 93% in the evaluation of the average and standard deviation of the peak signal-to-noise ratio on thirty experiments. And this method has a fast processing speed, and it only takes about 0.85s to divide an image on average, which meets the needs of engineering applications.

Semantic-Aware Fusion Network Based on Super-Resolution.

The aim of infrared and visible image fusion is to generate a fused image that not only contains salient targets and rich texture details, but also facilitates high-level vision tasks. However, due to the hardware limitations of digital cameras and other devices, there are more low-resolution images in the existing datasets, and low-resolution images are often accompanied by the problem of losing details and structural information. At the same time, existing fusion algorithms focus too much on the visual quality of the fused images, while ignoring the requirements of high-level vision tasks. To address the above challenges, in this paper, we skillfully unite the super-resolution network, fusion network and segmentation network, and propose a super-resolution-based semantic-aware fusion network. First, we design a super-resolution network based on a multi-branch hybrid attention module (MHAM), which aims to enhance the quality and details of the source image, enabling the fusion network to integrate the features of the source image more accurately. Then, a comprehensive information extraction module (STDC) is designed in the fusion network to enhance the network's ability to extract finer-grained complementary information from the source image. Finally, the fusion network and segmentation network are jointly trained to utilize semantic loss to guide the semantic information back to the fusion network, which effectively improves the performance of the fused images on high-level vision tasks. Extensive experiments show that our method is more effective than other state-of-the-art image fusion methods. In particular, our fused images not only have excellent visual perception effects, but also help to improve the performance of high-level vision tasks.