Texture Details Research Articles

SDAM: A dual attention mechanism for high-quality fusion of infrared and visible images.

Image fusion of infrared and visible images to obtain high-quality fusion images with prominent infrared targets has important applications in various engineering fields. However, current fusion processes encounter problems such as unclear texture details and imbalanced infrared targets and texture detailed information, which lead to information loss. To address these issues, this paper proposes a method for infrared and visible image fusion based on a specific dual-attention mechanism (SDAM). This method employs an end-to-end network structure, which includes the design of channel attention and spatial attention mechanisms. Through these mechanisms, the method can fully exploit the texture details in the visible images while preserving the salient information in the infrared images. Additionally, an optimized loss function is designed to combine content loss, edge loss, and structure loss to achieve better fusion effects. This approach can fully utilize the texture detailed information of visible images and prominent information in infrared images, while maintaining better brightness and contrast, which improves the visual effect of fusion images. Through conducted ablation experiments and comparative evaluations on public datasets, our research findings demonstrate that the SDAM method exhibits superior performance in both subjective and objective assessments compared to the current state-of-the-art fusion methods.

A modality separation approach for facial sketch synthesis

AbstractThe technology for face‐to‐sketch synthesis transforms optical face images into a sketch‐style format. However, traditional style losses are insufficient to discern the modal differences between optical and sketch domain images, leading to unclear images. At the same time, generated images lack clarity due to traditional approaches' disregard for high‐frequency texture. To address these issues, a modality separation approach for facial sketch synthesis is proposed. First, a modality separation structure is proposed, using a quicksort algorithm to merge features of optical and sketch images as target modality (positive samples), ensuring the generated images' feature distribution matches real sketches. By controlling the Euclidean distance between generated images (anchors) and both target and filtered modality (positive and negative samples), irrelevant information is effectively filtered out. Next, an edge‐promoting module feeds processed blurry sketch images into the discriminator to enhance robustness. Lastly, a detail optimization module uses Laplacian filtering to extract high‐frequency texture from optical face images for local enhancement. Experimental validation on CUHK, AR, and XM2VTS datasets shows that this method outperforms mainstream sketch face synthesis methods in terms of Fréchet inception distance and learned perceptual image patch similarity, producing more realistic and natural images with richer texture details.

Sterilization of image steganography using self-supervised convolutional neural network

Background With the development of steganography technology, lawbreakers can implement covert communication in social networks more easily, exacerbating network security risks. Sterilization of image steganography methods can eliminate secret messages to block the transmission of illegal covert communication. However, existing methods overly rely on cover-stego image pairs and are unable to sanitize unknown image, which reduces stego image blocking rate in social networks. Methods To address the above problems, this paper proposes an effective sterilization of image steganography method using self-supervised convolutional neural network (SS-Net), which does not require any prior knowledge of image steganography schemes. SS-Net includes a purification module and a refinement module. Firstly, the pixel-shuffle down-sampling in purification module is adopted to reduce the spatial correlation of pixels in the stgeo image, and improve the learning mode from supervised learning to self-supervised learning. Secondly, centrally masked convolutions and dilated convolution residual blocks are merged to eliminate secret messages and avoid image quality degradation. Finally, a refinement module is employed to improve image texture details and boundaries. Results A series of experiments show that SS-Net from BOSSbase test sets is able to balance the destruction of secret messages with image quality, achieving 100% blocking rate of stego image. Meanwhile, our method outperforms the state-of-the-art methods in secret messages elimination ability and image quality preserving ability.

AFIRE: Adaptive FusionNet for illumination-robust feature extraction in heterogeneous imaging environments

The fusion of infrared and visible images aims to synthesize a fused image that incorporates richer information by leveraging the distinct characteristics of each modality. However, the disparate quality of input images in terms of infrared and visible light significantly impacts fusion performance. To address this issue, we propose a novel deep adaptive fusion method called Adaptive FusionNet for Illumination-Robust Feature Extraction (AFIRE). This method involves the interactive processing of two input features and dynamically adjusts the fusion weights based on varying illumination conditions. Specifically, we introduce a novel interactive extraction structure during the feature extraction stage for both infrared and visible light, enabling the capture of more complementary information. Additionally, we design a Deep Adaptive Fusion module to assess the quality of input features and perform weighted fusion through a channel attention mechanism. Finally, a new loss function is formulated by incorporating the entropy and median of input images to guide the training of the fusion network. Extensive experiments demonstrate that AFIRE outperforms state-of-the-art methods in preserving pixel intensity distribution and texture details. Source code is available at: https://www.github.com/ISCLab-Bistu/AFIRE.

Texture graph transformer for prostate cancer classification

Prostate cancer classification plays a pivotal role in the diagnosis and treatment of this disease. In this paper, we present a novel approach called the texture graph transformer, which combines texture analysis techniques, graph-based representations, and multi-head attention to enhance the accuracy of prostate cancer classification. Our texture graph transformer adeptly captures intricate texture details and patterns within prostate cancer lesion regions, efficiently improving the model’s ability to discern between different cancer types. By leveraging graph-based representations, our model constructs texture graphs that effectively capture the spatial and textural relationships among various regions in MRI images, enabling comprehensive feature extraction. The incorporation of a texture graph multi-head attention further enhances the model’s capacity to capture long-range dependencies and global context, with a specific focus on informative regions. Through extensive experiments conducted on a dataset comprising 226 MRI images, we demonstrate the efficacy of our proposed approach in significantly improving the performance of prostate cancer classification.

Three-Dimensional Dense Reconstruction: A Review of Algorithms and Datasets.

Three-dimensional dense reconstruction involves extracting the full shape and texture details of three-dimensional objects from two-dimensional images. Although 3D reconstruction is a crucial and well-researched area, it remains an unsolved challenge in dynamic or complex environments. This work provides a comprehensive overview of classical 3D dense reconstruction techniques, including those based on geometric and optical models, as well as approaches leveraging deep learning. It also discusses the datasets used for deep learning and evaluates the performance and the strengths and limitations of deep learning methods on these datasets.

MEEAFusion: Multi-Scale Edge Enhancement and Joint Attention Mechanism Based Infrared and Visible Image Fusion.

Infrared and visible image fusion can integrate rich edge details and salient infrared targets, resulting in high-quality images suitable for advanced tasks. However, most available algorithms struggle to fully extract detailed features and overlook the interaction of complementary features across different modal images during the feature fusion process. To address this gap, this study presents a novel fusion method based on multi-scale edge enhancement and a joint attention mechanism (MEEAFusion). Initially, convolution kernels of varying scales were utilized to obtain shallow features with multiple receptive fields unique to the source image. Subsequently, a multi-scale gradient residual block (MGRB) was developed to capture the high-level semantic information and low-level edge texture information of the image, enhancing the representation of fine-grained features. Then, the complementary feature between infrared and visible images was defined, and a cross-transfer attention fusion block (CAFB) was devised with joint spatial attention and channel attention to refine the critical supplemental information. This allowed the network to obtain fused features that were rich in both common and complementary information, thus realizing feature interaction and pre-fusion. Lastly, the features were reconstructed to obtain the fused image. Extensive experiments on three benchmark datasets demonstrated that the MEEAFusion proposed in this research has considerable strengths in terms of rich texture details, significant infrared targets, and distinct edge contours, and it achieves superior fusion performance.

RFSC-net: Re-parameterization forward semantic compensation network in low-light environments

Although detectors currently perform well in well-light conditions, their accuracy decreases due to insufficient object information. In addressing this issue, we propose the Re-parameterization Forward Semantic Compensation Network (RFSC-Net). We propose the Reparameterization Residual Efficient Layer Aggregation Networks (RSELAN) for feature extraction, which integrates the concepts of re-parameterization and the Efficient Layer Aggregation Networks (ELAN). While focusing on the fusion of feature maps of the same dimension, it also incorporates upward fusion of lower-level feature maps, enhancing the detailed texture information in higher-level features. Our proposed Forward Semantic Compensation Feature Fusion (FSCFF) network reduces interference from high-level to low-level semantic information, retaining finer details to improve detection accuracy in low-light conditions. Experiments on the low-light ExDark and DarkFace datasets show that RFSC-Net improves mAP by 2% on ExDark and 0.5% on DarkFace over the YOLOv8n baseline, without an increase in parameter counts. Additionally, AP50 is enhanced by 2.1% on ExDark and 1.1% on DarkFace, with a mere 3.7 ms detection latency on ExDark.

GAN-HA: A generative adversarial network with a novel heterogeneous dual-discriminator network and a new attention-based fusion strategy for infrared and visible image fusion

Infrared and visible image fusion (IVIF) aims to preserve thermal radiation information from infrared images while integrating texture details from visible images. Thermal radiation information is mainly expressed through image intensities, while texture details are typically expressed through image gradients. However, existing dual-discriminator generative adversarial networks (GANs) often rely on two structurally identical discriminators for learning, which do not fully account for the distinct learning needs of infrared and visible image information. To this end, this paper proposes a novel GAN with a heterogeneous dual-discriminator network and an attention-based fusion strategy (GAN-HA). Specifically, recognizing the intrinsic differences between infrared and visible images, we propose, for the first time, a novel heterogeneous dual-discriminator network to simultaneously capture thermal radiation information and texture details. The two discriminators in this network are structurally different, including a salient discriminator for infrared images and a detailed discriminator for visible images. They are able to learn rich image intensity information and image gradient information, respectively. In addition, a new attention-based fusion strategy is designed in the generator to appropriately emphasize the learned information from different source images, thereby improving the information representation ability of the fusion result. In this way, the fused images generated by GAN-HA can more effectively maintain both the salience of thermal targets and the sharpness of textures. Extensive experiments on various public datasets demonstrate the superiority of GAN-HA over other state-of-the-art (SOTA) algorithms while showcasing its higher potential for practical applications.

CT and MRI Image Fusion via Coupled Feature-Learning GAN

The fusion of multimodal medical images, particularly CT and MRI, is driven by the need to enhance the diagnostic process by providing clinicians with a single, comprehensive image that encapsulates all necessary details. Existing fusion methods often exhibit a bias towards features from one of the source images, making it challenging to simultaneously preserve both structural information and textural details. Designing an effective fusion method that can preserve more discriminative information is therefore crucial. In this work, we propose a Coupled Feature-Learning GAN (CFGAN) to fuse the multimodal medical images into a single informative image. The proposed method establishes an adversarial game between the discriminators and a couple of generators. First, the coupled generators are trained to generate two real-like fused images, which are then used to deceive the two coupled discriminators. Subsequently, the two discriminators are devised to minimize the structural distance to ensure the abundant information in the original source images is well-maintained in the fused image. We further empower the generators to be robust under various scales by constructing a discriminative feature extraction (DFE) block with different dilation rates. Moreover, we introduce a cross-dimension interaction attention (CIA) block to refine the feature representations. The qualitative and quantitative experiments on common benchmarks demonstrate the competitive performance of the CFGAN compared to other state-of-the-art methods.

A Texture-Considerate Convolutional Neural Network Approach for Color Consistency in Remote Sensing Imagery

Remote sensing allows us to conduct large-scale scientific studies that require extensive mapping and the amalgamation of numerous images. However, owing to variations in radiation, atmospheric conditions, sensor perspectives, and land cover, significant color discrepancies often arise between different images, necessitating color consistency adjustments for effective image mosaicking and applications. Existing methods for color consistency adjustment in remote sensing images struggle with complex one-to-many nonlinear color-mapping relationships, often resulting in texture distortions. To address these challenges, this study proposes a convolutional neural network-based color consistency method for remote sensing cartography that considers both global and local color mapping and texture mapping constrained by the source domain. This method effectively handles complex color-mapping relationships while minimizing texture distortions in the target image. Comparative experiments on remote sensing images from different times, sensors, and resolutions demonstrated that our method achieved superior color consistency, preserved fine texture details, and provided visually appealing outcomes, assisting in generating large-area data products.

Double-Exposure Algorithm: A Powerful Approach to Address the Accuracy Issues of Fractional Vegetation Extraction under Shadow Conditions

When recording the vegetation distribution with a camera, shadows can form due to factors like camera angle and direct sunlight. These shadows result in the loss of pixel information and texture details, significantly reducing the accuracy of fractional vegetation coverage (FVC) extraction. To address this issue, this study proposes an efficient double-exposure algorithm. The method reconstructs the pixel information in shadow areas by fusing normal-exposure and overexposed images. This approach overcomes the limitations of the camera’s dynamic range in capturing pixel information in shadowed regions. The study evaluates images with five levels of overexposure combined with five vegetation extraction indices. The aim is to determine the best-performing double-exposure combination under shadow conditions and the most suitable vegetation index. Experimental results reveal that the R² value between the best vegetation index and the FVC calculated from the fused double-exposure images and the ground truth FVC increases from 0.750 to 0.969. The root mean square error (RMSE) reduces from 0.146 to 0.046, and the intersection over union (IOU) increases from 0.856 to 0.943. These results demonstrate the excellent vegetation extraction capability of the double-exposure algorithm under shadow conditions, offering a straightforward and effective solution to low accuracy of FVC in shadowed areas.

A novel pansharpening model based on two parallel network architectures

ABSTRACT Pansharpening is an important technology for obtaining high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) images and high-resolution panchromatic (PAN) images. Although many pansharpening models have emerged by taking advantage of deep learning (DL) technology, there remains a pressing need to further assess pansharpening accuracy and stability when LRMS images with complex land-cover types. What’s more, these models often overlook the exploitation of PAN images’ inherent high-frequency information. To address these issues, we propose a pansharpening model combining multi-level and multi-scale network architectures. The multi-level network architecture is used to build spatial-spectral dependence on LRMS-PAN pairs, and strengthen the network’s feature capture capability by keeping the multi-level texture details. The multi-scale architecture is subsequently used to extract the spatial structure and deep texture of the PAN images at different scales. Downsampled experiments and real experiments in four standard datasets show that the proposed model achieves a state-of-the-art performance.

3D Reconstruction of Indoor Scenes Based on 3DGS Models

Abstract Abstract With the rapid development of computer vision and artificial intelligence technologies, indoor scene reconstruction has been more and more widely used in the fields of virtual reality, augmented reality and architectural design. In this paper, we study an indoor scene reconstruction method based on the 3DGS model, which has been widely used in computer graphics and vision processing with powerful scene representation and rendering capabilities. In this study, we optimize the 3DGS model to enhance the detail preservation and realism of the reconstruction results by adjusting the opacity of the Gaussian function. We used the Replica dataset and the self-harvested dataset for model training. Through experimental validation, the peak signal-to-noise ratio as well as the structural similarity ratio of the reconstruction results of the optimized model have an improvement effect of more than 1%, which indicates that the optimized model has a significant improvement in detail retention and realism, and the reconstructed scene performs more realistically in terms of texture details and light and shadow effects.

An underwater image enhancement method based on multi-scale layer decomposition and fusion

High-quality underwater images can intuitively reflect the most realistic underwater conditions, guiding for underwater environmental monitoring and resource exploration strongly. But when factors like light absorption affect underwater optical imaging, it has been found that poor visibility and blurred texture details occur in acquired images, posing challenges for the identification and detection of underwater targets. To obtain natural images, an enhancement algorithm is proposed based on multi-scale layer decomposition and fusion. The algorithm employs different strategies to recover image attenuation information from both local and global perspectives, generating two complementary preprocessed fusion inputs. For fusion input 1, operations are conducted in the RGB color space. Initially, the mean proportion of each color channel is used to identify the attenuated color channel. Then, a local compensation strategy is adaptively applied to restore the pixel intensity of the attenuated color channel. Finally, a statistical color correction method is used to eliminate color cast in the image. Fusion input 2 involves two processing stages. In the Lab color space, the algorithm uses the grayscale information to reduce the deviation in the mean values of channels a and b globally. The local mean information of the component L enhances detail textures. In the RGB color space, linear stretching is applied to correct color deviations. To fuse the structural features of two complementary preprocessed inputs and avoid interference between signals from different layers, the color channels of fused input image are first decomposed into muti-scale structural layers based on structural priors. Then, the image enhancement is achieved through layer-by-layer fusion of the corresponding color channels of the two inputs. By testing and analyzing with the, it was found that the proposed method can improve the clarity of attenuated images in various underwater scenarios of UIEBD and RUIE datasets effectively, enhancing image detail and texture richness, increases contrast, and achieving natural and comfortable visual quality. Compared with the quantitative metrics of 14 other algorithms, the proposed algorithm shows an average score improvement of 10.14, 90.48, and 2.06, respectively, in metrics AG (average gradient), EI (edge intensity), and NIQE (natural image quality evaluator). In the RUIE dataset, it shows an average score improvement of 10.21, 94.76, and 1.86, respectively.

SeACPFusion: An Adaptive Fusion Network for Infrared and Visible Images based on brightness perception

Generating a single fused image that highlights important targets and preserves textural details is the aim of fusing visible and infrared images. The majority of deep learning-based fusion algorithms now in use can produce decent fusion outcomes; however, the modeling process still lacks consideration of the different amounts of information in different scenes or regions. Thus, we propose in this research SeACPFusion, a luminance-aware adaptive fusion network for infrared and visible images, which adaptively preserves the intensity information of the noticeable targets of the source images with the texture information of the background in an optimal ratio. Specifically, we design pixel-level luminance loss (PBL) to direct the fusion model’s training in real-time, and PBL retains the optimal intensity information according to the pixel luminance ratio of different source images. In addition, we designed the Channel Transformer (CTF) to consider the relationship between different attributes from the point of view of the feature channel and to focus on the key information by using the self-focusing mechanism to achieve the goal of adaptive fusion. Our extensive tests on the MSRS, RoadScene, and TNO datasets demonstrate that SeACPFusion surpasses nine representative deep learning methods on six objective metrics and achieves the best visual results in scenes such as overexposure or underexposure. In addition, the relatively efficient operation and fewer model parameters make our algorithm promising as a preprocessing module for downstream complicated vision tasks.

FCLFusion: A frequency-aware and collaborative learning for infrared and visible image fusion

Infrared and visible image fusion (IVIF) aims to integrate the advantages of different modal images. Most existing deep learning-based methods often focus on a single IVIF task and ignore the effect of frequency information on the fusion results, which do not fully preserve saliency structures and important texture details. The core idea of this paper is based on the following observation: (1) image content can be characterized by different frequency domain components, low frequency represents base information, such as saliency structure, while high frequency contains texture detail information. (2) multi-tasks learning generally achieve better performance than single-task. Based on these observations, we propose a fusion model called Frequency-aware and Collaborative Learning (FCLFusion) for infrared and visible images. This model takes image fusion as the main task and introduces image reconstruction as an auxiliary task to collaboratively optimize the network, thereby improving the fusion quality. Specifically, we transform spatial domain features to the frequency domain and develop a frequency feature fusion module for guiding the primary network to generate the fused image. The sub-network generates the reconstructed images. Also, we preserve the saliency and detail features via frequency skip connections. Moreover, we propose a hybrid loss function that consists of two terms: frequency loss and self-supervised reconstruction loss. The former aims to prevent information loss in the frequency domain, while the latter improves the extraction of vital information. Extensive experiments verified on three public datasets demonstrate that our FCLFusion outperforms ten state-of-the-art fusion models.

Infrared and visible image fusion based on semi-global weighted least squares and guided edge-aware filters

Most of the existing image fusion methods are dedicated to extracting the respective private features in the source image to reconstruct to get a fused image that contains rich information. However, its neglect of the enhancement of texture details in the source image as well as the preservation of background structures, which makes the fusion result look less impressive. To this end, an infrared and visible image fusion method based on a semi-global weighted least squares (SGWLS) method and guided edge-aware filter (GEAF) is proposed. First, a SGWLS method is applied to decompose the source image into base and detail layers containing different scale information. Then, the detail layer are texture-enhanced by using the scale coefficients. Due to the lack of edge features in the original base layer, for this reason, a guided edge-aware filter is designed, in which the enhanced detail layer was used as the guidance image as a way to perform a quadratic feature enhancement fusion for the base layer in order to retain richer gradient information. Finally, reconstruction of the fused detail and base layers by using inverse transformation to obtain the fusion result. Compared with nine state-of-the-art fusion algorithms, a large number of experimental results demonstrate the superior performance of the proposed method in highlighting texture details and background features, as well as the obvious advantages in SF and AG metrics.

JOA‐GAN: An improved single‐image super‐resolution network for remote sensing based on GAN

AbstractImage super‐resolution (SR) has been widely applied in remote sensing to generate high‐resolution (HR) images without increasing hardware costs. However, SR is a severe ill‐posed problem. As deep learning advances, existing methods have solved this problem to a certain extent. However, the complex spatial distribution of remote sensing images still poses a challenge in effectively extracting abundant high‐frequency details from the images. Here, a single‐image super‐resolution (SISR) network based on the generative adversarial network (GAN) for remote sensing is presented, called JOA‐GAN. Firstly, a joint‐attention module (JOA) is proposed to focus the network on high‐frequency regions in remote sensing images to enhance the quality of image reconstruction. In the generator network, a multi‐scale densely connected feature extraction block (ERRDB) is proposed, which acquires features at different scales using MSconv blocks containing multi‐scale convolutions and automatically adjusts the features by JOA. In the discriminator network, the relative discriminator is used to compute the relative probability instead of the absolute probability, which helps the network learn clearer and more realistic texture details. JOA‐GAN is compared with other advanced methods, and the results demonstrate that JOA‐GAN has improved objective evaluation metrics and achieved superior visual effects.

WaveFusionNet: Infrared and visible image fusion based on multi-scale feature encoder–decoder and discrete wavelet decomposition

To merge complementary information from multimodal images, such as thermal saliency from infrared images and texture details from visible images, traditional multi-scale transform-based methods have been extensively studied, with deep learning-based methods gaining significant popularity in recent years. However, there has been limited research on optimally combining the advantages of these two categories in fusion. In this paper, we propose a novel infrared and visible image fusion (IVIF) framework, WaveFusionNet, which integrates precise frequency feature decomposition from the discrete wavelet transform (DWT) with the comprehensive feature extraction from the multi-scale encoder. Firstly, we train an encoder–decoder network for multi-scale feature extraction and image reconstruction. DWT is used for down-sampling with minimal information loss by decomposing extracted features into low and high-frequency sub-bands. Next, a dual-band feature fusion (DBFF) module is trained to merge these sub-bands by integrating a spatial feature transform-based sub-network for low-frequency fusion and a maximum absolute value selection strategy for fusing high-frequencies. Finally, all fused sub-bands are fed into the pre-trained decoder to reconstruct the final image. Experimental results on three benchmark datasets (TNO, Roadscene, and MSRS) demonstrate that the proposed fusion method outperforms recent IVIF methods in both quantitative assessment and visual perception while maintaining competitive time complexity.